[Python-modules-commits] [python-boltons] 01/03: New upstream version 17.1.0
Hugo Lefeuvre
hle at moszumanska.debian.org
Sat Sep 23 16:15:23 UTC 2017
This is an automated email from the git hooks/post-receive script.
hle pushed a commit to branch master
in repository python-boltons.
commit 1e2935c1b2a674f2df5a4ea6a517de6a73d73277
Author: Hugo Lefeuvre <hle at debian.org>
Date: Sat Sep 23 18:06:53 2017 +0200
New upstream version 17.1.0
---
.gitignore | 3 +
.travis.yml | 21 +-
CHANGELOG.md | 91 +++-
README.md | 5 +
TODO.rst | 1 +
boltons/cacheutils.py | 71 ++-
boltons/debugutils.py | 2 +-
boltons/ecoutils.py | 14 +-
boltons/fileutils.py | 2 +-
boltons/funcutils.py | 102 +++-
boltons/ioutils.py | 404 +++++++++++++++
boltons/iterutils.py | 188 +++++--
boltons/mathutils.py | 30 ++
boltons/setutils.py | 2 +-
boltons/statsutils.py | 16 +-
boltons/tableutils.py | 12 +-
boltons/tbutils.py | 4 +-
boltons/txurl_notes.md | 60 +++
boltons/typeutils.py | 3 +
boltons/urlutils.py | 1230 ++++++++++++++++++++++++++++++++++++++++++++
docs/conf.py | 6 +-
docs/index.rst | 12 +-
docs/ioutils.rst | 83 +++
docs/mathutils.rst | 7 +-
requirements-test.txt | 10 +-
setup.py | 31 +-
tests/.coveragerc | 9 +
tests/test_cacheutils.py | 27 +-
tests/test_funcutils.py | 22 +
tests/test_funcutils_fb.py | 55 ++
tests/test_ioutils.py | 369 +++++++++++++
tests/test_iterutils.py | 49 ++
tests/test_tbutils.py | 7 +-
tests/test_typeutils.py | 7 +
tests/test_urlutils.py | 440 ++++++++++++++++
35 files changed, 3297 insertions(+), 98 deletions(-)
diff --git a/.gitignore b/.gitignore
index 97f7d7f..604a8e8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
docs/_build
tmp.py
+htmlcov/
*.py[cod]
@@ -45,3 +46,5 @@ nosetests.xml
# Vim
*.sw[op]
+
+.cache/
diff --git a/.travis.yml b/.travis.yml
index d15f6a8..0a0a072 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,12 +1,31 @@
language: python
sudo: false
cache: pip
+
+# Python targets, as defined by https://github.com/travis-ci/travis-build/blob
+# /master/spec/build/script/python_spec.rb and https://github.com/travis-ci
+# /travis-build/blob/master/lib/travis/build/script/python.rb
python:
+ # Standard release https://docs.travis-ci.com/user/languages
+ # /python#choosing-python-versions-to-test-against
- "2.6"
- "2.7"
+ - "3.3"
- "3.4"
- "3.5"
- - "pypy"
+ - "3.6"
+ - "3.7-dev" # a.k.a "nightly"
+
+ # PyPy2.7: https://doc.pypy.org/en/latest
+ # /index-of-release-notes.html#cpython-2-7-compatible-versions
+ - pypy # PyPy2.7 2.5.0
+ - pypy-5.1
+ - pypy-5.3
+ - pypy-5.4
+
+ # PyPy3.3: https://doc.pypy.org/en/latest
+ # /index-of-release-notes.html#cpython-3-3-compatible-versions
+ - pypy3.3-5.2-alpha1
install: "pip install -r requirements-test.txt"
script: "py.test --doctest-modules boltons tests"
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 20e6913..be83250 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,8 +1,77 @@
boltons Changelog
=================
-Since February 20, 2013 there have been 28 releases and 950 commits for
-an average of one 34-commit release every 6.4 weeks.
+Since February 20, 2013 there have been 31 releases and 1043 commits for
+an average of one 33-commit release every 6.5 weeks.
+
+17.0.0
+------
+*(January 24, 2017)*
+
+Several tweaks and enhancements to ring in the new year.
+
+* [tbutils][tbutils] objects like the
+ [ExceptionInfo][tbutils.ExceptionInfo] are now more easily
+ JSON-serializable thanks to a tweak to [Callpoint][tbutils.Callpoint].
+* SpooledIO objects like
+ [ioutils.SpooledBytesIO][ioutils.SpooledBytesIO] are now
+ `bool`-able.
+* [iterutils.bucketize][iterutils.bucketize] gains the
+ `value_transform` and `key_filter` arguments.
+* [cachedproperty][cacheutils.cachedproperty] properly maintains
+ docstring
+* [funcutils.wraps][funcutils.wraps] maintains a reference to the
+ wrapped function with `__wrapped__` attribute.
+* A bit of cleanup to be backwards compatible to Python 3.3
+
+16.5.1
+------
+*(November 6, 2016)*
+
+Mostly bug fixes and various tweaks, optimizations, and
+documentation. Also added a bit of functionality in the form of
+[ioutils][ioutils] and some GUID stuff.
+
+* Add [ioutils][ioutils] with
+ [SpooledStringIO][ioutils.SpooledStringIO] and
+ [SpooledBytesIO][ioutils.SpooledBytesIO], two in-memory file-like
+ objects, like the stdlib [StringIO][StringIO], except that they
+ automatically spill over to disk when they reach a configurable
+ size.
+* Add [iterutils.GUIDerator][iterutils.GUIDerator] and
+ [iterutils.SequentialGUIDerator][iterutils.SequentialGUIDerator],
+ two methods of getting random iterables.
+* Add [mathutils.clamp][mathutils.clamp], a combined min-max function,
+ like numpy's clip.
+* Optimized [iterutils.first][iterutils.first].
+* Enabled spillover kwargs in [funcutils.wraps][funcutils.wraps]
+* fix for default [remap][iterutils.remap] set support, fixes [#84][i84]
+* improving and testing exceptions around classmethod and staticmethod
+ for [funcutils.wraps][funcutils.wraps] and
+ [FunctionBuilder][funcutils.FunctionBuilder], fixes [#86][i86] to
+ the degree possible.
+
+
+16.5.0
+------
+*(July 16, 2016)*
+
+A few minor changes, and medium-sized breaking change to
+[cacheutils][cacheutils].
+
+* [cacheutils][cacheutils] caching decorators now take the
+ function/method into account by default. This was done by adding the
+ scoped argument to [@cached][cacheutils.cached] and
+ [@cachedmethod][cacheutils.cachedmethod] (and removing selfish from
+ cachedmethod). also fixed a bug in a cachedmethod test, as well as
+ added docs for scoped and key arguments. all of this to fix [#83][i83].
+* [tableutils.Table][tableutils.Table] cell html can be customized by
+ overriding `get_cell_html` method.
+* [funcutils.total_ordering][funcutils.total_ordering], a
+ [functools.total_ordering][functools.total_ordering] backport for
+ python 2.6.
+* [funcutils.FunctionBuilder][funcutils.FunctionBuilder] function
+ names are now configurable.
16.4.1
------
@@ -522,7 +591,7 @@ added in this release.
* fix a slightly nasty namedlist bug
* make OrderedMultiDict.get()'s default allow singulars
* sync over ExceptionInfo
- * add from_current() classmethod with depth option to Callpoint class
+ * add from_current() classmethod with depth option to [Callpoint][tbutils.Callpoint] class
for easier instantiation
* it's called a numeronym
* add a repr to ParsedTB. A bit verbose, but better than nothing.
@@ -530,7 +599,7 @@ added in this release.
out to logs, the command line, etc.
* improve test output and make assertion that new except hook is the
same as the builtin.
- * update tbutils to use the more-powerful Callpoint type.
+ * update tbutils to use the more-powerful [Callpoint][tbutils.Callpoint] type.
* copy_function
* partially clean up partial stuff
* first version of the namedlist
@@ -666,6 +735,8 @@ added in this release.
[os.replace]: https://docs.python.org/3/library/os.html#os.replace
+[functools.total_ordering]: https://docs.python.org/2/library/functools.html#functools.total_ordering
+[StringIO]: https://docs.python.org/2/library/stringio.html
[cacheutils]: http://boltons.readthedocs.org/en/latest/cacheutils.html
[cacheutils.LRU]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.LRU
@@ -693,6 +764,7 @@ added in this release.
[fileutils.DummyFile]: http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.DummyFile
[funcutils.FunctionBuilder]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.FunctionBuilder
[funcutils.partial_ordering]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.partial_ordering
+[funcutils.total_ordering]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.total_ordering
[funcutils.wraps]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.wraps
[gcutils.GCToggler]: http://boltons.readthedocs.org/en/latest/gcutils.html#boltons.gcutils.GCToggler
[gcutils.get_all]: http://boltons.readthedocs.org/en/latest/gcutils.html#boltons.gcutils.get_all
@@ -705,6 +777,12 @@ added in this release.
[i30]: https://github.com/mahmoud/boltons/issues/30
[i41]: https://github.com/mahmoud/boltons/issues/41
[i79]: https://github.com/mahmoud/boltons/pull/79
+[i83]: https://github.com/mahmoud/boltons/issues/83
+[i84]: https://github.com/mahmoud/boltons/issues/84
+[i86]: https://github.com/mahmoud/boltons/issues/86
+[ioutils]: http://boltons.readthedocs.org/en/latest/ioutils.html
+[ioutils.SpooledBytesIO]: http://boltons.readthedocs.org/en/latest/ioutils.html#boltons.ioutils.SpooledBytesIO
+[ioutils.SpooledStringIO]: http://boltons.readthedocs.org/en/latest/ioutils.html#boltons.ioutils.SpooledStringIO
[iterutils]: http://boltons.readthedocs.org/en/latest/iterutils.html
[iterutils.backoff]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.backoff
[iterutils.backoff_iter]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.backoff_iter
@@ -713,7 +791,10 @@ added in this release.
[iterutils.first]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.first
[iterutils.backoff]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.backoff
[iterutils.frange]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.frange
+[iterutils.GUIDerator]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.GUIDerator
+[iterutils.SequentialGUIDerator]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.SequentialGUIDerator
[iterutils.is_container]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.is_container
+[iterutils.bucketize]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.bucketize
[iterutils.one]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.one
[iterutils.pairwise]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.pairwise
[iterutils.same]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.same
@@ -726,6 +807,7 @@ added in this release.
[jsonutils.JSONLIterator]: http://boltons.readthedocs.org/en/latest/jsonutils.html#boltons.jsonutils.JSONLIterator
[mathutils.ceil]: http://boltons.readthedocs.org/en/latest/mathutils.html#boltons.mathutils.ceil
[mathutils.floor]: http://boltons.readthedocs.org/en/latest/mathutils.html#boltons.mathutils.floor
+[mathutils.clamp]: http://boltons.readthedocs.org/en/latest/mathutils.html#boltons.mathutils.clamp
[socketutils]: http://boltons.readthedocs.org/en/latest/socketutils.html
[socketutils.BufferedSocket]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.BufferedSocket
[socketutils.BufferedSocket.recv]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.BufferedSocket.recv
@@ -759,6 +841,7 @@ added in this release.
[tableutils]: http://boltons.readthedocs.org/en/latest/tableutils.html
[tableutils.Table]: http://boltons.readthedocs.org/en/latest/tableutils.html#boltons.tableutils.Table
[tbutils]: http://boltons.readthedocs.org/en/latest/tbutils.html
+[tbutils.Callpoint]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.Callpoint
[tbutils.ExceptionInfo]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ExceptionInfo
[tbutils.ParsedException]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ParsedException
[tbutils.ParsedException.to_string]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ParsedException.to_string
diff --git a/README.md b/README.md
index 199e76d..bbde090 100644
--- a/README.md
+++ b/README.md
@@ -2,6 +2,10 @@
*boltons should be builtins.*
+<a href="https://boltons.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat"></a>
+<a href="https://pypi.python.org/pypi/boltons"><img src="https://img.shields.io/pypi/v/boltons.svg"></a>
+<a href="http://calver.org"><img src="https://img.shields.io/badge/calver-YY.MINOR.MICRO-22bfda.svg"></a>
+
**Boltons** is a set of over 160 BSD-licensed, pure-Python utilities
in the same spirit as — and yet conspicuously missing from —
[the standard library][stdlib], including:
@@ -32,6 +36,7 @@ Boltons is tested against Python 2.6, 2.7, 3.4, 3.5, and PyPy.
[tbinfo]: https://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.TracebackInfo
[fileutils]: https://boltons.readthedocs.org/en/latest/fileutils.html#module-boltons.fileutils
+[ioutils]: https://boltons.readthedocs.org/en/latest/ioutils.html#module-boltons.ioutils
[dictutils]: https://boltons.readthedocs.org/en/latest/dictutils.html#module-boltons.dictutils
[queueutils]: https://boltons.readthedocs.org/en/latest/queueutils.html#module-boltons.queueutils
[iterutils]: https://boltons.readthedocs.org/en/latest/iterutils.html#module-boltons.iterutils
diff --git a/TODO.rst b/TODO.rst
index 037679c..1f5cdb1 100644
--- a/TODO.rst
+++ b/TODO.rst
@@ -3,6 +3,7 @@ TODO
- dummy context manager
- dummy file
+- urlutils
cacheutils
----------
diff --git a/boltons/cacheutils.py b/boltons/cacheutils.py
index 0b01104..d8f3620 100644
--- a/boltons/cacheutils.py
+++ b/boltons/cacheutils.py
@@ -425,7 +425,8 @@ class _HashedKey(list):
return '%s(%s)' % (self.__class__.__name__, list.__repr__(self))
-def make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
+def make_cache_key(args, kwargs, typed=False,
+ kwarg_mark=_KWARG_MARK,
fasttypes=frozenset([int, str, frozenset, type(None)])):
"""Make a generic key from a function's positional and keyword
arguments, suitable for use in caches. Arguments within *args* and
@@ -444,6 +445,9 @@ def make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
.. _hashable: https://docs.python.org/2/glossary.html#term-hashable
"""
+
+ # key = [func_name] if func_name else []
+ # key.extend(args)
key = list(args)
if kwargs:
sorted_items = sorted(kwargs.items())
@@ -465,7 +469,7 @@ class CachedFunction(object):
"""This type is used by :func:`cached`, below. Instances of this
class are used to wrap functions in caching logic.
"""
- def __init__(self, func, cache, typed=False):
+ def __init__(self, func, cache, scoped=True, typed=False, key=None):
self.func = func
if callable(cache):
self.get_cache = cache
@@ -478,11 +482,13 @@ class CachedFunction(object):
def _get_cache():
return cache
self.get_cache = _get_cache
+ self.scoped = scoped
self.typed = typed
+ self.key_func = key or make_cache_key
def __call__(self, *args, **kwargs):
cache = self.get_cache()
- key = make_cache_key(args, kwargs, typed=self.typed)
+ key = self.key_func(args, kwargs, typed=self.typed)
try:
ret = cache[key]
except KeyError:
@@ -491,8 +497,9 @@ class CachedFunction(object):
def __repr__(self):
cn = self.__class__.__name__
- if self.typed:
- return "%s(func=%r, typed=%r)" % (cn, self.func, self.typed)
+ if self.typed or not self.scoped:
+ return ("%s(func=%r, scoped=%r, typed=%r)"
+ % (cn, self.func, self.scoped, self.typed))
return "%s(func=%r)" % (cn, self.func)
@@ -500,7 +507,7 @@ class CachedMethod(object):
"""Similar to :class:`CachedFunction`, this type is used by
:func:`cachedmethod` to wrap methods in caching logic.
"""
- def __init__(self, func, cache, typed=False, selfish=True):
+ def __init__(self, func, cache, scoped=True, typed=False, key=None):
self.func = func
if isinstance(cache, basestring):
self.get_cache = attrgetter(cache)
@@ -510,30 +517,30 @@ class CachedMethod(object):
and callable(getattr(cache, '__setitem__', None))):
raise TypeError('expected cache to be an attribute name,'
' dict-like object, or callable returning'
- ' a dict-like object, not %r'
- % cache)
+ ' a dict-like object, not %r' % cache)
else:
def _get_cache(obj):
return cache
self.get_cache = _get_cache
+ self.scoped = scoped
self.typed = typed
- self.selfish = selfish
+ self.key_func = key or make_cache_key
self.bound_to = None
def __get__(self, obj, objtype=None):
if obj is None:
return self
cls = self.__class__
- ret = cls(self.func, self.get_cache,
- typed=self.typed, selfish=self.selfish)
+ ret = cls(self.func, self.get_cache, typed=self.typed,
+ scoped=self.scoped, key=self.key_func)
ret.bound_to = obj
return ret
def __call__(self, *args, **kwargs):
obj = args[0] if self.bound_to is None else self.bound_to
cache = self.get_cache(obj)
- key_args = (self.bound_to,) + args if self.selfish else args
- key = make_cache_key(key_args, kwargs, typed=self.typed)
+ key_args = (self.bound_to, self.func) + args if self.scoped else args
+ key = self.key_func(key_args, kwargs, typed=self.typed)
try:
ret = cache[key]
except KeyError:
@@ -544,14 +551,14 @@ class CachedMethod(object):
def __repr__(self):
cn = self.__class__.__name__
- args = (cn, self.func, self.typed, self.selfish)
+ args = (cn, self.func, self.scoped, self.typed)
if self.bound_to is not None:
args += (self.bound_to,)
- return ('<%s func=%r typed=%r selfish=%r bound_to=%r>' % args)
- return ("%s(func=%r, typed=%r, selfish=%r)" % args)
+ return ('<%s func=%r scoped=%r typed=%r bound_to=%r>' % args)
+ return ("%s(func=%r, scoped=%r, typed=%r)" % args)
-def cached(cache, typed=False):
+def cached(cache, scoped=True, typed=False, key=None):
"""Cache any function with the cache object of your choosing. Note
that the function wrapped should take only `hashable`_ arguments.
@@ -561,6 +568,12 @@ def cached(cache, typed=False):
:class:`LRI` are good choices, but a plain :class:`dict`
can work in some cases, as well. This argument can also be
a callable which accepts no arguments and returns a mapping.
+ scoped (bool): Whether the function itself is part of the
+ cache key. ``True`` by default, different functions will
+ not read one another's cache entries, but can evict one
+ another's results. ``False`` can be useful for certain
+ shared cache use cases. More advanced behavior can be
+ produced through the *key* argument.
typed (bool): Whether to factor argument types into the cache
check. Default ``False``, setting to ``True`` causes the
cache keys for ``3`` and ``3.0`` to be considered unequal.
@@ -579,11 +592,11 @@ def cached(cache, typed=False):
"""
def cached_func_decorator(func):
- return CachedFunction(func, cache, typed=typed)
+ return CachedFunction(func, cache, scoped=scoped, typed=typed, key=key)
return cached_func_decorator
-def cachedmethod(cache, typed=False, selfish=True):
+def cachedmethod(cache, scoped=True, typed=False, key=None):
"""Similar to :func:`cached`, ``cachedmethod`` is used to cache
methods based on their arguments, using any :class:`dict`-like
*cache* object.
@@ -592,12 +605,18 @@ def cachedmethod(cache, typed=False, selfish=True):
cache (str/Mapping/callable): Can be the name of an attribute
on the instance, any Mapping/:class:`dict`-like object, or
a callable which returns a Mapping.
+ scoped (bool): Whether the method itself and the object it is
+ bound to are part of the cache keys. ``True`` by default,
+ different methods will not read one another's cache
+ results. ``False`` can be useful for certain shared cache
+ use cases. More advanced behavior can be produced through
+ the *key* arguments.
typed (bool): Whether to factor argument types into the cache
check. Default ``False``, setting to ``True`` causes the
cache keys for ``3`` and ``3.0`` to be considered unequal.
- selfish (bool): Whether an instance's ``self`` argument should
- be considered in the cache key. Use this to share cache
- objects across instances.
+ key (callable): A callable with a signature that matches
+ :func:`make_cache_key` that returns a tuple of hashable
+ values to be used as the key in the cache.
>>> class Lowerer(object):
... def __init__(self):
@@ -612,9 +631,10 @@ def cachedmethod(cache, typed=False, selfish=True):
'wow who could guess caching could be so neat'
>>> len(lowerer.cache)
1
+
"""
def cached_method_decorator(func):
- return CachedMethod(func, cache, typed=typed, selfish=selfish)
+ return CachedMethod(func, cache, scoped=scoped, typed=typed, key=key)
return cached_method_decorator
@@ -628,15 +648,14 @@ class cachedproperty(object):
allows the cache to be cleared with :func:`delattr`, or through
manipulating the object's ``__dict__``.
"""
-
def __init__(self, func):
+ self.__doc__ = getattr(func, '__doc__')
self.func = func
def __get__(self, obj, objtype=None):
if obj is None:
return self
- value = self.func(obj)
- setattr(obj, self.func.__name__, value)
+ value = obj.__dict__[self.func.__name__] = self.func(obj)
return value
def __repr__(self):
diff --git a/boltons/debugutils.py b/boltons/debugutils.py
index a8605ab..03f3c6c 100644
--- a/boltons/debugutils.py
+++ b/boltons/debugutils.py
@@ -21,7 +21,7 @@ try:
except ImportError:
_UNSET = object()
-__all__ = ['pdb_on_signal', 'pdb_on_exception']
+__all__ = ['pdb_on_signal', 'pdb_on_exception', 'wrap_trace']
def pdb_on_signal(signalnum=None):
diff --git a/boltons/ecoutils.py b/boltons/ecoutils.py
index f9c5a75..ae7b2d8 100644
--- a/boltons/ecoutils.py
+++ b/boltons/ecoutils.py
@@ -1,14 +1,16 @@
# -*- coding: utf-8 -*-
-"""The larger a programming ecosystem gets, the greater the chances of
-runtime variability become. Currently, Python is one of the most
-widely deployed high-level programming environments available, making
-it a viable target for all manner of application. But it's important
-to know what you're working with.
+"""As a programming ecosystem grows, so do the chances of runtime
+variability.
+
+Python boasts one of the widest deployments for a high-level
+programming environment, making it a viable target for all manner of
+application. But with breadth comes variance, so it's important to
+know what you're working with.
Some basic variations that are common among development machines:
* **Executable runtime**: CPython, PyPy, Jython, etc., plus build date and compiler
-* **Language version**: 2.4, 2.5, 2.6, 2.7... 3.3, 3.4, 3.5
+* **Language version**: 2.4, 2.5, 2.6, 2.7... 3.4, 3.5, 3.6
* **Host operating system**: Windows, OS X, Ubuntu, Debian, CentOS, RHEL, etc.
* **Features**: 64-bit, IPv6, Unicode character support (UCS-2/UCS-4)
* **Built-in library support**: OpenSSL, threading, SQLite, zlib
diff --git a/boltons/fileutils.py b/boltons/fileutils.py
index dab6afb..499086c 100644
--- a/boltons/fileutils.py
+++ b/boltons/fileutils.py
@@ -471,7 +471,7 @@ def iter_find_files(directory, patterns, ignored=None):
>>> filenames = sorted(iter_find_files(_CUR_DIR, '*.py'))
>>> os.path.basename(filenames[-1])
- 'typeutils.py'
+ 'urlutils.py'
Or, Python files while ignoring emacs lockfiles:
diff --git a/boltons/funcutils.py b/boltons/funcutils.py
index 5579d13..c3a11c6 100644
--- a/boltons/funcutils.py
+++ b/boltons/funcutils.py
@@ -58,7 +58,7 @@ def mro_items(type_obj):
"""Takes a type and returns an iterator over all class variables
throughout the type hierarchy (respecting the MRO).
- >>> sorted(set([k for k, v in mro_items(int) if not k.startswith('__') and not callable(v)]))
+ >>> sorted(set([k for k, v in mro_items(int) if not k.startswith('__') and 'bytes' not in k and not callable(v)]))
['denominator', 'imag', 'numerator', 'real']
"""
# TODO: handle slots?
@@ -262,30 +262,50 @@ def wraps(func, injected=None, **kw):
func (function): The callable whose attributes are to be copied.
injected (list): An optional list of argument names which
should not appear in the new wrapper's signature.
- update_dict(bool): Whether to copy other, non-standard
+ update_dict (bool): Whether to copy other, non-standard
attributes of *func* over to the wrapper. Defaults to True.
+ inject_to_varkw (bool): Ignore missing arguments when a
+ ``**kwargs``-type catch-all is present. Defaults to True.
For more in-depth wrapping of functions, see the
:class:`FunctionBuilder` type, on which wraps was built.
+
"""
+ # TODO: maybe automatically use normal wraps in the very rare case
+ # that the signatures actually match and no adapter is needed.
if injected is None:
injected = []
elif isinstance(injected, basestring):
injected = [injected]
+ else:
+ injected = list(injected)
+
+ if isinstance(func, (classmethod, staticmethod)):
+ raise TypeError('wraps does not support wrapping classmethods and'
+ ' staticmethods, change the order of wrapping to'
+ ' wrap the underlying function: %r'
+ % (getattr(func, '__func__', None),))
update_dict = kw.pop('update_dict', True)
+ inject_to_varkw = kw.pop('inject_to_varkw', True)
if kw:
raise TypeError('unexpected kwargs: %r' % kw.keys())
fb = FunctionBuilder.from_func(func)
for arg in injected:
- fb.remove_arg(arg)
+ try:
+ fb.remove_arg(arg)
+ except MissingArgument:
+ if inject_to_varkw and fb.varkw is not None:
+ continue # keyword arg will be caught by the varkw
+ raise
fb.body = 'return _call(%s)' % fb.get_invocation_str()
def wrapper_wrapper(wrapper_func):
execdict = dict(_call=wrapper_func, _func=func)
fully_wrapped = fb.get_func(execdict, with_dict=update_dict)
+ fully_wrapped.__wrapped__ = func # ref to the original function (#115)
return fully_wrapped
@@ -341,6 +361,8 @@ class FunctionBuilder(object):
but only for the *kwonlyargs*. **Python 3 only.**
annotations (dict): Mapping of type hints and so
forth. **Python 3 only.**
+ filename (str): The filename that will appear in
+ tracebacks. Defaults to "boltons.funcutils.FunctionBuilder".
indent (int): Number of spaces with which to indent the
function *body*. Values less than 1 will result in an error.
dict (dict): Any other attributes which should be added to the
@@ -386,7 +408,8 @@ class FunctionBuilder(object):
'dict': dict,
'module': lambda: None,
'body': lambda: 'pass',
- 'indent': lambda: 4}
+ 'indent': lambda: 4,
+ 'filename': lambda: 'boltons.funcutils.FunctionBuilder'}
_defaults.update(_argspec_defaults)
@@ -458,6 +481,9 @@ class FunctionBuilder(object):
"""
# TODO: copy_body? gonna need a good signature regex.
# TODO: might worry about __closure__?
+ if not callable(func):
+ raise TypeError('expected callable object, not %r' % (func,))
+
kwargs = {'name': func.__name__,
'doc': func.__doc__,
'module': func.__module__,
@@ -539,15 +565,18 @@ class FunctionBuilder(object):
try:
self.args.remove(arg_name)
except ValueError:
- raise ValueError('arg %r not found in %s argument list: %r'
- % (arg_name, self.name, self.args))
+ exc = MissingArgument('arg %r not found in %s argument list: %r'
+ % (arg_name, self.name, self.args))
+ exc.arg_name = arg_name
+ raise exc
d_dict.pop(arg_name, None)
self.defaults = tuple([d_dict[a] for a in self.args if a in d_dict])
return
def _compile(self, src, execdict):
- filename = ('<boltons.funcutils.FunctionBuilder-%d>'
- % (next(self._compile_count),))
+
+ filename = ('<%s-%d>'
+ % (self.filename, next(self._compile_count),))
try:
code = compile(src, filename, 'single')
exec(code, execdict)
@@ -556,6 +585,10 @@ class FunctionBuilder(object):
return execdict
+class MissingArgument(ValueError):
+ pass
+
+
def _indent(text, margin, newline='\n', key=bool):
"based on boltons.strutils.indent"
indented_lines = [(margin + line if key(line) else line)
@@ -563,4 +596,57 @@ def _indent(text, margin, newline='\n', key=bool):
return newline.join(indented_lines)
+try:
+ from functools import total_ordering # 2.7+
+except ImportError:
+ # python 2.6
+ def total_ordering(cls):
+ """Class decorator that fills in missing comparators/ordering
+ methods. Backport of :func:`functools.total_ordering` to work
+ with Python 2.6.
+
+ Code from http://code.activestate.com/recipes/576685/
+ """
+ convert = {
+ '__lt__': [
+ ('__gt__',
+ lambda self, other: not (self < other or self == other)),
+ ('__le__',
+ lambda self, other: self < other or self == other),
+ ('__ge__',
+ lambda self, other: not self < other)],
+ '__le__': [
+ ('__ge__',
+ lambda self, other: not self <= other or self == other),
+ ('__lt__',
+ lambda self, other: self <= other and not self == other),
+ ('__gt__',
+ lambda self, other: not self <= other)],
+ '__gt__': [
+ ('__lt__',
+ lambda self, other: not (self > other or self == other)),
+ ('__ge__',
+ lambda self, other: self > other or self == other),
+ ('__le__',
+ lambda self, other: not self > other)],
+ '__ge__': [
+ ('__le__',
+ lambda self, other: (not self >= other) or self == other),
+ ('__gt__',
+ lambda self, other: self >= other and not self == other),
+ ('__lt__',
+ lambda self, other: not self >= other)]
+ }
+ roots = set(dir(cls)) & set(convert)
+ if not roots:
+ raise ValueError('must define at least one ordering operation:'
+ ' < > <= >=')
+ root = max(roots) # prefer __lt__ to __le__ to __gt__ to __ge__
+ for opname, opfunc in convert[root]:
+ if opname not in roots:
+ opfunc.__name__ = opname
+ opfunc.__doc__ = getattr(int, opname).__doc__
+ setattr(cls, opname, opfunc)
+ return cls
+
# end funcutils.py
diff --git a/boltons/ioutils.py b/boltons/ioutils.py
new file mode 100644
index 0000000..b32e5c8
--- /dev/null
+++ b/boltons/ioutils.py
@@ -0,0 +1,404 @@
+# -*- coding: UTF-8 -*-
+
+# Coding decl above needed for rendering the emdash properly in the
+# documentation.
+
+"""
+Module ``ioutils`` implements a number of helper classes and functions which
+are useful when dealing with input, output, and bytestreams in a variety of
+ways.
+"""
+import os
+import sys
+from abc import (
+ ABCMeta,
+ abstractmethod,
+ abstractproperty,
+)
+from errno import EINVAL
+from io import BytesIO
+from codecs import EncodedFile
+from tempfile import TemporaryFile
+
+# Python2/3 compat
+if sys.version_info[0] == 3:
+ text_type = str
+ binary_type = bytes
+else:
+ text_type = unicode
+ binary_type = str
+
+READ_CHUNK_SIZE = 21333
+"""
+Number of bytes to read at a time. The value is ~ 1/3rd of 64k which means that
+the value will easily fit in the L2 cache of most processors even if every
+codepoint in a string is three bytes long which makes it a nice fast default
+value.
+"""
+
+
+class SpooledIOBase(object):
+ """
+ The SpooledTempoaryFile class doesn't support a number of attributes and
+ methods that a StringIO instance does. This brings the api as close to
+ compatible as possible with StringIO so that it may be used as a near
+ drop-in replacement to save memory.
+
+ Another issue with SpooledTemporaryFile is that the spooled file is always
+ a cStringIO rather than a StringIO which causes issues with some of our
+ tools.
+ """
+ __metaclass__ = ABCMeta
+
+ def __init__(self, max_size=5000000):
+ self._max_size = max_size
+
+ @abstractmethod
+ def read(self, n=-1):
+ """Read n characters from the buffer"""
+
+ @abstractmethod
+ def write(self, s):
+ """Write into the buffer"""
+
+ @abstractmethod
+ def seek(self, pos, mode=0):
+ """Seek to a specific point in a file"""
+
+ @abstractmethod
+ def readline(self, length=None):
+ """Returns the next available line"""
+
+ @abstractmethod
+ def readlines(self, sizehint=0):
+ """Returns a list of all lines from the current position forward"""
+
+ @abstractmethod
+ def rollover(self):
+ """Roll file-like-object over into a real temporary file"""
+
+ @abstractmethod
+ def tell(self):
+ """Return the current position"""
+
+ @abstractproperty
+ def buffer(self):
+ """Should return a flo instance"""
+
+ @abstractproperty
+ def _rolled(self):
+ """Returns whether the file has been rolled to a real file or not"""
+
+ @abstractproperty
+ def len(self):
+ """Returns the length of the data"""
+
+ def _get_softspace(self):
+ return self.buffer.softspace
+
+ def _set_softspace(self, val):
+ self.buffer.softspace = val
+
+ softspace = property(_get_softspace, _set_softspace)
+
+ @property
+ def _file(self):
+ return self.buffer
+
+ def close(self):
+ return self.buffer.close()
+
+ def flush(self):
+ return self.buffer.flush()
+
+ def isatty(self):
+ return self.buffer.isatty()
+
+ def next(self):
+ return self.readline()
+
+ @property
+ def closed(self):
+ return self.buffer.closed
+
+ @property
+ def pos(self):
+ return self.tell()
+
+ @property
+ def buf(self):
+ return self.getvalue()
+
+ def fileno(self):
+ self.rollover()
+ return self.buffer.fileno()
+
+ def truncate(self, size=None):
+ """
+ Custom version of truncate that takes either no arguments (like the
+ real SpooledTemporaryFile) or a single argument that truncates the
+ value to a certain index location.
+ """
+ if size is None:
+ return self.buffer.truncate()
+
+ if size < 0:
+ raise IOError(EINVAL, "Negative size not allowed")
+
+ # Emulate truncation to a particular location
+ pos = self.tell()
+ self.seek(size)
+ self.buffer.truncate()
+ if pos < size:
+ self.seek(pos)
+
+ def getvalue(self):
+ """Return the entire files contents"""
+ pos = self.tell()
+ self.seek(0)
+ val = self.read()
+ self.seek(pos)
+ return val
+
+ def __len__(self):
+ return self.len
+
+ def __iter__(self):
+ yield self.readline()
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *args):
+ self._file.close()
+
+ def __eq__(self, other):
+ if isinstance(other, self.__class__):
+ return self.getvalue() == other.getvalue()
+ return False
+
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
+ def __bool__(self):
+ return True
+
+ __nonzero__=__bool__
+
+
+class SpooledBytesIO(SpooledIOBase):
+ """
+ SpooledBytesIO is a spooled file-like-object that only accepts bytes. On
+ Python 2.x this means the 'str' type; on Python 3.x this means the 'bytes'
+ type. Bytes are written in and retrieved exactly as given, but it will
+ raise TypeErrors if something other than bytes are written.
+
+ Example::
+
+ >>> from boltons import ioutils
+ >>> with ioutils.SpooledBytesIO() as f:
+ ... f.write(b"Happy IO")
+ ... _ = f.seek(0)
+ ... isinstance(f.getvalue(), ioutils.binary_type)
+ True
+ """
+
+ def read(self, n=-1):
+ return self.buffer.read(n)
+
+ def write(self, s):
+ if not isinstance(s, binary_type):
+ raise TypeError("{0} expected, got {1}".format(
+ binary_type.__name__,
+ type(s).__name__
+ ))
+
+ if self.tell() + len(s) >= self._max_size:
+ self.rollover()
+ self.buffer.write(s)
+
+ def seek(self, pos, mode=0):
+ return self.buffer.seek(pos, mode)
+
+ def readline(self, length=None):
+ return self.buffer.readline(length)
+
+ def readlines(self, sizehint=0):
+ return self.buffer.readlines(sizehint)
+
+ def rollover(self):
+ """Roll the StringIO over to a TempFile"""
+ if not self._rolled:
+ tmp = TemporaryFile()
+ pos = self.buffer.tell()
+ tmp.write(self.buffer.getvalue())
+ tmp.seek(pos)
+ self.buffer.close()
+ self._buffer = tmp
+
+ @property
+ def _rolled(self):
+ return not isinstance(self.buffer, BytesIO)
+
+ @property
+ def buffer(self):
+ try:
+ return self._buffer
+ except AttributeError:
+ self._buffer = BytesIO()
+ return self._buffer
+
+ @property
+ def len(self):
+ """Determine the length of the file"""
+ pos = self.tell()
+ if self._rolled:
+ self.seek(0)
+ val = os.fstat(self.fileno()).st_size
+ else:
... 3243 lines suppressed ...
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/python-boltons.git
More information about the Python-modules-commits
mailing list