[Python-modules-commits] [python-boltons] 01/03: Imported Upstream version 16.4.1
Hugo Lefeuvre
hle at moszumanska.debian.org
Wed Sep 14 12:04:15 UTC 2016
This is an automated email from the git hooks/post-receive script.
hle pushed a commit to branch master
in repository python-boltons.
commit c493b873f428cd2fab9e6f2f3f42d3db00060e50
Author: Hugo Lefeuvre <hle at debian.org>
Date: Wed Sep 14 14:03:12 2016 +0200
Imported Upstream version 16.4.1
---
.travis.yml | 3 +
CHANGELOG.md | 124 +++++++++-
TODO.rst | 7 +-
appveyor.yml | 51 ++++
boltons/cacheutils.py | 173 ++++++++++++-
boltons/debugutils.py | 208 ++++++++++++++++
boltons/ecoutils.py | 487 +++++++++++++++++++++++++++++++++++++
boltons/fileutils.py | 8 +-
boltons/funcutils.py | 368 +++++++++++++++++++++++++++-
boltons/iterutils.py | 18 +-
boltons/socketutils.py | 12 +-
boltons/statsutils.py | 382 ++++++++++++++++++++++++++++-
boltons/strutils.py | 129 +++++++++-
boltons/tbutils.py | 25 ++
boltons/timeutils.py | 121 ++++++++-
boltons/typeutils.py | 2 +
docs/cacheutils.rst | 13 +-
docs/conf.py | 4 +-
docs/ecoutils.rst | 5 +
docs/funcutils.rst | 24 +-
docs/index.rst | 1 +
docs/timeutils.rst | 6 +-
setup.py | 2 +-
tests/conftest.py | 16 ++
tests/test_cacheutils.py | 138 ++++++++++-
tests/test_debugutils_trace.py | 88 +++++++
tests/test_ecoutils.py | 16 ++
tests/test_funcutils_fb.py | 122 ++++++++++
tests/test_funcutils_fb_py3.py | 66 +++++
tests/test_socketutils.py | 18 +-
tests/test_statsutils_histogram.py | 104 ++++++++
tests/test_strutils.py | 19 ++
tests/test_tbutils.py | 18 --
tests/test_tbutils_parsed_exc.py | 53 ++++
tests/test_timeutils.py | 24 +-
35 files changed, 2763 insertions(+), 92 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index d9e52d4..d15f6a8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,3 +10,6 @@ python:
install: "pip install -r requirements-test.txt"
script: "py.test --doctest-modules boltons tests"
+branches:
+ except:
+ - function_builder
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 900c744..20e6913 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,8 +1,104 @@
boltons Changelog
=================
-Since February 20, 2013 there have been 23 releases and 826 commits for
-an average of one 35-commit release every 7.5 weeks.
+Since February 20, 2013 there have been 28 releases and 950 commits for
+an average of one 34-commit release every 6.4 weeks.
+
+16.4.1
+------
+*(June 14, 2016)*
+
+This release primarily contains several [statsutils][statsutils] updates.
+
+* The biggest change was the addition of
+ [Stats.format_histogram][statsutils.Stats.format_histogram] complete
+ with Freedman bin selection and other useful options.
+* Added inter-quartile range (iqr) to [statsutils.Stats][statsutils.Stats]
+* Adding mad (median absolute deviation) to
+ [Stats.describe][statsutils.Stats.describe], since median and
+ std_dev were already there.
+
+16.4.0
+------
+*(June 8, 2016)*
+
+another significant release, thanks to the addition of funcutils.wraps
+and funcutils.FunctionBuilder. also iterutils.chunked speedup, and
+tbutils.ParsedException.to_string.
+
+ * [funcutils.wraps][funcutils.wraps]: Just like functools.wraps, but
+ can preserve the function signature as well.
+ * [funcutils.FunctionBuilder][funcutils.FunctionBuilder]: The basis
+ for [funcutils.wraps][funcutils.wraps], this full-featured type
+ enables programmatically creating functions, from scratch or from
+ existing functions. Supports all Python 2 and 3 function features.
+ * [ecoutils][ecoutils]: Python 2.4 and 2.5 support.
+ * [iterutils][iterutils]: optimize
+ [iterutils.chunked_iter][iterutils.chunked_iter] (2-5x faster
+ depending on runtime). [See #79][i79].
+ * [tbutils][tbutils]: add the
+ [ParsedException.to_string][tbutils.ParsedException.to_string]
+ method, to convert parsed exceptions back into strings, possibly
+ after manipulation
+ * switch FunctionBuilder on Python 2 to be congruent with Python 3
+ (keywords attribute renamed to varkw, preview users might have to
+ adjust)
+
+16.3.1
+------
+*(May 24, 2016)*
+
+Just a couple of [ecoutils][ecoutils] follow-ons, removing uuid
+dependency and adding the ability to scrub identifiable data.
+
+16.3.0
+------
+*(May 23, 2016)*
+
+Big, big update. Lots of additions, a few bugfixes.
+
+* [ecoutils][ecoutils] - Python runtime/environment profile generator
+* [timeutils.strpdate][timeutils.strpdate] - like datetime.datetime.strpdate but for date
+* [timeutils.daterange][timeutils.daterange] - like range() but for datetime.date objects
+* [strutils.parse_int_list][strutils.parse_int_list]
+ and [strutils.format_int_list][strutils.format_int_list]
+* [cacheutils][cacheutils]
+ * [cachedproperty][cacheutils.cachedproperty]
+ * [cacheutils.cachedmethod][cacheutils.cachedmethod]
+ * [cacheutils.cached][cacheutils.cached] now accepts a callable, as well.
+ * ``cacheutils.make_cache_key`` is now public, should others need it
+* [statsutils.Stats][statsutils.Stats] update, several new methods,
+ including [Stats.describe][statsutils.Stats.describe]
+* A few [socketutils][socketutils] platform tweaks
+* `debugutils.wrap_trace` preview
+
+16.2.2
+------
+*(May 3, 2016)*
+
+many small tweaks to socketutils.BufferedSocket, including optional
+inclusion of the delimiter in recv_until. also undid the enabling of bak
+files with AtomicSaver on windows
+
+ * Small [socketutils.BufferedSocket][socketutils.BufferedSocket] tweaks
+ * make recv_until conditionally return the delimiter (by default it
+ does not). also fix a NetstringException inheritance typo
+ * [socketutils][socketutils]: rename BufferedSocket.recv_lock to
+ _recv_lock, and same for send_lock.
+ * add a bunch of simple passthrough methods to better fill out
+ socket's API
+ * add .fileno/.close/.shutdown to [socketutils.BufferedSocket][socketutils.BufferedSocket]
+ * added type/family/proto
+ [socketutils.BufferedSocket][socketutils.BufferedSocket]
+ passthrough properties
+ * BufferedSocket: also lock on .shutdown()
+ * adding an rbuf_unconsumed attribute for post-close debugging, per
+ @doublereedkurt's request
+ * `getsendbuffer()` returns a bytestring and `recv_size()` uses the proper
+ `._recvsize` on the first socket fetch
+ * [fileutils.AtomicSaver][fileutils.AtomicSaver]: revert bak file as
+ it was causing confusion, per [nvie/pip-tools#351](https://github.com/nvie/pip-tools/issues/351)
+
16.2.1
------
@@ -19,9 +115,9 @@ lot of [socketutils.BufferedSocket][socketutils.BufferedSocket] changes.
*maxsize*, *n* became *size*, *marker* became *delimiter*, etc.
* [BufferedSocket][socketutils.BufferedSocket] BufferedSocket became
threadsafe
- * [BufferedSocket.recv][socket.BufferedSocket.recv] now always returns the
+ * [BufferedSocket.recv][socketutils.BufferedSocket.recv] now always returns the
contents of the internal buffer before doing a socket call.
- * [BufferedSocket.recv_close][BufferedSocket.recv_close] now exists
+ * [BufferedSocket.recv_close][socketutils.BufferedSocket.recv_close] now exists
to receive until the sending end closes the connection.
* Can now pass *recvsize* to
[BufferedSocket][socketutils.BufferedSocket] constructor to tune
@@ -142,7 +238,7 @@ with significantly improved behavior and code factoring.
* [strutils.iter_splitlines][strutils.iter_splitlines] is now in the docs.
* [cacheutils][cacheutils]: now imports RLock from the right place for python 2
* [statsutils][statsutils]: Only `delattr` when `hasattr` in
- [Stats.clear_cache][statsutils.Stats.clearcache]
+ [Stats.clear_cache][statsutils.Stats.clear_cache]
* [statsutils.Stats][statsutils.Stats]: Add
[Stats.get_zscore][statsutils.Stats.get_zscore] to support
calculating the [z-score][zscore] (see also: t-statistic)
@@ -575,6 +671,8 @@ added in this release.
[cacheutils.LRU]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.LRU
[cacheutils.ThresholdCounter]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.ThresholdCounter
[cacheutils.cached]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.cached
+[cacheutils.cachedmethod]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.cachedmethod
+[cacheutils.cachedproperty]: http://boltons.readthedocs.org/en/latest/cacheutils.html#boltons.cacheutils.cachedproperty
[debugutils.pdb_on_signal]: http://boltons.readthedocs.org/en/latest/debugutils.html#boltons.debugutils.pdb_on_signal
[dictutils.OMD]: http://boltons.readthedocs.org/en/latest/dictutils.html#boltons.dictutils.OMD
[dictutils.OMD.pop]: http://boltons.readthedocs.org/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict.pop
@@ -582,6 +680,7 @@ added in this release.
[dictutils.OMD.setdefault]: http://boltons.readthedocs.org/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict.setdefault
[dictutils.OrderedMultiDict]: http://boltons.readthedocs.org/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict
[dictutils.OrderedMultiDict.get_inverted]: http://boltons.readthedocs.org/en/latest/dictutils.html#boltons.dictutils.OrderedMultiDict.get_inverted
+[ecoutils]: http://boltons.readthedocs.org/en/latest/ecoutils.html
[excutils.ParsedException]: http://boltons.readthedocs.org/en/latest/excutils.html#boltons.excutils.ParsedException
[fileutils]: http://boltons.readthedocs.org/en/latest/fileutils.html
[fileutils.replace]: http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.replace
@@ -592,7 +691,9 @@ added in this release.
[fileutils.iter_find_files]: http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.iter_find_files
[fileutils.mkdir_p]: http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.mkdir_p
[fileutils.DummyFile]: http://boltons.readthedocs.org/en/latest/fileutils.html#boltons.fileutils.DummyFile
+[funcutils.FunctionBuilder]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.FunctionBuilder
[funcutils.partial_ordering]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.partial_ordering
+[funcutils.wraps]: http://boltons.readthedocs.org/en/latest/funcutils.html#boltons.funcutils.wraps
[gcutils.GCToggler]: http://boltons.readthedocs.org/en/latest/gcutils.html#boltons.gcutils.GCToggler
[gcutils.get_all]: http://boltons.readthedocs.org/en/latest/gcutils.html#boltons.gcutils.get_all
[gcutils.is_tracked]: http://boltons.readthedocs.org/en/latest/gcutils.html#boltons.gcutils.is_tracked
@@ -603,6 +704,8 @@ added in this release.
[i21]: https://github.com/mahmoud/boltons/issues/21
[i30]: https://github.com/mahmoud/boltons/issues/30
[i41]: https://github.com/mahmoud/boltons/issues/41
+[i79]: https://github.com/mahmoud/boltons/pull/79
+[iterutils]: http://boltons.readthedocs.org/en/latest/iterutils.html
[iterutils.backoff]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.backoff
[iterutils.backoff_iter]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.backoff_iter
[iterutils.chunked]: http://boltons.readthedocs.org/en/latest/iterutils.html#boltons.iterutils.chunked
@@ -623,6 +726,7 @@ added in this release.
[jsonutils.JSONLIterator]: http://boltons.readthedocs.org/en/latest/jsonutils.html#boltons.jsonutils.JSONLIterator
[mathutils.ceil]: http://boltons.readthedocs.org/en/latest/mathutils.html#boltons.mathutils.ceil
[mathutils.floor]: http://boltons.readthedocs.org/en/latest/mathutils.html#boltons.mathutils.floor
+[socketutils]: http://boltons.readthedocs.org/en/latest/socketutils.html
[socketutils.BufferedSocket]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.BufferedSocket
[socketutils.BufferedSocket.recv]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.BufferedSocket.recv
[socketutils.BufferedSocket.recv_until]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.BufferedSocket.recv_until
@@ -630,7 +734,9 @@ added in this release.
[socketutils.NetstringSocket]: http://boltons.readthedocs.org/en/latest/socketutils.html#boltons.socketutils.NetstringSocket
[statsutils]: http://boltons.readthedocs.org/en/latest/statsutils.html
[statsutils.Stats]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats
-[statsutils.Stats.clearcache]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats.clear_cache
+[statsutils.Stats.clear_cache]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats.clear_cache
+[statsutils.Stats.describe]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats.describe
+[statsutils.Stats.format_histogram]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats.format_histogram
[statsutils.Stats.get_zscore]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.Stats.get_zscore
[statsutils.median]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.median
[statsutils.trimean]: http://boltons.readthedocs.org/en/latest/statsutils.html#boltons.statsutils.trimean
@@ -648,15 +754,21 @@ added in this release.
[strutils.pluralize]: http://boltons.readthedocs.org/en/latest/strutils.html#boltons.strutils.pluralize
[strutils.is_ascii]: http://boltons.readthedocs.org/en/latest/strutils.html#boltons.strutils.is_ascii
[strutils.is_uuid]: http://boltons.readthedocs.org/en/latest/strutils.html#boltons.strutils.is_uuid
+[strutils.parse_int_list]: http://boltons.readthedocs.org/en/latest/strutils.html#boltons.strutils.parse_int_list
+[strutils.format_int_list]: http://boltons.readthedocs.org/en/latest/strutils.html#boltons.strutils.format_int_list
[tableutils]: http://boltons.readthedocs.org/en/latest/tableutils.html
[tableutils.Table]: http://boltons.readthedocs.org/en/latest/tableutils.html#boltons.tableutils.Table
+[tbutils]: http://boltons.readthedocs.org/en/latest/tbutils.html
[tbutils.ExceptionInfo]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ExceptionInfo
[tbutils.ParsedException]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ParsedException
+[tbutils.ParsedException.to_string]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.ParsedException.to_string
[tbutils.TracebackInfo]: http://boltons.readthedocs.org/en/latest/tbutils.html#boltons.tbutils.TracebackInfo
+[timeutils.daterange]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.daterange
[timeutils.decimal_relative_time]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.decimal_relative_time
[timeutils.dt_to_timestamp]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.dt_to_timestamp
[timeutils.isoparse]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.isoparse
[timeutils.parse_timedelta]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.parse_timedelta
+[timeutils.strpdate]: http://boltons.readthedocs.org/en/latest/timeutils.html#boltons.timeutils.strpdate
[typeutils.get_all_subclasses]: http://boltons.readthedocs.org/en/latest/typeutils.html#boltons.typeutils.get_all_subclasses
[typeutils.make_sentinel]: http://boltons.readthedocs.org/en/latest/typeutils.html#boltons.typeutils.make_sentinel
[zscore]: https://en.wikipedia.org/wiki/Standard_score
diff --git a/TODO.rst b/TODO.rst
index 462e0ce..037679c 100644
--- a/TODO.rst
+++ b/TODO.rst
@@ -27,6 +27,11 @@ jsonutils
misc?
-----
+- wrap_trace debug utility. Takes an object, looks at its dir, wraps
+ everything callable, with a hook. Needs an enable/disable flag.
+ - get/set/call/return/exception
+ - __slots__
+
- Tracking proxy. An object that always succeeds for all operations, saving the call history.
- Top/Bottom singletons (greater than and less than everything)
@@ -45,7 +50,7 @@ tbutils
statsutils
----------
-- percentiles
+- dirty bit auto clears cache on property access
- geometric mean (2 ** sum(log(a, b, ...))
funcutils
diff --git a/appveyor.yml b/appveyor.yml
new file mode 100644
index 0000000..dfa566f
--- /dev/null
+++ b/appveyor.yml
@@ -0,0 +1,51 @@
+# What Python version is installed where:
+# http://www.appveyor.com/docs/installed-software#python
+
+# This configuration based on:
+# https://github.com/audreyr/cookiecutter/commit/3c4685f536afda3be93da3fe3039cec0ab0d60a3
+
+branches:
+ except:
+ - function_builder
+
+environment:
+ matrix:
+ - PYTHON: "C:\\Python27-x64"
+ TOX_ENV: "py27"
+
+ - PYTHON: "C:\\Python35-x64"
+ TOX_ENV: "py35"
+
+
+init:
+ - set PATH=%PYTHON%;%PYTHON%\Scripts;%PATH%
+ - "git config --system http.sslcainfo \"C:\\Program Files\\Git\\mingw64\\ssl\\certs\\ca-bundle.crt\""
+ - "%PYTHON%/python -V"
+ - "%PYTHON%/python -c \"import struct;print(8 * struct.calcsize(\'P\'))\""
+
+install:
+ - "%PYTHON%/Scripts/easy_install -U pip"
+ - "%PYTHON%/Scripts/pip install tox"
+ - "%PYTHON%/Scripts/pip install wheel"
+
+build: false # Not a C# project, build stuff at the test step instead.
+
+test_script:
+ - "%PYTHON%/Scripts/tox -e %TOX_ENV%"
+
+after_test:
+ - "%PYTHON%/python setup.py bdist_wheel"
+ - ps: "ls dist"
+
+on_success:
+ # Report coverage results to codecov.io
+ # and export tox environment variables
+ - "%PYTHON%/Scripts/pip install codecov"
+ - set OS=WINDOWS
+ - "%PYTHON%/Scripts/codecov -e TOX_ENV OS"
+
+artifacts:
+ - path: dist\*
+
+#on_success:
+# - TODO: upload the content of dist/*.whl to a public wheelhouse
diff --git a/boltons/cacheutils.py b/boltons/cacheutils.py
index 40bdb72..0b01104 100644
--- a/boltons/cacheutils.py
+++ b/boltons/cacheutils.py
@@ -33,10 +33,10 @@ Learn more about `caching algorithms on Wikipedia
# TODO: TimedLRI
# TODO: support 0 max_size?
-__all__ = ['LRI', 'LRU', 'cached', 'ThresholdCache']
import itertools
from collections import deque
+from operator import attrgetter
try:
from threading import RLock
@@ -50,7 +50,7 @@ except Exception:
pass
try:
- from typeutils import make_sentinel
+ from boltons.typeutils import make_sentinel
_MISSING = make_sentinel(var_name='_MISSING')
_KWARG_MARK = make_sentinel(var_name='_KWARG_MARK')
except ImportError:
@@ -60,7 +60,9 @@ except ImportError:
try:
xrange
except NameError:
+ # py3
xrange = range
+ unicode, str, bytes, basestring = str, bytes, bytes, (str, bytes)
PREV, NEXT, KEY, VALUE = range(4) # names for the link fields
DEFAULT_MAX_SIZE = 128
@@ -419,12 +421,16 @@ class _HashedKey(list):
def __hash__(self):
return self.hash_value
+ def __repr__(self):
+ return '%s(%s)' % (self.__class__.__name__, list.__repr__(self))
+
-def _make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
- fasttypes=frozenset([int, str, frozenset, type(None)])):
- """Make a cache key from optionally typed positional and keyword
- arguments. If *typed* is ``True``, ``3`` and ``3.0`` will be
- treated as separate keys.
+def make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
+ fasttypes=frozenset([int, str, frozenset, type(None)])):
+ """Make a generic key from a function's positional and keyword
+ arguments, suitable for use in caches. Arguments within *args* and
+ *kwargs* must be `hashable`_. If *typed* is ``True``, ``3`` and
+ ``3.0`` will be treated as separate keys.
The key is constructed in a way that is flat as possible rather than
as a nested structure that would take more memory.
@@ -432,6 +438,11 @@ def _make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
If there is only a single argument and its data type is known to cache
its hash value, then that argument is returned without a wrapper. This
saves space and improves lookup speed.
+
+ >>> tuple(make_cache_key(('a', 'b'), {'c': ('d')}))
+ ('a', 'b', _KWARG_MARK, ('c', 'd'))
+
+ .. _hashable: https://docs.python.org/2/glossary.html#term-hashable
"""
key = list(args)
if kwargs:
@@ -446,19 +457,36 @@ def _make_cache_key(args, kwargs, typed=False, kwarg_mark=_KWARG_MARK,
return key[0]
return _HashedKey(key)
+# for backwards compatibility in case someone was importing it
+_make_cache_key = make_cache_key
+
class CachedFunction(object):
+ """This type is used by :func:`cached`, below. Instances of this
+ class are used to wrap functions in caching logic.
+ """
def __init__(self, func, cache, typed=False):
self.func = func
- self.cache = cache
+ if callable(cache):
+ self.get_cache = cache
+ elif not (callable(getattr(cache, '__getitem__', None))
+ and callable(getattr(cache, '__setitem__', None))):
+ raise TypeError('expected cache to be a dict-like object,'
+ ' or callable returning a dict-like object, not %r'
+ % cache)
+ else:
+ def _get_cache():
+ return cache
+ self.get_cache = _get_cache
self.typed = typed
def __call__(self, *args, **kwargs):
- key = _make_cache_key(args, kwargs, typed=self.typed)
+ cache = self.get_cache()
+ key = make_cache_key(args, kwargs, typed=self.typed)
try:
- ret = self.cache[key]
+ ret = cache[key]
except KeyError:
- ret = self.cache[key] = self.func(*args, **kwargs)
+ ret = cache[key] = self.func(*args, **kwargs)
return ret
def __repr__(self):
@@ -468,14 +496,71 @@ class CachedFunction(object):
return "%s(func=%r)" % (cn, self.func)
+class CachedMethod(object):
+ """Similar to :class:`CachedFunction`, this type is used by
+ :func:`cachedmethod` to wrap methods in caching logic.
+ """
+ def __init__(self, func, cache, typed=False, selfish=True):
+ self.func = func
+ if isinstance(cache, basestring):
+ self.get_cache = attrgetter(cache)
+ elif callable(cache):
+ self.get_cache = cache
+ elif not (callable(getattr(cache, '__getitem__', None))
+ and callable(getattr(cache, '__setitem__', None))):
+ raise TypeError('expected cache to be an attribute name,'
+ ' dict-like object, or callable returning'
+ ' a dict-like object, not %r'
+ % cache)
+ else:
+ def _get_cache(obj):
+ return cache
+ self.get_cache = _get_cache
+ self.typed = typed
+ self.selfish = selfish
+ self.bound_to = None
+
+ def __get__(self, obj, objtype=None):
+ if obj is None:
+ return self
+ cls = self.__class__
+ ret = cls(self.func, self.get_cache,
+ typed=self.typed, selfish=self.selfish)
+ ret.bound_to = obj
+ return ret
+
+ def __call__(self, *args, **kwargs):
+ obj = args[0] if self.bound_to is None else self.bound_to
+ cache = self.get_cache(obj)
+ key_args = (self.bound_to,) + args if self.selfish else args
+ key = make_cache_key(key_args, kwargs, typed=self.typed)
+ try:
+ ret = cache[key]
+ except KeyError:
+ if self.bound_to is not None:
+ args = (self.bound_to,) + args
+ ret = cache[key] = self.func(*args, **kwargs)
+ return ret
+
+ def __repr__(self):
+ cn = self.__class__.__name__
+ args = (cn, self.func, self.typed, self.selfish)
+ if self.bound_to is not None:
+ args += (self.bound_to,)
+ return ('<%s func=%r typed=%r selfish=%r bound_to=%r>' % args)
+ return ("%s(func=%r, typed=%r, selfish=%r)" % args)
+
+
def cached(cache, typed=False):
- """Cache any function with the cache instance of your choosing. Note
+ """Cache any function with the cache object of your choosing. Note
that the function wrapped should take only `hashable`_ arguments.
Args:
cache (Mapping): Any :class:`dict`-like object suitable for
use as a cache. Instances of the :class:`LRU` and
- :class:`LRI` are good choices.
+ :class:`LRI` are good choices, but a plain :class:`dict`
+ can work in some cases, as well. This argument can also be
+ a callable which accepts no arguments and returns a mapping.
typed (bool): Whether to factor argument types into the cache
check. Default ``False``, setting to ``True`` causes the
cache keys for ``3`` and ``3.0`` to be considered unequal.
@@ -491,12 +576,74 @@ def cached(cache, typed=False):
1
.. _hashable: https://docs.python.org/2/glossary.html#term-hashable
+
"""
def cached_func_decorator(func):
return CachedFunction(func, cache, typed=typed)
return cached_func_decorator
+def cachedmethod(cache, typed=False, selfish=True):
+ """Similar to :func:`cached`, ``cachedmethod`` is used to cache
+ methods based on their arguments, using any :class:`dict`-like
+ *cache* object.
+
+ Args:
+ cache (str/Mapping/callable): Can be the name of an attribute
+ on the instance, any Mapping/:class:`dict`-like object, or
+ a callable which returns a Mapping.
+ typed (bool): Whether to factor argument types into the cache
+ check. Default ``False``, setting to ``True`` causes the
+ cache keys for ``3`` and ``3.0`` to be considered unequal.
+ selfish (bool): Whether an instance's ``self`` argument should
+ be considered in the cache key. Use this to share cache
+ objects across instances.
+
+ >>> class Lowerer(object):
+ ... def __init__(self):
+ ... self.cache = LRI()
+ ...
+ ... @cachedmethod('cache')
+ ... def lower(self, text):
+ ... return text.lower()
+ ...
+ >>> lowerer = Lowerer()
+ >>> lowerer.lower('WOW WHO COULD GUESS CACHING COULD BE SO NEAT')
+ 'wow who could guess caching could be so neat'
+ >>> len(lowerer.cache)
+ 1
+ """
+ def cached_method_decorator(func):
+ return CachedMethod(func, cache, typed=typed, selfish=selfish)
+ return cached_method_decorator
+
+
+class cachedproperty(object):
+ """The ``cachedproperty`` is used similar to :class:`property`, except
+ that the wrapped method is only called once. This is commonly used
+ to implement lazy attributes.
+
+ After the property has been accessed, the value is stored on the
+ instance itself, using the same name as the cachedproperty. This
+ allows the cache to be cleared with :func:`delattr`, or through
+ manipulating the object's ``__dict__``.
+ """
+
+ def __init__(self, func):
+ self.func = func
+
+ def __get__(self, obj, objtype=None):
+ if obj is None:
+ return self
+ value = self.func(obj)
+ setattr(obj, self.func.__name__, value)
+ return value
+
+ def __repr__(self):
+ cn = self.__class__.__name__
+ return '<%s func=%s>' % (cn, self.func)
+
+
class ThresholdCounter(object):
"""A **bounded** dict-like Mapping from keys to counts. The
ThresholdCounter automatically compacts after every (1 /
diff --git a/boltons/debugutils.py b/boltons/debugutils.py
index cec147a..a8605ab 100644
--- a/boltons/debugutils.py
+++ b/boltons/debugutils.py
@@ -5,6 +5,22 @@ applications. Currently this focuses on ways to use :mod:`pdb`, the
built-in Python debugger.
"""
+import sys
+import time
+
+try:
+ basestring
+ from repr import Repr
+except NameError:
+ basestring = (str, bytes) # py3
+ from reprlib import Repr
+
+try:
+ from typeutils import make_sentinel
+ _UNSET = make_sentinel(var_name='_UNSET')
+except ImportError:
+ _UNSET = object()
+
__all__ = ['pdb_on_signal', 'pdb_on_exception']
@@ -64,3 +80,195 @@ def pdb_on_exception(limit=100):
pdb.post_mortem(exc_tb)
sys.excepthook = pdb_excepthook
+ return
+
+_repr_obj = Repr()
+_repr_obj.maxstring = 50
+_repr_obj.maxother = 50
+brief_repr = _repr_obj.repr
+
+
+# events: call, return, get, set, del, raise
+def trace_print_hook(event, label, obj, attr_name,
+ args=(), kwargs={}, result=_UNSET):
+ fargs = (event.ljust(6), time.time(), label.rjust(10),
+ obj.__class__.__name__, attr_name)
+ if event == 'get':
+ tmpl = '%s %s - %s - %s.%s -> %s'
+ fargs += (brief_repr(result),)
+ elif event == 'set':
+ tmpl = '%s %s - %s - %s.%s = %s'
+ fargs += (brief_repr(args[0]),)
+ elif event == 'del':
+ tmpl = '%s %s - %s - %s.%s'
+ else: # call/return/raise
+ tmpl = '%s %s - %s - %s.%s(%s)'
+ fargs += (', '.join([brief_repr(a) for a in args]),)
+ if kwargs:
+ tmpl = '%s %s - %s - %s.%s(%s, %s)'
+ fargs += (', '.join(['%s=%s' % (k, brief_repr(v))
+ for k, v in kwargs.items()]),)
+ if result is not _UNSET:
+ tmpl += ' -> %s'
+ fargs += (brief_repr(result),)
+ print(tmpl % fargs)
+ return
+
+
+def wrap_trace(obj, hook=trace_print_hook,
+ which=None, events=None, label=None):
+ """Monitor an object for interactions. Whenever code calls a method,
+ gets an attribute, or sets an attribute, an event is called. By
+ default the trace output is printed, but a custom tracing *hook*
+ can be passed.
+
+ Args:
+ obj (object): New- or old-style object to be traced. Built-in
+ objects like lists and dicts also supported.
+ hook (callable): A function called once for every event. See
+ below for details.
+ which (str): One or more attribute names to trace, or a
+ function accepting attribute name and value, and returing
+ True/False.
+ events (str): One or more kinds of events to call *hook*
+ on. Expected values are ``['get', 'set', 'del', 'call',
+ 'raise', 'return']``. Defaults to all events.
+ label (str): A name to associate with the traced object
+ Defaults to hexadecimal memory address, similar to repr.
+
+ The object returned is not the same object as the one passed
+ in. It will not pass identity checks. However, it will pass
+ :func:`isinstance` checks, as it is a new instance of a new
+ subtype of the object passed.
+
+ """
+ # other actions: pdb.set_trace, print, aggregate, aggregate_return
+ # (like aggregate but with the return value)
+
+ # TODO: test classmethod/staticmethod/property
+ # TODO: wrap __dict__ for old-style classes?
+
+ if isinstance(which, basestring):
+ which_func = lambda attr_name, attr_val: attr_name == which
+ elif callable(getattr(which, '__contains__', None)):
+ which_func = lambda attr_name, attr_val: attr_name in which
+ elif which is None or callable(which):
+ which_func = which
+ else:
+ raise TypeError('expected attr name(s) or callable, not: %r' % which)
+
+ label = label or hex(id(obj))
+
+ if isinstance(events, basestring):
+ events = [events]
+ do_get = not events or 'get' in events
+ do_set = not events or 'set' in events
+ do_del = not events or 'del' in events
+ do_call = not events or 'call' in events
+ do_raise = not events or 'raise' in events
+ do_return = not events or 'return' in events
+
+ def wrap_method(attr_name, func, _hook=hook, _label=label):
+ def wrapped(*a, **kw):
+ a = a[1:]
+ if do_call:
+ hook(event='call', label=_label, obj=obj,
+ attr_name=attr_name, args=a, kwargs=kw)
+ if do_raise:
+ try:
+ ret = func(*a, **kw)
+ except:
+ if not hook(event='raise', label=_label, obj=obj,
+ attr_name=attr_name, args=a, kwargs=kw,
+ result=sys.exc_info()):
+ raise
+ else:
+ ret = func(*a, **kw)
+ if do_return:
+ hook(event='return', label=_label, obj=obj,
+ attr_name=attr_name, args=a, kwargs=kw, result=ret)
+ return ret
+
+ wrapped.__name__ = func.__name__
+ wrapped.__doc__ = func.__doc__
+ try:
+ wrapped.__module__ = func.__module__
+ except Exception:
+ pass
+ try:
+ if func.__dict__:
+ wrapped.__dict__.update(func.__dict__)
+ except Exception:
+ pass
+ return wrapped
+
+ def __getattribute__(self, attr_name):
+ ret = type(obj).__getattribute__(obj, attr_name)
+ if callable(ret): # wrap any bound methods
+ ret = type(obj).__getattribute__(self, attr_name)
+ if do_get:
+ hook('get', label, obj, attr_name, (), {}, result=ret)
+ return ret
+
+ def __setattr__(self, attr_name, value):
+ type(obj).__setattr__(obj, attr_name, value)
+ if do_set:
+ hook('set', label, obj, attr_name, (value,), {})
+ return
+
+ def __delattr__(self, attr_name):
+ type(obj).__delattr__(obj, attr_name)
+ if do_del:
+ hook('del', label, obj, attr_name, (), {})
+ return
+
+ attrs = {}
+ for attr_name in dir(obj):
+ try:
+ attr_val = getattr(obj, attr_name)
+ except Exception:
+ continue
+
+ if not callable(attr_val) or attr_name in ('__new__',):
+ continue
+ elif which_func and not which_func(attr_name, attr_val):
+ continue
+
+ if attr_name == '__getattribute__':
+ wrapped_method = __getattribute__
+ elif attr_name == '__setattr__':
+ wrapped_method = __setattr__
+ elif attr_name == '__delattr__':
+ wrapped_method = __delattr__
+ else:
+ wrapped_method = wrap_method(attr_name, attr_val)
+ attrs[attr_name] = wrapped_method
+
+ cls_name = obj.__class__.__name__
+ if cls_name == cls_name.lower():
+ type_name = 'traced_' + cls_name
+ else:
+ type_name = 'Traced' + cls_name
+
+ if hasattr(obj, '__mro__'):
+ bases = (obj.__class__,)
+ else:
+ # need new-style class for even basic wrapping of callables to
+ # work. getattribute won't work for old-style classes of course.
+ bases = (obj.__class__, object)
+
+ trace_type = type(type_name, bases, attrs)
+ for cls in trace_type.__mro__:
+ try:
+ return cls.__new__(trace_type)
+ except Exception:
+ pass
+ raise TypeError('unable to wrap_trace %r instance %r'
+ % (obj.__class__, obj))
+
+
+if __name__ == '__main__':
+ obj = wrap_trace({})
+ obj['hi'] = 'hello'
+ obj.fail
+ import pdb;pdb.set_trace()
diff --git a/boltons/ecoutils.py b/boltons/ecoutils.py
new file mode 100644
index 0000000..f9c5a75
--- /dev/null
+++ b/boltons/ecoutils.py
@@ -0,0 +1,487 @@
+# -*- coding: utf-8 -*-
+"""The larger a programming ecosystem gets, the greater the chances of
+runtime variability become. Currently, Python is one of the most
+widely deployed high-level programming environments available, making
+it a viable target for all manner of application. But it's important
+to know what you're working with.
+
+Some basic variations that are common among development machines:
+
+* **Executable runtime**: CPython, PyPy, Jython, etc., plus build date and compiler
+* **Language version**: 2.4, 2.5, 2.6, 2.7... 3.3, 3.4, 3.5
+* **Host operating system**: Windows, OS X, Ubuntu, Debian, CentOS, RHEL, etc.
+* **Features**: 64-bit, IPv6, Unicode character support (UCS-2/UCS-4)
+* **Built-in library support**: OpenSSL, threading, SQLite, zlib
+* **User environment**: umask, ulimit, working directory path
+* **Machine info**: CPU count, hostname, filesystem encoding
+
+See the full example profile below for more.
+
+ecoutils was created to quantify that variability. ecoutils quickly
+produces an information-dense description of critical runtime factors,
+with minimal side effects. In short, ecoutils is like browser and user
+agent analytics, but for Python environments.
+
+Transmission and collection
+---------------------------
+
+The data is all JSON serializable, and is suitable for sending to a
+central analytics server. An HTTP-backed service for this can be found
+at: https://github.com/mahmoud/espymetrics/
+
+Notable omissions
+-----------------
+
+Due to space constraints (and possibly latency constraints), the
+following information is deemed not dense enough, and thus omitted:
+
+* :data:`sys.path`
+* full :mod:`sysconfig`
+* environment variables (:data:`os.environ`)
+
+Compatibility
+-------------
+
+So far ecoutils has has been tested on Python 2.4, 2.5, 2.6, 2.7, 3.4,
+3.5, and PyPy. Various versions have been tested on Ubuntu, Debian,
+RHEL, OS X, FreeBSD, and Windows 7.
+
+.. note:: Boltons typically only support back to Python 2.6, but due
+ to its nature, ecoutils extends backwards compatibility to Python
+ 2.4 and 2.5.
+
+Profile generation
+------------------
+
+Profiles are generated by :func:`ecoutils.get_profile`.
+
+When run as a module, ecoutils will call :func:`~ecoutils.get_profile`
+and print a profile in JSON format::
+
+ $ python -m boltons.ecoutils
+ {
+ "_eco_version": "1.0.0",
+ "cpu_count": 4,
+ "cwd": "/home/mahmoud/projects/boltons",
+ "fs_encoding": "UTF-8",
+ "guid": "6b139e7bbf5ad4ed8d4063bf6235b4d2",
+ "hostfqdn": "mahmoud-host",
+ "hostname": "mahmoud-host",
+ "linux_dist_name": "Ubuntu",
+ "linux_dist_version": "14.04",
+ "python": {
+ "argv": "boltons/ecoutils.py",
+ "bin": "/usr/bin/python",
+ "build_date": "Jun 22 2015 17:58:13",
+ "compiler": "GCC 4.8.2",
+ "features": {
+ "64bit": true,
+ "expat": "expat_2.1.0",
+ "ipv6": true,
+ "openssl": "OpenSSL 1.0.1f 6 Jan 2014",
+ "readline": true,
+ "sqlite": "3.8.2",
+ "threading": true,
+ "tkinter": "8.6",
+ "unicode_wide": true,
+ "zlib": "1.2.8"
+ },
+ "version": "2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2]",
+ "version_info": [
+ 2,
+ 7,
+ 6,
+ "final",
+ 0
+ ]
+ },
+ "time_utc": "2016-05-24 07:59:40.473140",
+ "time_utc_offset": -8.0,
+ "ulimit_hard": 4096,
+ "ulimit_soft": 1024,
+ "umask": "002",
+ "uname": {
+ "machine": "x86_64",
+ "node": "mahmoud-host",
+ "processor": "x86_64",
+ "release": "3.13.0-85-generic",
+ "system": "Linux",
+ "version": "#129-Ubuntu SMP Thu Mar 17 20:50:15 UTC 2016"
+ },
+ "username": "mahmoud"
+ }
+
+``pip install boltons`` and try it yourself!
+
+"""
+
+import re
+import os
+import sys
+import time
+import pprint
+import random
+import socket
+import struct
+import getpass
+import datetime
+import platform
+
+ECO_VERSION = '1.0.1' # see version history below
+
+PY_GT_2 = sys.version_info[0] > 2
+
+try:
+ getrandbits = random.SystemRandom().getrandbits
+ HAVE_URANDOM = True
+except Exception:
+ HAVE_URANDOM = False
+ getrandbits = random.getrandbits
+
+
+# 128-bit GUID just like a UUID, but backwards compatible to 2.4
+INSTANCE_ID = hex(getrandbits(128))[2:-1].lower()
+
+IS_64BIT = struct.calcsize("P") > 4
+HAVE_UCS4 = getattr(sys, 'maxunicode', 0) > 65536
+HAVE_READLINE = True
+
+try:
+ import readline
+except Exception:
+ HAVE_READLINE = False
+
+try:
+ import sqlite3
+ SQLITE_VERSION = sqlite3.sqlite_version
+except Exception:
+ # note: 2.5 and older have sqlite, but not sqlite3
+ SQLITE_VERSION = ''
+
+
+try:
+
+ import ssl
+ try:
+ OPENSSL_VERSION = ssl.OPENSSL_VERSION
+ except AttributeError:
... 2638 lines suppressed ...
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/python-boltons.git
More information about the Python-modules-commits
mailing list