[Python-modules-commits] [python-urllib3] 03/13: Import python-urllib3_1.19.1.orig.tar.gz
Daniele Tricoli
eriol-guest at moszumanska.debian.org
Tue Dec 6 00:15:36 UTC 2016
This is an automated email from the git hooks/post-receive script.
eriol-guest pushed a commit to branch master
in repository python-urllib3.
commit 0dbe96444faad3b0e3c39c482d8a7244cc7f9a8d
Author: Daniele Tricoli <eriol at mornie.org>
Date: Tue Dec 6 00:55:06 2016 +0100
Import python-urllib3_1.19.1.orig.tar.gz
---
CHANGES.rst | 100 +++++
CONTRIBUTORS.txt | 10 +
Makefile | 20 +-
PKG-INFO | 250 ++++++------
README.rst | 170 +++------
docs/Makefile | 2 +-
docs/README | 14 -
docs/_templates/fonts.html | 1 +
docs/advanced-usage.rst | 265 +++++++++++++
docs/collections.rst | 13 -
docs/conf.py | 52 ++-
docs/contrib.rst | 102 -----
docs/contributing.rst | 83 ++++
docs/doc-requirements.txt | 12 -
docs/exceptions.rst | 9 -
docs/helpers.rst | 57 ---
docs/index.rst | 424 +++------------------
docs/managers.rst | 131 -------
docs/pools.rst | 86 -----
docs/recipes.rst | 41 --
docs/reference/index.rst | 90 +++++
docs/reference/urllib3.contrib.rst | 37 ++
docs/reference/urllib3.util.rst | 70 ++++
docs/requirements.txt | 4 +
docs/security.rst | 262 -------------
docs/user-guide.rst | 417 ++++++++++++++++++++
dummyserver/certs/server.ip_san.crt | 21 +
dummyserver/handlers.py | 50 ++-
dummyserver/proxy.py | 1 +
dummyserver/server.py | 8 +
setup.cfg | 12 +-
setup.py | 22 +-
test/contrib/test_gae_manager.py | 31 +-
test/contrib/test_pyopenssl.py | 39 +-
test/test_connectionpool.py | 11 +-
test/test_fields.py | 9 +
test/test_response.py | 105 ++++-
test/test_retry.py | 50 ++-
test/test_util.py | 100 ++++-
test/with_dummyserver/test_chunked_transfer.py | 26 ++
test/with_dummyserver/test_connectionpool.py | 107 +++++-
test/with_dummyserver/test_https.py | 124 +++---
test/with_dummyserver/test_proxy_poolmanager.py | 10 +
test/with_dummyserver/test_socketlevel.py | 100 +++++
urllib3.egg-info/PKG-INFO | 250 ++++++------
urllib3.egg-info/SOURCES.txt | 20 +-
urllib3.egg-info/pbr.json | 1 -
urllib3.egg-info/requires.txt | 9 +-
urllib3/__init__.py | 3 +-
urllib3/connection.py | 88 +++--
urllib3/connectionpool.py | 45 ++-
urllib3/contrib/appengine.py | 109 ++++--
urllib3/contrib/ntlmpool.py | 7 +-
urllib3/contrib/pyopenssl.py | 309 +++++++++------
urllib3/contrib/socks.py | 16 +-
urllib3/exceptions.py | 32 ++
urllib3/fields.py | 2 +-
urllib3/filepost.py | 2 +-
urllib3/packages/ssl_match_hostname/__init__.py | 8 +-
.../packages/ssl_match_hostname/_implementation.py | 58 ++-
urllib3/poolmanager.py | 8 +-
urllib3/request.py | 5 +-
urllib3/response.py | 100 ++++-
urllib3/util/connection.py | 1 +
urllib3/util/response.py | 7 +
urllib3/util/retry.py | 110 +++++-
urllib3/util/ssl_.py | 40 +-
urllib3/util/timeout.py | 15 +-
urllib3/util/url.py | 19 +-
69 files changed, 2959 insertions(+), 1853 deletions(-)
diff --git a/CHANGES.rst b/CHANGES.rst
index fdf6a6e..a36cbb1 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,6 +1,106 @@
Changes
=======
+1.19.1 (2016-11-16)
+-------------------
+
+* Fixed AppEngine import that didn't function on Python 3.5. (Pull #1025)
+
+1.19 (2016-11-03)
+-----------------
+
+* urllib3 now respects Retry-After headers on 413, 429, and 503 responses when
+ using the default retry logic. (Pull #955)
+
+* Remove markers from setup.py to assist ancient setuptools versions. (Issue
+ #986)
+
+* Disallow superscripts and other integerish things in URL ports. (Issue #989)
+
+* Allow urllib3's HTTPResponse.stream() method to continue to work with
+ non-httplib underlying FPs. (Pull #990)
+
+* Empty filenames in multipart headers are now emitted as such, rather than
+ being supressed. (Issue #1015)
+
+* Prefer user-supplied Host headers on chunked uploads. (Issue #1009)
+
+
+1.18.1 (2016-10-27)
+-------------------
+
+* CVE-2016-9015. Users who are using urllib3 version 1.17 or 1.18 along with
+ PyOpenSSL injection and OpenSSL 1.1.0 *must* upgrade to this version. This
+ release fixes a vulnerability whereby urllib3 in the above configuration
+ would silently fail to validate TLS certificates due to erroneously setting
+ invalid flags in OpenSSL's ``SSL_CTX_set_verify`` function. These erroneous
+ flags do not cause a problem in OpenSSL versions before 1.1.0, which
+ interprets the presence of any flag as requesting certificate validation.
+
+ There is no PR for this patch, as it was prepared for simultaneous disclosure
+ and release. The master branch received the same fix in PR #1010.
+
+
+1.18 (2016-09-26)
+-----------------
+
+* Fixed incorrect message for IncompleteRead exception. (PR #973)
+
+* Accept ``iPAddress`` subject alternative name fields in TLS certificates.
+ (Issue #258)
+
+* Fixed consistency of ``HTTPResponse.closed`` between Python 2 and 3.
+ (Issue #977)
+
+* Fixed handling of wildcard certificates when using PyOpenSSL. (Issue #979)
+
+
+1.17 (2016-09-06)
+-----------------
+
+* Accept ``SSLContext`` objects for use in SSL/TLS negotiation. (Issue #835)
+
+* ConnectionPool debug log now includes scheme, host, and port. (Issue #897)
+
+* Substantially refactored documentation. (Issue #887)
+
+* Used URLFetch default timeout on AppEngine, rather than hardcoding our own.
+ (Issue #858)
+
+* Normalize the scheme and host in the URL parser (Issue #833)
+
+* ``HTTPResponse`` contains the last ``Retry`` object, which now also
+ contains retries history. (Issue #848)
+
+* Timeout can no longer be set as boolean, and must be greater than zero.
+ (PR #924)
+
+* Removed pyasn1 and ndg-httpsclient from dependencies used for PyOpenSSL. We
+ now use cryptography and idna, both of which are already dependencies of
+ PyOpenSSL. (PR #930)
+
+* Fixed infinite loop in ``stream`` when amt=None. (Issue #928)
+
+* Try to use the operating system's certificates when we are using an
+ ``SSLContext``. (PR #941)
+
+* Updated cipher suite list to allow ChaCha20+Poly1305. AES-GCM is preferred to
+ ChaCha20, but ChaCha20 is then preferred to everything else. (PR #947)
+
+* Updated cipher suite list to remove 3DES-based cipher suites. (PR #958)
+
+* Removed the cipher suite fallback to allow HIGH ciphers. (PR #958)
+
+* Implemented ``length_remaining`` to determine remaining content
+ to be read. (PR #949)
+
+* Implemented ``enforce_content_length`` to enable exceptions when
+ incomplete data chunks are received. (PR #949)
+
+* Dropped connection start, dropped connection reset, redirect, forced retry,
+ and new HTTPS connection log levels to DEBUG, from INFO. (PR #967)
+
+
1.16 (2016-06-11)
-----------------
diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt
index 85df95a..9ef5c69 100644
--- a/CONTRIBUTORS.txt
+++ b/CONTRIBUTORS.txt
@@ -202,5 +202,15 @@ In chronological order:
* Jeremy Cline <jeremy at jcline.org>
* Added connection pool keys by scheme
+* Aviv Palivoda <palaviv at gmail.com>
+ * History list to Retry object.
+ * HTTPResponse contains the last Retry object.
+
+* Nate Prewitt <nate.prewitt at gmail.com>
+ * Ensure timeouts are not booleans and greater than zero.
+ * Fixed infinite loop in ``stream`` when amt=None.
+ * Added length_remaining to determine remaining data to be read.
+ * Added enforce_content_length to raise exception when incorrect content-length received.
+
* [Your name or handle] <[email or website]>
* [Brief summary of your changes]
diff --git a/Makefile b/Makefile
index 8236cc8..3e80cf4 100644
--- a/Makefile
+++ b/Makefile
@@ -2,50 +2,64 @@ REQUIREMENTS_FILE=dev-requirements.txt
REQUIREMENTS_OUT=dev-requirements.txt.log
SETUP_OUT=*.egg-info
-
+.PHONY: all
all: setup requirements
+.PHONY: virtualenv
virtualenv:
ifndef VIRTUAL_ENV
$(error Must be run inside of a virtualenv)
endif
+.PHONY: setup
setup: virtualenv $(SETUP_OUT)
+.PHONY: $(SETUP_OUT)
$(SETUP_OUT): setup.py setup.cfg
python setup.py develop
touch $(SETUP_OUT)
+.PHONY: requirements
requirements: setup $(REQUIREMENTS_OUT)
+.PHONY: piprot
piprot: setup
pip install piprot
piprot -x $(REQUIREMENTS_FILE)
+.PHONY: $(REQUIREMENTS_OUT)
$(REQUIREMENTS_OUT): $(REQUIREMENTS_FILE)
pip install -r $(REQUIREMENTS_FILE) | tee -a $(REQUIREMENTS_OUT)
python setup.py develop
+.PHONY: clean
clean:
find . -name "*.py[oc]" -delete
find . -name "__pycache__" -delete
rm -f $(REQUIREMENTS_OUT)
rm -rf docs/_build build/ dist/
+.PHONY: test
test: requirements
nosetests
+.PHONY: test-all
test-all: requirements
tox
+.PHONY: test-gae
test-gae: requirements
+ifndef GAE_PYTHONPATH
+ $(error GAE_PYTHONPATH must be set)
+endif
tox -e gae
+.PHONY: docs
docs:
- cd docs && pip install -r doc-requirements.txt && make html
+ tox -e docs
+.PHONY: release
release:
./release.sh
-.PHONY: docs
diff --git a/PKG-INFO b/PKG-INFO
index fc5d416..6bc2d4b 100644
--- a/PKG-INFO
+++ b/PKG-INFO
@@ -1,176 +1,198 @@
Metadata-Version: 1.1
Name: urllib3
-Version: 1.16
+Version: 1.19.1
Summary: HTTP library with thread-safe connection pooling, file post, and more.
Home-page: https://urllib3.readthedocs.io/
Author: Andrey Petrov
Author-email: andrey.petrov at shazow.net
License: MIT
-Description: =======
- urllib3
+Description: urllib3
=======
.. image:: https://travis-ci.org/shazow/urllib3.png?branch=master
+ :alt: Build status on Travis
:target: https://travis-ci.org/shazow/urllib3
+ .. image:: https://readthedocs.org/projects/urllib3/badge/?version=latest
+ :alt: Documentation Status
+ :target: https://urllib3.readthedocs.io/en/latest/
+
+ .. image:: https://img.shields.io/pypi/v/urllib3.svg?maxAge=86400
+ :alt: PyPI version
+ :target: https://pypi.python.org/pypi/urllib3
+
.. image:: https://www.bountysource.com/badge/tracker?tracker_id=192525
+ :alt: Bountysource
:target: https://www.bountysource.com/trackers/192525-urllib3?utm_source=192525&utm_medium=shield&utm_campaign=TRACKER_BADGE
+ urllib3 is a powerful, *sanity-friendly* HTTP client for Python. Much of the
+ Python ecosystem already uses urllib3 and you should too.
+ urllib3 brings many critical features that are missing from the Python
+ standard libraries:
- Highlights
- ==========
+ - Thread safety.
+ - Connection pooling.
+ - Client-side SSL/TLS verification.
+ - File uploads with multipart encoding.
+ - Helpers for retrying requests and dealing with HTTP redirects.
+ - Support for gzip and deflate encoding.
+ - Proxy support for HTTP and SOCKS.
+ - 100% test coverage.
- - Re-use the same socket connection for multiple requests
- (``HTTPConnectionPool`` and ``HTTPSConnectionPool``)
- (with optional client-side certificate verification).
- - File posting (``encode_multipart_formdata``).
- - Built-in redirection and retries (optional).
- - Supports gzip and deflate decoding.
- - Proxy over HTTP or SOCKS.
- - Thread-safe and sanity-safe.
- - Works with AppEngine, gevent, and eventlib.
- - Tested on Python 2.6+, Python 3.3+, and PyPy, with 100% unit test coverage.
- - Small and easy to understand codebase perfect for extending and building upon.
- For a more comprehensive solution, have a look at
- `Requests <http://python-requests.org/>`_ which is also powered by ``urllib3``.
+ urllib3 is powerful and easy to use::
+ >>> import urllib3
+ >>> http = urllib3.PoolManager()
+ >>> r = http.request('GET', 'http://httpbin.org/robots.txt')
+ >>> r.status
+ 200
+ >>> r.data
+ 'User-agent: *\nDisallow: /deny\n'
- You might already be using urllib3!
- ===================================
+ Installing
+ ----------
- ``urllib3`` powers `many great Python libraries
- <https://sourcegraph.com/search?q=package+urllib3>`_, including ``pip`` and
- ``requests``.
+ urllib3 can be installed with `pip <https://pip.pypa.io>`_::
+ $ pip install urllib3
- What's wrong with urllib and urllib2?
- =====================================
+ Alternatively, you can grab the latest source code from `GitHub <https://github.com/shazow/urllib3>`_::
- There are two critical features missing from the Python standard library:
- Connection re-using/pooling and file posting. It's not terribly hard to
- implement these yourself, but it's much easier to use a module that already
- did the work for you.
+ $ git clone git://github.com/shazow/urllib3.git
+ $ python setup.py install
- The Python standard libraries ``urllib`` and ``urllib2`` have little to do
- with each other. They were designed to be independent and standalone, each
- solving a different scope of problems, and ``urllib3`` follows in a similar
- vein.
+ Documentation
+ -------------
- Why do I want to reuse connections?
- ===================================
+ urllib3 has usage and reference documentation at `urllib3.readthedocs.io <https://urllib3.readthedocs.io>`_.
- Performance. When you normally do a urllib call, a separate socket
- connection is created with each request. By reusing existing sockets
- (supported since HTTP 1.1), the requests will take up less resources on the
- server's end, and also provide a faster response time at the client's end.
- With some simple benchmarks (see `test/benchmark.py
- <https://github.com/shazow/urllib3/blob/master/test/benchmark.py>`_
- ), downloading 15 URLs from google.com is about twice as fast when using
- HTTPConnectionPool (which uses 1 connection) than using plain urllib (which
- uses 15 connections).
- This library is perfect for:
+ Contributing
+ ------------
- - Talking to an API
- - Crawling a website
- - Any situation where being able to post files, handle redirection, and
- retrying is useful. It's relatively lightweight, so it can be used for
- anything!
+ urllib3 happily accepts contributions. Please see our
+ `contributing documentation <https://urllib3.readthedocs.io/en/latest/contributing.html>`_
+ for some tips on getting started.
+ Maintainers
+ -----------
- Examples
- ========
+ - `@lukasa <https://github.com/lukasa>`_ (Cory Benfield)
+ - `@sigmavirus24 <https://github.com/sigmavirus24>`_ (Ian Cordasco)
+ - `@shazow <https://github.com/shazow>`_ (Andrey Petrov)
- Go to `urllib3.readthedocs.org <https://urllib3.readthedocs.io>`_
- for more nice syntax-highlighted examples.
+ 👋
- But, long story short::
+ Sponsorship
+ -----------
- import urllib3
+ If your company benefits from this library, please consider `sponsoring its
+ development <https://urllib3.readthedocs.io/en/latest/contributing.html#sponsorship>`_.
- http = urllib3.PoolManager()
- r = http.request('GET', 'http://google.com/')
+ Changes
+ =======
- print r.status, r.data
+ 1.19.1 (2016-11-16)
+ -------------------
- The ``PoolManager`` will take care of reusing connections for you whenever
- you request the same host. For more fine-grained control of your connection
- pools, you should look at `ConnectionPool
- <https://urllib3.readthedocs.io/#connectionpool>`_.
+ * Fixed AppEngine import that didn't function on Python 3.5. (Pull #1025)
+ 1.19 (2016-11-03)
+ -----------------
- Run the tests
- =============
+ * urllib3 now respects Retry-After headers on 413, 429, and 503 responses when
+ using the default retry logic. (Pull #955)
- We use some external dependencies, multiple interpreters and code coverage
- analysis while running test suite. Our ``Makefile`` handles much of this for
- you as long as you're running it `inside of a virtualenv
- <http://docs.python-guide.org/en/latest/dev/virtualenvs/>`_::
+ * Remove markers from setup.py to assist ancient setuptools versions. (Issue
+ #986)
- $ make test
- [... magically installs dependencies and runs tests on your virtualenv]
- Ran 182 tests in 1.633s
+ * Disallow superscripts and other integerish things in URL ports. (Issue #989)
- OK (SKIP=6)
+ * Allow urllib3's HTTPResponse.stream() method to continue to work with
+ non-httplib underlying FPs. (Pull #990)
- Note that code coverage less than 100% is regarded as a failing run. Some
- platform-specific tests are skipped unless run in that platform. To make sure
- the code works in all of urllib3's supported platforms, you can run our ``tox``
- suite::
+ * Empty filenames in multipart headers are now emitted as such, rather than
+ being supressed. (Issue #1015)
- $ make test-all
- [... tox creates a virtualenv for every platform and runs tests inside of each]
- py26: commands succeeded
- py27: commands succeeded
- py32: commands succeeded
- py33: commands succeeded
- py34: commands succeeded
+ * Prefer user-supplied Host headers on chunked uploads. (Issue #1009)
- Our test suite `runs continuously on Travis CI
- <https://travis-ci.org/shazow/urllib3>`_ with every pull request.
+ 1.18.1 (2016-10-27)
+ -------------------
- Contributing
- ============
+ * CVE-2016-9015. Users who are using urllib3 version 1.17 or 1.18 along with
+ PyOpenSSL injection and OpenSSL 1.1.0 *must* upgrade to this version. This
+ release fixes a vulnerability whereby urllib3 in the above configuration
+ would silently fail to validate TLS certificates due to erroneously setting
+ invalid flags in OpenSSL's ``SSL_CTX_set_verify`` function. These erroneous
+ flags do not cause a problem in OpenSSL versions before 1.1.0, which
+ interprets the presence of any flag as requesting certificate validation.
- Thank you for giving back to urllib3. Please meet our jolly team
- of code-sherpas:
+ There is no PR for this patch, as it was prepared for simultaneous disclosure
+ and release. The master branch received the same fix in PR #1010.
- Maintainers
- -----------
- - `@lukasa <https://github.com/lukasa>`_ (Cory Benfield)
- - `@sigmavirus24 <https://github.com/sigmavirus24>`_ (Ian Cordasco)
- - `@shazow <https://github.com/shazow>`_ (Andrey Petrov)
+ 1.18 (2016-09-26)
+ -----------------
- 👋
+ * Fixed incorrect message for IncompleteRead exception. (PR #973)
- Getting Started
- ---------------
+ * Accept ``iPAddress`` subject alternative name fields in TLS certificates.
+ (Issue #258)
- #. `Check for open issues <https://github.com/shazow/urllib3/issues>`_ or open
- a fresh issue to start a discussion around a feature idea or a bug. There is
- a *Contributor Friendly* tag for issues that should be ideal for people who
- are not very familiar with the codebase yet.
- #. Fork the `urllib3 repository on Github <https://github.com/shazow/urllib3>`_
- to start making your changes.
- #. Write a test which shows that the bug was fixed or that the feature works
- as expected.
- #. Send a pull request and bug the maintainer until it gets merged and published.
- :) Make sure to add yourself to ``CONTRIBUTORS.txt``.
+ * Fixed consistency of ``HTTPResponse.closed`` between Python 2 and 3.
+ (Issue #977)
+ * Fixed handling of wildcard certificates when using PyOpenSSL. (Issue #979)
- Sponsorship
- ===========
- If your company benefits from this library, please consider `sponsoring its
- development <https://urllib3.readthedocs.io/en/latest/#sponsorship>`_.
+ 1.17 (2016-09-06)
+ -----------------
+ * Accept ``SSLContext`` objects for use in SSL/TLS negotiation. (Issue #835)
+
+ * ConnectionPool debug log now includes scheme, host, and port. (Issue #897)
+
+ * Substantially refactored documentation. (Issue #887)
+
+ * Used URLFetch default timeout on AppEngine, rather than hardcoding our own.
+ (Issue #858)
+
+ * Normalize the scheme and host in the URL parser (Issue #833)
+
+ * ``HTTPResponse`` contains the last ``Retry`` object, which now also
+ contains retries history. (Issue #848)
+
+ * Timeout can no longer be set as boolean, and must be greater than zero.
+ (PR #924)
+
+ * Removed pyasn1 and ndg-httpsclient from dependencies used for PyOpenSSL. We
+ now use cryptography and idna, both of which are already dependencies of
+ PyOpenSSL. (PR #930)
+
+ * Fixed infinite loop in ``stream`` when amt=None. (Issue #928)
+
+ * Try to use the operating system's certificates when we are using an
+ ``SSLContext``. (PR #941)
+
+ * Updated cipher suite list to allow ChaCha20+Poly1305. AES-GCM is preferred to
+ ChaCha20, but ChaCha20 is then preferred to everything else. (PR #947)
+
+ * Updated cipher suite list to remove 3DES-based cipher suites. (PR #958)
+
+ * Removed the cipher suite fallback to allow HIGH ciphers. (PR #958)
+
+ * Implemented ``length_remaining`` to determine remaining content
+ to be read. (PR #949)
+
+ * Implemented ``enforce_content_length`` to enable exceptions when
+ incomplete data chunks are received. (PR #949)
+
+ * Dropped connection start, dropped connection reset, redirect, forced retry,
+ and new HTTPS connection log levels to DEBUG, from INFO. (PR #967)
- Changes
- =======
1.16 (2016-06-11)
-----------------
diff --git a/README.rst b/README.rst
index bf92abc..7c7a338 100644
--- a/README.rst
+++ b/README.rst
@@ -1,134 +1,71 @@
-=======
urllib3
=======
.. image:: https://travis-ci.org/shazow/urllib3.png?branch=master
+ :alt: Build status on Travis
:target: https://travis-ci.org/shazow/urllib3
-.. image:: https://www.bountysource.com/badge/tracker?tracker_id=192525
- :target: https://www.bountysource.com/trackers/192525-urllib3?utm_source=192525&utm_medium=shield&utm_campaign=TRACKER_BADGE
-
-
-Highlights
-==========
-
-- Re-use the same socket connection for multiple requests
- (``HTTPConnectionPool`` and ``HTTPSConnectionPool``)
- (with optional client-side certificate verification).
-- File posting (``encode_multipart_formdata``).
-- Built-in redirection and retries (optional).
-- Supports gzip and deflate decoding.
-- Proxy over HTTP or SOCKS.
-- Thread-safe and sanity-safe.
-- Works with AppEngine, gevent, and eventlib.
-- Tested on Python 2.6+, Python 3.3+, and PyPy, with 100% unit test coverage.
-- Small and easy to understand codebase perfect for extending and building upon.
- For a more comprehensive solution, have a look at
- `Requests <http://python-requests.org/>`_ which is also powered by ``urllib3``.
-
-
-You might already be using urllib3!
-===================================
-
-``urllib3`` powers `many great Python libraries
-<https://sourcegraph.com/search?q=package+urllib3>`_, including ``pip`` and
-``requests``.
-
-
-What's wrong with urllib and urllib2?
-=====================================
-
-There are two critical features missing from the Python standard library:
-Connection re-using/pooling and file posting. It's not terribly hard to
-implement these yourself, but it's much easier to use a module that already
-did the work for you.
-
-The Python standard libraries ``urllib`` and ``urllib2`` have little to do
-with each other. They were designed to be independent and standalone, each
-solving a different scope of problems, and ``urllib3`` follows in a similar
-vein.
-
+.. image:: https://readthedocs.org/projects/urllib3/badge/?version=latest
+ :alt: Documentation Status
+ :target: https://urllib3.readthedocs.io/en/latest/
-Why do I want to reuse connections?
-===================================
+.. image:: https://img.shields.io/pypi/v/urllib3.svg?maxAge=86400
+ :alt: PyPI version
+ :target: https://pypi.python.org/pypi/urllib3
-Performance. When you normally do a urllib call, a separate socket
-connection is created with each request. By reusing existing sockets
-(supported since HTTP 1.1), the requests will take up less resources on the
-server's end, and also provide a faster response time at the client's end.
-With some simple benchmarks (see `test/benchmark.py
-<https://github.com/shazow/urllib3/blob/master/test/benchmark.py>`_
-), downloading 15 URLs from google.com is about twice as fast when using
-HTTPConnectionPool (which uses 1 connection) than using plain urllib (which
-uses 15 connections).
-
-This library is perfect for:
-
-- Talking to an API
-- Crawling a website
-- Any situation where being able to post files, handle redirection, and
- retrying is useful. It's relatively lightweight, so it can be used for
- anything!
-
-
-Examples
-========
-
-Go to `urllib3.readthedocs.org <https://urllib3.readthedocs.io>`_
-for more nice syntax-highlighted examples.
-
-But, long story short::
-
- import urllib3
+.. image:: https://www.bountysource.com/badge/tracker?tracker_id=192525
+ :alt: Bountysource
+ :target: https://www.bountysource.com/trackers/192525-urllib3?utm_source=192525&utm_medium=shield&utm_campaign=TRACKER_BADGE
- http = urllib3.PoolManager()
+urllib3 is a powerful, *sanity-friendly* HTTP client for Python. Much of the
+Python ecosystem already uses urllib3 and you should too.
+urllib3 brings many critical features that are missing from the Python
+standard libraries:
- r = http.request('GET', 'http://google.com/')
+- Thread safety.
+- Connection pooling.
+- Client-side SSL/TLS verification.
+- File uploads with multipart encoding.
+- Helpers for retrying requests and dealing with HTTP redirects.
+- Support for gzip and deflate encoding.
+- Proxy support for HTTP and SOCKS.
+- 100% test coverage.
- print r.status, r.data
+urllib3 is powerful and easy to use::
-The ``PoolManager`` will take care of reusing connections for you whenever
-you request the same host. For more fine-grained control of your connection
-pools, you should look at `ConnectionPool
-<https://urllib3.readthedocs.io/#connectionpool>`_.
+ >>> import urllib3
+ >>> http = urllib3.PoolManager()
+ >>> r = http.request('GET', 'http://httpbin.org/robots.txt')
+ >>> r.status
+ 200
+ >>> r.data
+ 'User-agent: *\nDisallow: /deny\n'
+Installing
+----------
-Run the tests
-=============
+urllib3 can be installed with `pip <https://pip.pypa.io>`_::
-We use some external dependencies, multiple interpreters and code coverage
-analysis while running test suite. Our ``Makefile`` handles much of this for
-you as long as you're running it `inside of a virtualenv
-<http://docs.python-guide.org/en/latest/dev/virtualenvs/>`_::
+ $ pip install urllib3
- $ make test
- [... magically installs dependencies and runs tests on your virtualenv]
- Ran 182 tests in 1.633s
+Alternatively, you can grab the latest source code from `GitHub <https://github.com/shazow/urllib3>`_::
- OK (SKIP=6)
+ $ git clone git://github.com/shazow/urllib3.git
+ $ python setup.py install
-Note that code coverage less than 100% is regarded as a failing run. Some
-platform-specific tests are skipped unless run in that platform. To make sure
-the code works in all of urllib3's supported platforms, you can run our ``tox``
-suite::
- $ make test-all
- [... tox creates a virtualenv for every platform and runs tests inside of each]
- py26: commands succeeded
- py27: commands succeeded
- py32: commands succeeded
- py33: commands succeeded
- py34: commands succeeded
+Documentation
+-------------
-Our test suite `runs continuously on Travis CI
-<https://travis-ci.org/shazow/urllib3>`_ with every pull request.
+urllib3 has usage and reference documentation at `urllib3.readthedocs.io <https://urllib3.readthedocs.io>`_.
Contributing
-============
+------------
-Thank you for giving back to urllib3. Please meet our jolly team
-of code-sherpas:
+urllib3 happily accepts contributions. Please see our
+`contributing documentation <https://urllib3.readthedocs.io/en/latest/contributing.html>`_
+for some tips on getting started.
Maintainers
-----------
@@ -139,23 +76,8 @@ Maintainers
👋
-Getting Started
----------------
-
-#. `Check for open issues <https://github.com/shazow/urllib3/issues>`_ or open
- a fresh issue to start a discussion around a feature idea or a bug. There is
- a *Contributor Friendly* tag for issues that should be ideal for people who
- are not very familiar with the codebase yet.
-#. Fork the `urllib3 repository on Github <https://github.com/shazow/urllib3>`_
- to start making your changes.
-#. Write a test which shows that the bug was fixed or that the feature works
- as expected.
-#. Send a pull request and bug the maintainer until it gets merged and published.
- :) Make sure to add yourself to ``CONTRIBUTORS.txt``.
-
-
Sponsorship
-===========
+-----------
If your company benefits from this library, please consider `sponsoring its
-development <https://urllib3.readthedocs.io/en/latest/#sponsorship>`_.
+development <https://urllib3.readthedocs.io/en/latest/contributing.html#sponsorship>`_.
diff --git a/docs/Makefile b/docs/Makefile
index 135c543..66e3ee5 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -2,7 +2,7 @@
#
# You can set these variables from the command line.
-SPHINXOPTS =
+SPHINXOPTS = '-W'
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
diff --git a/docs/README b/docs/README
deleted file mode 100644
index 9126c73..0000000
--- a/docs/README
+++ /dev/null
@@ -1,14 +0,0 @@
-# Building the Docs
-
-First install Sphinx:
-
- pip install sphinx
-
-Install pyopenssl and certifi dependencies, to avoid some build errors. (Optional)
-
- # This step is optional
- pip install ndg-httpsclient pyasn1 certifi
-
-Then build:
-
- cd docs && make html
diff --git a/docs/_templates/fonts.html b/docs/_templates/fonts.html
new file mode 100644
index 0000000..c52c8f2
--- /dev/null
+++ b/docs/_templates/fonts.html
@@ -0,0 +1 @@
+<link href='https://fonts.googleapis.com/css?family=Roboto|Roboto+Mono' rel='stylesheet' type='text/css'>
diff --git a/docs/advanced-usage.rst b/docs/advanced-usage.rst
new file mode 100644
index 0000000..f28c83c
--- /dev/null
+++ b/docs/advanced-usage.rst
@@ -0,0 +1,265 @@
+Advanced Usage
+==============
+
+.. currentmodule:: urllib3
+
+
+Customizing pool behavior
+-------------------------
+
+The :class:`~poolmanager.PoolManager` class automatically handles creating
+:class:`~connectionpool.ConnectionPool` instances for each host as needed. By
+default, it will keep a maximum of 10 :class:`~connectionpool.ConnectionPool`
+instances. If you're making requests to many different hosts it might improve
+performance to increase this number::
+
+ >>> import urllib3
+ >>> http = urllib3.PoolManager(num_pools=50)
+
+However, keep in mind that this does increase memory and socket consumption.
+
+Similarly, the :class:`~connectionpool.ConnectionPool` class keeps a pool
+of individual :class:`~connection.HTTPConnection` instances. These connections
+are used during an individual request and returned to the pool when the request
+is complete. By default only one connection will be saved for re-use. If you
+are making many requests to the same host simultaneously it might improve
+performance to increase this number::
+
+ >>> import urllib3
+ >>> http = urllib3.PoolManager(maxsize=10)
+ # Alternatively
+ >>> http = urllib3.HTTPConnectionPool('google.com', maxsize=10)
+
+The behavior of the pooling for :class:`~connectionpool.ConnectionPool` is
+different from :class:`~poolmanager.PoolManager`. By default, if a new
+request is made and there is no free connection in the pool then a new
+connection will be created. However, this connection will not be saved if more
+than ``maxsize`` connections exist. This means that ``maxsize`` does not
+determine the maximum number of connections that can be open to a particular
+host, just the maximum number of connections to keep in the pool. However, if you specify ``block=True`` then there can be at most ``maxsize`` connections
+open to a particular host::
+
+ >>> http = urllib3.PoolManager(maxsize=10, block=True)
+ # Alternatively
+ >>> http = urllib3.HTTPConnectionPool('google.com', maxsize=10, block=True)
+
+Any new requests will block until a connection is available from the pool.
+This is a great way to prevent flooding a host with too many connections in
+multi-threaded applications.
+
+.. _stream:
+
+Streaming and IO
+----------------
+
+When dealing with large responses it's often better to stream the response
+content::
+
+ >>> import urllib3
+ >>> http = urllib3.PoolManager()
+ >>> r = http.request(
+ ... 'GET',
+ ... 'http://httpbin.org/bytes/1024',
+ ... preload_content=False)
+ >>> for chunk in r.stream(32):
+ ... print(chunk)
+ b'...'
+ b'...'
+ ...
+ >>> r.release_conn()
+
+Setting ``preload_content`` to ``False`` means that urllib3 will stream the
+response content. :meth:`~response.HTTPResponse.stream` lets you iterate over
+chunks of the response content.
+
+.. note:: When using ``preload_content=False``, you should call
+ :meth:`~response.HTTPResponse.release_conn` to release the http connection
+ back to the connection pool so that it can be re-used.
+
+However, you can also treat the :class:`~response.HTTPResponse` instance as
+a file-like object. This allows you to do buffering::
+
+ >>> r = http.request(
+ ... 'GET',
+ ... 'http://httpbin.org/bytes/1024',
+ ... preload_content=False)
+ >>> r.read(4)
+ b'\x88\x1f\x8b\xe5'
+
+Calls to :meth:`~response.HTTPResponse.read()` will block until more response
+data is available.
+
+ >>> import io
+ >>> reader = io.BufferedReader(r, 8)
+ >>> reader.read(4)
+ >>> r.release_conn()
+
+You can use this file-like object to do things like decode the content using
+:mod:`codecs`::
+
+ >>> import codecs
+ >>> reader = codecs.getreader('utf-8')
+ >>> r = http.request(
+ ... 'GET',
+ ... 'http://httpbin.org/ip',
+ ... preload_content=False)
+ >>> json.load(reader(r))
+ {'origin': '127.0.0.1'}
+ >>> r.release_conn()
+
+.. _proxies:
+
+Proxies
+-------
+
+You can use :class:`~poolmanager.ProxyManager` to tunnel requests through an
+HTTP proxy::
+
+ >>> import urllib3
+ >>> proxy = urllib3.ProxyManager('http://localhost:3128/')
+ >>> proxy.request('GET', 'http://google.com/')
+
+The usage of :class:`~poolmanager.ProxyManager` is the same as
+:class:`~poolmanager.PoolManager`.
+
+You can use :class:`~contrib.socks.SOCKSProxyManager` to connect to SOCKS4 or
+SOCKS5 proxies. In order to use SOCKS proxies you will need to install
+`PySocks <https://pypi.python.org/pypi/PySocks>`_ or install urllib3 with the
+``socks`` extra::
+
+ pip install urllib3[socks]
+
+Once PySocks is installed, you can use
+:class:`~contrib.socks.SOCKSProxyManager`::
+
+ >>> from urllib3.contrib.socks import SOCKSProxyManager
+ >>> proxy = SOCKSProxyManager('socks5://localhost:8889/')
+ >>> proxy.request('GET', 'http://google.com/')
+
+
+.. _ssl_custom:
+
+Custom SSL certificates and client certificates
+-----------------------------------------------
+
+Instead of using `certifi <https://certifi.io/en/latest>`_ you can provide your
+own certificate authority bundle. This is useful for cases where you've
+generated your own certificates or when you're using a private certificate
+authority. Just provide the full path to the certificate bundle when creating a
... 5731 lines suppressed ...
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/python-urllib3.git
More information about the Python-modules-commits
mailing list