[Git][debian-gis-team/mapproxy][master] 11 commits: Update branch in gbp.conf & Vcs-Git URL.
Bas Couwenberg (@sebastic)
gitlab at salsa.debian.org
Sun Aug 15 10:33:19 BST 2021
Bas Couwenberg pushed to branch master at Debian GIS Project / mapproxy
Commits:
a542c5fc by Bas Couwenberg at 2021-07-13T13:27:51+02:00
Update branch in gbp.conf & Vcs-Git URL.
- - - - -
3d3f3804 by Bas Couwenberg at 2021-07-13T13:28:13+02:00
New upstream version 1.13.1
- - - - -
a78a780c by Bas Couwenberg at 2021-07-13T13:28:19+02:00
Update upstream source from tag 'upstream/1.13.1'
Update to upstream version '1.13.1'
with Debian dir dcf8b2fd1eb263b9dd0beff7bfd2cbcbd9a0c00a
- - - - -
eccca97f by Bas Couwenberg at 2021-07-13T13:29:00+02:00
New upstream release.
- - - - -
3ac191b0 by Bas Couwenberg at 2021-07-13T13:30:22+02:00
Set distribution to experimental.
- - - - -
51d5e5b1 by Bas Couwenberg at 2021-07-14T16:27:25+02:00
New upstream version 1.13.2
- - - - -
97dbf298 by Bas Couwenberg at 2021-07-14T16:27:31+02:00
Update upstream source from tag 'upstream/1.13.2'
Update to upstream version '1.13.2'
with Debian dir 5f15a7b65941f0aac3a5c157c463a0c3bd031081
- - - - -
e8a63820 by Bas Couwenberg at 2021-07-14T16:27:45+02:00
New upstream release.
- - - - -
518f0036 by Bas Couwenberg at 2021-07-14T16:28:25+02:00
Set distribution to experimental.
- - - - -
ca677983 by Bas Couwenberg at 2021-08-15T11:24:48+02:00
Revert "Update branch in gbp.conf & Vcs-Git URL."
This reverts commit a542c5fc90186a84044b7bfb49b2759875f6888c.
- - - - -
e5434fc4 by Bas Couwenberg at 2021-08-15T11:24:58+02:00
Move from experimental to unstable.
- - - - -
18 changed files:
- .travis.yml
- CHANGES.txt
- debian/changelog
- doc/conf.py
- doc/configuration.rst
- doc/configuration_examples.rst
- doc/sources.rst
- mapproxy/client/http.py
- mapproxy/config/defaults.py
- mapproxy/config/loader.py
- mapproxy/config/spec.py
- mapproxy/config_template/base_config/full_example.yaml
- mapproxy/config_template/base_config/mapproxy.yaml
- mapproxy/service/demo.py
- mapproxy/test/system/fixture/layer.yaml
- mapproxy/test/system/test_wms.py
- mapproxy/test/unit/test_client.py
- setup.py
Changes:
=====================================
.travis.yml
=====================================
@@ -10,6 +10,7 @@ python:
services:
- couchdb
- redis-server
+ - docker
addons:
apt:
@@ -31,6 +32,8 @@ env:
global:
- MAPPROXY_TEST_COUCHDB=http://127.0.0.1:5984
- MAPPROXY_TEST_REDIS=127.0.0.1:6379
+ - MAPPROXY_TEST_RIAK_HTTP=http://localhost:8098
+ - MAPPROXY_TEST_RIAK_PBC=pbc://localhost:8087
# do not load /etc/boto.cfg with Python 3 incompatible plugin
# https://github.com/travis-ci/travis-ci/issues/5246#issuecomment-166460882
@@ -48,6 +51,9 @@ cache:
directories:
- $HOME/.cache/pip
+before_install:
+ - docker run --detach --rm --publish 8087:8087 --publish 8098:8098 basho/riak-kv
+
install:
- "pip install -r requirements-tests.txt"
- "if [[ $USE_LATEST_PILLOW = '1' ]]; then pip install -U Pillow; fi"
=====================================
CHANGES.txt
=====================================
@@ -1,3 +1,24 @@
+Nightly
+~~~~~~~~~~~~~~~~~
+
+1.13.2 2021-07-14
+~~~~~~~~~~~~~~~~~
+
+Fixes:
+
+- Hotfix: Fixup demo service (#528).
+
+1.13.1 2021-07-13
+~~~~~~~~~~~~~~~~~
+
+Improvements:
+
+- Support cookie management for HTTP sources.
+
+Fixes:
+
+- Security fix for local file disclosure (#526).
+
1.13.0 2020-11-18
~~~~~~~~~~~~~~~~~
=====================================
debian/changelog
=====================================
@@ -1,9 +1,22 @@
-mapproxy (1.13.0-2) UNRELEASED; urgency=medium
+mapproxy (1.13.2-1) unstable; urgency=medium
+ * Move from experimental to unstable.
+
+ -- Bas Couwenberg <sebastic at debian.org> Sun, 15 Aug 2021 11:24:56 +0200
+
+mapproxy (1.13.2-1~exp1) experimental; urgency=medium
+
+ * New upstream release.
+
+ -- Bas Couwenberg <sebastic at debian.org> Wed, 14 Jul 2021 16:28:11 +0200
+
+mapproxy (1.13.1-1~exp1) experimental; urgency=medium
+
+ * New upstream release.
* Bump Standards-Version to 4.5.1, no changes.
* Update watch file for GitHub URL changes.
- -- Bas Couwenberg <sebastic at debian.org> Sat, 28 Nov 2020 13:50:22 +0100
+ -- Bas Couwenberg <sebastic at debian.org> Tue, 13 Jul 2021 13:30:08 +0200
mapproxy (1.13.0-1) unstable; urgency=medium
=====================================
doc/conf.py
=====================================
@@ -51,7 +51,7 @@ copyright = u'Oliver Tonnhofer, Omniscale'
# The short X.Y version.
version = '1.13'
# The full version, including alpha/beta/rc tags.
-release = '1.13.0'
+release = '1.13.2'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
=====================================
doc/configuration.rst
=====================================
@@ -1024,6 +1024,15 @@ Add additional HTTP headers to all requests to your sources.
Sets the ``Access-control-allow-origin`` header to HTTP responses for `Cross-origin resource sharing <http://en.wikipedia.org/wiki/Cross-origin_resource_sharing>`_. This header is required for WebGL or Canvas web clients. Defaults to `*`. Leave empty to disable the header. This option is only available in `globals`.
+``manage_cookies``
+^^^^^^^^^^^^^^^^^^
+
+.. versionadded:: 1.14.0
+
+Enables MapProxy cookie management for HTTP sources. When enabled MapProxy will accept and store server cookies. Accepted cookies will be passed
+back to the source on subsequent requests. Usefull for sources which require to maintain an HTTP session to work efficiently, maybe in combination
+with basic authentication. Depending on your deployment MapProxy will still start multiple sessions (e.g. one per MapProxy process).
+Cookie handling is based on Python `CookieJar <https://docs.python.org/3/library/http.cookiejar.html>`_. Disabled by default.
``hide_error_details``
^^^^^^^^^^^^^^^^^^^^^^
=====================================
doc/configuration_examples.rst
=====================================
@@ -667,7 +667,7 @@ Then we can add a layer with all available dimensions::
- 3000
default: 0
-You can know access this layer with the elevation and time dimensions via the WMTS KVP service.
+You can now access this layer with the elevation and time dimensions via the WMTS KVP service.
The RESTful service requires a custom URL template that contains the dimensions. For example::
services:
@@ -708,6 +708,8 @@ You can disable the certificate verification if you you don't need it.
url: https://username:mypassword@example.org/service?
layers: securelayer
+.. note:: If the source requires session handling through cookies, have a look at the ``manage_cookies`` configuration option.
+
.. _http_proxy:
Access sources through HTTP proxy
@@ -734,11 +736,28 @@ You can also set this in your :ref:`server script <server_script>`::
Add a username and password to the URL if your HTTP proxy requires authentication. For example ``http://username:password@example.com:3128``.
+.. note:: If the source requires session handling through cookies, have a look at the ``manage_cookies`` configuration option.
+
You can use the ``no_proxy`` environment variable if you need to bypass the proxy for some hosts::
$ export no_proxy="localhost,127.0.0.1,196.168.1.99"
+Cookie Management
+=================
+
+MapProxy can handle server cookies of HTTP sources, like browsers do. That is, MapProxy accepts cookies and passes them back
+on subsequent calls. This is useful for sources that use cookie for session management or rate-limiting for example::
+
+ sources:
+ wms_with_session_management:
+ type: wms
+ http:
+ manage_cookies: True
+ req:
+ url: http://example.org/service?
+ layers: layer0
+
.. _paster_urlmap:
Serve multiple MapProxy instances
=====================================
doc/sources.rst
=====================================
@@ -184,6 +184,7 @@ You can configure the following HTTP related options for this source:
- ``client_timeout``
- ``ssl_ca_certs``
- ``ssl_no_cert_checks``
+- ``manage_cookies``
See :ref:`HTTP Options <http_ssl>` for detailed documentation.
@@ -432,6 +433,7 @@ You can configure the following HTTP related options for this source:
- ``client_timeout``
- ``ssl_ca_certs``
- ``ssl_no_cert_checks``
+- ``manage_cookies``
See :ref:`HTTP Options <http_ssl>` for detailed documentation.
=====================================
mapproxy/client/http.py
=====================================
@@ -29,12 +29,15 @@ from mapproxy.compat.modules import urlparse
if PY2:
import urllib2
- from urllib2 import URLError, HTTPError
+ from urllib2 import URLError, HTTPError, HTTPCookieProcessor
import httplib
+ from cookielib import CookieJar
else:
from urllib import request as urllib2
from urllib.error import URLError, HTTPError
+ from urllib.request import HTTPCookieProcessor
from http import client as httplib
+ from http.cookiejar import CookieJar
import socket
import ssl
@@ -131,8 +134,8 @@ class _URLOpenerCache(object):
def __init__(self):
self._opener = {}
- def __call__(self, ssl_ca_certs, url, username, password, insecure=False):
- cache_key = (ssl_ca_certs, insecure)
+ def __call__(self, ssl_ca_certs, url, username, password, insecure=False, manage_cookies=False):
+ cache_key = (ssl_ca_certs, insecure, manage_cookies)
if cache_key not in self._opener:
handlers = []
https_handler = build_https_handler(ssl_ca_certs, insecure)
@@ -143,6 +146,9 @@ class _URLOpenerCache(object):
handlers.append(authhandler)
authhandler = urllib2.HTTPDigestAuthHandler(passman)
handlers.append(authhandler)
+ if manage_cookies:
+ cj = CookieJar()
+ handlers.append(HTTPCookieProcessor(cj))
opener = urllib2.build_opener(*handlers)
@@ -161,7 +167,8 @@ create_url_opener = _URLOpenerCache()
class HTTPClient(object):
def __init__(self, url=None, username=None, password=None, insecure=False,
- ssl_ca_certs=None, timeout=None, headers=None, hide_error_details=False):
+ ssl_ca_certs=None, timeout=None, headers=None, hide_error_details=False,
+ manage_cookies=False):
self._timeout = timeout
if url and url.startswith('https'):
if insecure:
@@ -170,7 +177,7 @@ class HTTPClient(object):
raise HTTPClientError('No ca_certs file set (http.ssl_ca_certs). '
'Set file or disable verification with http.ssl_no_cert_checks option.')
- self.opener = create_url_opener(ssl_ca_certs, url, username, password, insecure=insecure)
+ self.opener = create_url_opener(ssl_ca_certs, url, username, password, insecure=insecure, manage_cookies=manage_cookies)
self.header_list = headers.items() if headers else []
self.hide_error_details = hide_error_details
=====================================
mapproxy/config/defaults.py
=====================================
@@ -94,4 +94,5 @@ http = dict(
method = 'AUTO',
access_control_allow_origin = '*',
hide_error_details = True,
+ manage_cookies = False,
)
=====================================
mapproxy/config/loader.py
=====================================
@@ -592,10 +592,12 @@ class SourceConfiguration(ConfigurationBase):
timeout = self.context.globals.get_value('http.client_timeout', self.conf)
headers = self.context.globals.get_value('http.headers', self.conf)
hide_error_details = self.context.globals.get_value('http.hide_error_details', self.conf)
+ manage_cookies = self.context.globals.get_value('http.manage_cookies', self.conf)
http_client = HTTPClient(url, username, password, insecure=insecure,
ssl_ca_certs=ssl_ca_certs, timeout=timeout,
- headers=headers, hide_error_details=hide_error_details)
+ headers=headers, hide_error_details=hide_error_details,
+ manage_cookies=manage_cookies)
return http_client, url
@memoize
=====================================
mapproxy/config/spec.py
=====================================
@@ -75,6 +75,7 @@ http_opts = {
'headers': {
anything(): str()
},
+ 'manage_cookies': bool(),
}
mapserver_opts = {
=====================================
mapproxy/config_template/base_config/full_example.yaml
=====================================
@@ -429,6 +429,21 @@ sources:
transparent: true
layers: securelayer
+ # WMS source that requires authentication and session management
+ # through HTTP cookies
+ session_source:
+ type: wms
+ http:
+ # Accept session cookies and forward on subsequent requests
+ manage_cookies: true
+ # Use basic auth header directly
+ headers:
+ Authorization: Basic YWRtaW46Z2Vvc2VydmVy
+ req:
+ url: https://my-service.com/service?
+ transparent: true
+ layers: securelayer
+
feature_info_source:
type: wms
wms_opts:
=====================================
mapproxy/config_template/base_config/mapproxy.yaml
=====================================
@@ -18,7 +18,7 @@
# first tile: http://localhost:8080/tiles/osm/webmercator/0/0/0.png
# TMS:
# note: TMS is not compatible with OSM/Google Maps/etc.
-# fist tile: http://localhost:8080/tms/1.0.0/osm/webmercator/0/0/0.png
+# first tile: http://localhost:8080/tms/1.0.0/osm/webmercator/0/0/0.png
# KML:
# initial doc: http://localhost:8080/kml/osm/webmercator
=====================================
mapproxy/service/demo.py
=====================================
@@ -68,6 +68,8 @@ class DemoServer(Server):
def handle(self, req):
if req.path.startswith('/demo/static/'):
+ if '..' in req.path:
+ return Response('file not found', content_type='text/plain', status=404)
filename = req.path.lstrip('/')
filename = static_filename(filename)
if not os.path.isfile(filename):
=====================================
mapproxy/test/system/fixture/layer.yaml
=====================================
@@ -126,6 +126,9 @@ layers:
- name: watermark_cache
title: TMS Cache + watermark
sources: [watermark_cache]
+ - name: wms_managed_cookies_cache
+ title: WMS with cookies management
+ sources: [wms_managed_cookies]
caches:
wms_cache:
@@ -164,6 +167,8 @@ caches:
disable_storage: true
watermark:
text: '@ Omniscale'
+ wms_managed_cookies_cache:
+ sources: [wms_managed_cookies]
sources:
direct:
@@ -244,3 +249,12 @@ sources:
coverage:
bbox: [-180,-90,170,80]
srs: 'EPSG:4326'
+ wms_managed_cookies:
+ type: wms
+ wms_opts:
+ featureinfo: True
+ req:
+ url: http://localhost:42423/service
+ layers: layer1
+ http:
+ manage_cookies: True
=====================================
mapproxy/test/system/test_wms.py
=====================================
@@ -195,6 +195,7 @@ class TestWMS111(SysTest):
"wms_cache_link_single",
"wms_cache_110",
"watermark_cache",
+ "wms_managed_cookies_cache",
]
)
assert layer_names == expected_names
@@ -902,6 +903,7 @@ class TestWMS110(SysTest):
"wms_cache_link_single",
"wms_cache_110",
"watermark_cache",
+ "wms_managed_cookies_cache",
]
)
assert layer_names == expected_names
@@ -1056,6 +1058,41 @@ class TestWMS110(SysTest):
assert "tms_cache is not queryable" in xml.xpath("//ServiceException/text()")[0]
assert validate_with_dtd(xml, "wms/1.1.0/exception_1_1_0.dtd")
+ def test_managed_cookies(self, app):
+ def assert_no_cookie(req_handler):
+ return 'Cookie' not in req_handler.headers
+
+ def assert_cookie(req_handler):
+ assert 'Cookie' in req_handler.headers
+ cookie_name, cookie_val = req_handler.headers['Cookie'].split(';')[0].split('=')
+ assert cookie_name == 'testcookie'
+ assert cookie_val == '42'
+ return True
+
+ url = (r"/service?LAYERs=layer1&SERVICE=WMS&FORMAT=image%2Fpng"
+ "&REQUEST=GetFeatureInfo&HEIGHT=200&SRS=EPSG%3A900913"
+ "&VERSION=1.1.1&BBOX=1000.0,400.0,2000.0,1400.0&styles="
+ "&WIDTH=200&QUERY_LAYERS=layer1&X=10&Y=20")
+ # First response has a Set-Cookie => with managed_cookies=True, mapproxy should send the
+ # cookie in the second request
+ expected_requests = [
+ (
+ {'path': url, 'req_assert_function': assert_no_cookie},
+ {'body': b'nothing', 'headers': {'Set-Cookie': "testcookie=42"}}
+ ),
+ (
+ {'path': url, 'req_assert_function': assert_cookie},
+ {'body': b'nothing'}
+ )
+ ]
+ with mock_httpd(("localhost", 42423), expected_requests):
+ self.common_fi_req.params["layers"] = "wms_managed_cookies_cache"
+ self.common_fi_req.params["query_layers"] = "wms_managed_cookies_cache"
+ resp = app.get(self.common_fi_req)
+ assert resp.body == b"nothing"
+ resp = app.get(self.common_fi_req)
+ assert resp.body == b"nothing"
+
class TestWMS100(SysTest):
config_file = "layer.yaml"
@@ -1119,6 +1156,7 @@ class TestWMS100(SysTest):
"wms_cache_link_single",
"wms_cache_110",
"watermark_cache",
+ "wms_managed_cookies_cache",
]
)
assert layer_names == expected_names
@@ -1334,6 +1372,7 @@ class TestWMS130(SysTest):
"wms_cache_link_single",
"wms_cache_110",
"watermark_cache",
+ "wms_managed_cookies_cache",
]
)
assert layer_names == expected_names
=====================================
mapproxy/test/unit/test_client.py
=====================================
@@ -185,6 +185,60 @@ class TestHTTPClient(object):
assert 0.1 <= duration1 < 0.5, duration1
assert 0.5 <= duration2 < 0.9, duration2
+ def test_manage_cookies_off(self):
+ """
+ Test the behavior when manage_cookies is off (the default). Cookies shouldn't be sent
+ """
+ self.client = HTTPClient()
+
+ def assert_no_cookie(req_handler):
+ return 'Cookie' not in req_handler.headers
+
+ test_requests = [
+ (
+ {'path': '/', 'req_assert_function': assert_no_cookie},
+ {'body': b'nothing', 'headers': {'Set-Cookie': "testcookie=42"}}
+ ),
+ (
+ {'path': '/', 'req_assert_function': assert_no_cookie},
+ {'body': b'nothing'}
+ )
+ ]
+ with mock_httpd(TESTSERVER_ADDRESS, test_requests):
+ self.client.open(TESTSERVER_URL + '/')
+ self.client.open(TESTSERVER_URL + '/')
+
+ def test_manage_cookies_on(self):
+ """
+ Test behavior of manage_cookies=True. Once the remote server sends a cookie back, it should
+ be included in future requests
+ """
+ self.client = HTTPClient(manage_cookies=True)
+
+ def assert_no_cookie(req_handler):
+ return 'Cookie' not in req_handler.headers
+
+ def assert_cookie(req_handler):
+ assert 'Cookie' in req_handler.headers
+ cookie_name, cookie_val = req_handler.headers['Cookie'].split(';')[0].split('=')
+ assert cookie_name == 'testcookie'
+ assert cookie_val == '42'
+ return True
+
+ test_requests = [
+ (
+ {'path': '/', 'req_assert_function': assert_no_cookie},
+ {'body': b'nothing', 'headers': {'Set-Cookie': "testcookie=42"}}
+ ),
+ (
+ {'path': '/', 'req_assert_function': assert_cookie},
+ {'body': b'nothing'}
+ )
+ ]
+ with mock_httpd(TESTSERVER_ADDRESS, test_requests):
+ self.client.open(TESTSERVER_URL + '/')
+ self.client.open(TESTSERVER_URL + '/')
+
# root certificates for google.com, if no ca-certificates.cert
# file is found
=====================================
setup.py
=====================================
@@ -54,7 +54,7 @@ def long_description(changelog_releases=10):
setup(
name='MapProxy',
- version="1.13.0",
+ version="1.13.2",
description='An accelerating proxy for tile and web map services',
long_description=long_description(7),
author='Oliver Tonnhofer',
View it on GitLab: https://salsa.debian.org/debian-gis-team/mapproxy/-/compare/a96745d44c5d4d57f98f4a24a28d766caeb459f2...e5434fc4b55d5278d1dc1b2ea5c8f9bf3bf97ff2
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/mapproxy/-/compare/a96745d44c5d4d57f98f4a24a28d766caeb459f2...e5434fc4b55d5278d1dc1b2ea5c8f9bf3bf97ff2
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20210815/440df4e3/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list