[pycsw] 01/04: Imported Upstream version 2.0.1+dfsg
Angelos Tzotsos
kalxas-guest at moszumanska.debian.org
Mon Sep 12 09:00:21 UTC 2016
This is an automated email from the git hooks/post-receive script.
kalxas-guest pushed a commit to branch master
in repository pycsw.
commit 68bda58bba5d016e3e0aed6383bb681d742ee9a5
Author: Angelos Tzotsos <gcpp.kalxas at gmail.com>
Date: Mon Sep 12 11:51:59 2016 +0300
Imported Upstream version 2.0.1+dfsg
---
README.txt | 2 +-
VERSION.txt | 2 +-
docs/_templates/indexsidebar.html | 8 ++-
docs/introduction.rst | 2 +-
pycsw/__init__.py | 2 +-
pycsw/ogc/csw/csw2.py | 9 ++-
pycsw/ogc/csw/csw3.py | 9 ++-
pycsw/plugins/repository/hhypermap/hhypermap.py | 76 ++++++++++++++++++----
pycsw/server.py | 5 +-
pycsw/wsgi.py | 9 ++-
requirements.txt | 2 +-
...xception-GetRepositoryItem-service-invalid1.xml | 7 ++
...xception-GetRepositoryItem-service-invalid2.xml | 7 ++
...Exception-GetRepositoryItem-version-invalid.xml | 7 ++
tests/index.html | 3 +
tests/suites/default/get/requests.txt | 3 +
16 files changed, 125 insertions(+), 28 deletions(-)
diff --git a/README.txt b/README.txt
index fe425e3..5d7fc70 100644
--- a/README.txt
+++ b/README.txt
@@ -5,7 +5,7 @@ pycsw is an OGC CSW server implementation written in Python.
pycsw fully implements the OpenGIS Catalogue Service Implementation Specification (Catalogue Service for the Web). Initial development started in 2010 (more formally announced in 2011). The project is certified OGC Compliant, and is an OGC Reference Implementation. Since 2015, pycsw is an official OSGeo Project.
-pycsw allows for the publishing and discovery of [[geospatial]] metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU). Existing repositories of geospatial metadata can also be exposed, providing a standards-based metadata and catalogue component of spatial data infrastructures.
+pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU). Existing repositories of geospatial metadata can also be exposed, providing a standards-based metadata and catalogue component of spatial data infrastructures.
pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X).
diff --git a/VERSION.txt b/VERSION.txt
index 227cea2..38f77a6 100644
--- a/VERSION.txt
+++ b/VERSION.txt
@@ -1 +1 @@
-2.0.0
+2.0.1
diff --git a/docs/_templates/indexsidebar.html b/docs/_templates/indexsidebar.html
index 1a91bfc..d47247e 100644
--- a/docs/_templates/indexsidebar.html
+++ b/docs/_templates/indexsidebar.html
@@ -1,9 +1,13 @@
<p>
- <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1325"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Servi [...]
+ <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Servi [...]
</p>
<p>
- <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1325"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Servi [...]
+ <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Servi [...]
+</p>
+
+<p>
+ <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Servi [...]
</p>
<p>
diff --git a/docs/introduction.rst b/docs/introduction.rst
index 94d4dcf..ed415e7 100644
--- a/docs/introduction.rst
+++ b/docs/introduction.rst
@@ -197,6 +197,6 @@ Functions
.. _`HHypermap`: https://github.com/cga-harvard/HHypermap
.. _`Open Data Catalog`: https://github.com/azavea/Open-Data-Catalog/
.. _`CKAN`: http://ckan.org/
-.. _`Compliant`: http://www.opengeospatial.org/resource/products/details/?pid=1325
+.. _`Compliant`: http://www.opengeospatial.org/resource/products/details/?pid=1374
.. _`OAI-PMH`: http://www.openarchives.org/OAI/openarchivesprotocol.html
.. _`GM03`: http://www.geocat.ch/internet/geocat/en/home/documentation/gm03.html
diff --git a/pycsw/__init__.py b/pycsw/__init__.py
index cfef463..d5581df 100644
--- a/pycsw/__init__.py
+++ b/pycsw/__init__.py
@@ -28,4 +28,4 @@
#
# =================================================================
-__version__ = '2.0.0'
+__version__ = '2.0.1'
diff --git a/pycsw/ogc/csw/csw2.py b/pycsw/ogc/csw/csw2.py
index 7a783f2..bd8a11e 100644
--- a/pycsw/ogc/csw/csw2.py
+++ b/pycsw/ogc/csw/csw2.py
@@ -1955,9 +1955,14 @@ class Csw2(object):
self.parent.context.namespaces),
exceptionCode=code, locator=locator)
- etree.SubElement(exception,
+ exception_text = etree.SubElement(exception,
util.nspath_eval('ows:ExceptionText',
- self.parent.context.namespaces)).text = text
+ self.parent.context.namespaces))
+
+ try:
+ exception_text.text = text
+ except ValueError as err:
+ exception_text.text = repr(text)
return node
diff --git a/pycsw/ogc/csw/csw3.py b/pycsw/ogc/csw/csw3.py
index 1c5f2df..587777c 100644
--- a/pycsw/ogc/csw/csw3.py
+++ b/pycsw/ogc/csw/csw3.py
@@ -2043,9 +2043,14 @@ class Csw3(object):
self.parent.context.namespaces),
exceptionCode=code, locator=locator)
- etree.SubElement(exception,
+ exception_text = etree.SubElement(exception,
util.nspath_eval('ows20:ExceptionText',
- self.parent.context.namespaces)).text = text
+ self.parent.context.namespaces))
+
+ try:
+ exception_text.text = text
+ except ValueError as err:
+ exception_text.text = repr(text)
return node
diff --git a/pycsw/plugins/repository/hhypermap/hhypermap.py b/pycsw/plugins/repository/hhypermap/hhypermap.py
index ac61c8f..c4cf9ba 100644
--- a/pycsw/plugins/repository/hhypermap/hhypermap.py
+++ b/pycsw/plugins/repository/hhypermap/hhypermap.py
@@ -28,16 +28,20 @@
#
# =================================================================
+import inspect
+
from django.db import models
from django.db import connection
from django.db.models import Avg, Max, Min, Count
from django.conf import settings
from pycsw.core import util
-from hypermap.aggregator.models import Layer, Service
+from hypermap.aggregator.models import Catalog, Layer, Service, Endpoint
+from hypermap.aggregator.utils import create_layer_from_metadata_xml
HYPERMAP_SERVICE_TYPES = {
# 'HHypermap enum': 'CSW enum'
+ 'http://www.opengis.net/cat/csw/2.0.2': 'OGC:CSW',
'http://www.opengis.net/wms': 'OGC:WMS',
'http://www.opengis.net/wmts/1.0': 'OGC:WMTS',
'https://wiki.osgeo.org/wiki/TMS': 'OSGeo:TMS',
@@ -161,35 +165,79 @@ class HHypermapRepository(object):
else: # no sort
return [str(total), query.all()[startposition:startposition+int(maxrecords)]]
- def insert(self, resourcetype, source):
+ def insert(self, resourcetype, source, insert_date=None):
''' Insert a record into the repository '''
- if resourcetype not in HYPERMAP_SERVICE_TYPES.keys():
- raise RuntimeError('Unsupported Service Type')
+ caller = inspect.stack()[1][3]
+
+ if caller == 'transaction': # insert of Layer
+ hhclass = 'Layer'
+ source = resourcetype
+ resourcetype = resourcetype.csw_schema
+ else: # insert of service
+ hhclass = 'Service'
+ if resourcetype not in HYPERMAP_SERVICE_TYPES.keys():
+ raise RuntimeError('Unsupported Service Type')
- return self._insert_or_update(resourcetype, source, mode='insert')
+ return self._insert_or_update(resourcetype, source, mode='insert', hhclass=hhclass)
- def _insert_or_update(self, resourcetype, source, mode='insert'):
+ def _insert_or_update(self, resourcetype, source, mode='insert', hhclass='Service'):
''' Insert or update a record in the repository '''
+ keywords = []
try:
- s = Service(type=HYPERMAP_SERVICE_TYPES[resourcetype], url=source)
- s.save()
+ if hhclass == 'Layer':
+ # TODO: better way of figuring out duplicates
+ match = Layer.objects.filter(name=source.name, title=source.title, abstract=source.abstract, is_monitored=False)
+ matches = match.all()
+ if matches:
+ if mode == 'insert':
+ raise RuntimeError('HHypermap error: Layer %d \'%s\' already exists' % (matches[0].id, source.title))
+ elif mode == 'update':
+ match.update(
+ name=source.name,
+ title=source.title,
+ abstract=source.abstract,
+ is_monitored=False,
+ xml=source.xml,
+ wkt_geometry=source.wkt_geometry,
+ anytext=util.get_anytext([source.title, source.abstract, source.keywords_csv])
+ )
+
+ res, keywords = create_layer_from_metadata_xml(resourcetype, source.xml, monitor=False)
+ else:
+ if resourcetype == 'http://www.opengis.net/cat/csw/2.0.2':
+ res = Endpoint(url=source)
+ else:
+ res = Service(type=HYPERMAP_SERVICE_TYPES[resourcetype], url=source)
+
+ if self.filter is not None:
+ res.catalog = Catalog.objects.get(id=int(self.filter.split()[-1]))
+ res.save()
+ if keywords:
+ for kw in keywords:
+ res.keywords.add(kw)
except Exception as err:
raise RuntimeError('HHypermap error: %s' % err)
# return a list of ids that were inserted or updated
ids = []
- for res in Service.objects.filter(url=source).all():
- ids.append({'identifier': res.id_string, 'title': res.title})
+
+ if hhclass == 'Layer':
+ ids.append({'identifier': res.uuid, 'title': res.title })
+ else:
+ if resourcetype == 'http://www.opengis.net/cat/csw/2.0.2':
+ for res in Endpoint.objects.filter(url=source).all():
+ ids.append({'identifier': res.uuid, 'title': res.url})
+ else:
+ for res in Service.objects.filter(url=source).all():
+ ids.append({'identifier': res.uuid, 'title': res.title})
+
return ids
-
+
def delete(self, constraint):
''' Delete a record from the repository '''
- # FIXME: id_string is a virtual property and cannot be queried against
- constraint['where'] = constraint['where'].replace('id_string', 'id')
-
results = self._get_repo_filter(Service.objects).extra(where=[constraint['where']],
params=constraint['values']).all()
deleted = len(results)
diff --git a/pycsw/server.py b/pycsw/server.py
index 3562d4d..f6081f9 100644
--- a/pycsw/server.py
+++ b/pycsw/server.py
@@ -530,7 +530,10 @@ class Csw(object):
# test version
kvp_version = self.kvp.get('version', '')
- kvp_version_integer = util.get_version_integer(kvp_version)
+ try:
+ kvp_version_integer = util.get_version_integer(kvp_version)
+ except Exception as err:
+ kvp_version_integer = 'invalid_value'
if (request != 'GetCapabilities' and
kvp_version_integer != own_version_integer):
error = 1
diff --git a/pycsw/wsgi.py b/pycsw/wsgi.py
index 65eea14..8a20f28 100644
--- a/pycsw/wsgi.py
+++ b/pycsw/wsgi.py
@@ -70,18 +70,23 @@ def application(env, start_response):
if 'PYCSW_CONFIG' in env:
config = env['PYCSW_CONFIG']
+ root = PYCSW_ROOT
+
+ if 'PYCSW_ROOT' in env:
+ root = env['PYCSW_ROOT']
+
if env['QUERY_STRING'].lower().find('config') != -1:
for kvp in env['QUERY_STRING'].split('&'):
if kvp.lower().find('config') != -1:
config = unquote(kvp.split('=')[1])
if not os.path.isabs(config):
- config = os.path.join(PYCSW_ROOT, config)
+ config = os.path.join(root, config)
if 'HTTP_HOST' in env and ':' in env['HTTP_HOST']:
env['HTTP_HOST'] = env['HTTP_HOST'].split(':')[0]
- env['local.app_root'] = PYCSW_ROOT
+ env['local.app_root'] = root
csw = server.Csw(config, env)
diff --git a/requirements.txt b/requirements.txt
index 612db02..b2e7090 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
geolinks==0.2.0
-lxml>=3.3.5
+lxml==3.6.2
OWSLib==0.10.3
pyproj==1.9.3
Shapely==1.3.1
diff --git a/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid1.xml b/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid1.xml
new file mode 100644
index 0000000..4e35a17
--- /dev/null
+++ b/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid1.xml
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- PYCSW_VERSION -->
+<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="en-US" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/ow [...]
+ <ows:Exception exceptionCode="InvalidParameterValue" locator="service">
+ <ows:ExceptionText>'Invalid value for service: CSW\x00. Value MUST be CSW'</ows:ExceptionText>
+ </ows:Exception>
+</ows:ExceptionReport>
diff --git a/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid2.xml b/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid2.xml
new file mode 100644
index 0000000..97ec989
--- /dev/null
+++ b/tests/expected/suites_default_get_Exception-GetRepositoryItem-service-invalid2.xml
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- PYCSW_VERSION -->
+<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="en-US" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/ow [...]
+ <ows:Exception exceptionCode="InvalidParameterValue" locator="service">
+ <ows:ExceptionText>"Invalid value for service: CSW\x00'. Value MUST be CSW"</ows:ExceptionText>
+ </ows:Exception>
+</ows:ExceptionReport>
diff --git a/tests/expected/suites_default_get_Exception-GetRepositoryItem-version-invalid.xml b/tests/expected/suites_default_get_Exception-GetRepositoryItem-version-invalid.xml
new file mode 100644
index 0000000..fd71233
--- /dev/null
+++ b/tests/expected/suites_default_get_Exception-GetRepositoryItem-version-invalid.xml
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- PYCSW_VERSION -->
+<ows20:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:ows20="http://www.opengis.net/ows/2.0" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="3.0.0" xml:lang="en-US" xsi:schemaLocation="http://www.opengis. [...]
+ <ows20:Exception exceptionCode="InvalidParameterValue" locator="version">
+ <ows20:ExceptionText>Invalid value for version: 2.0.2'. Value MUST be 2.0.2 or 3.0.0</ows20:ExceptionText>
+ </ows20:Exception>
+</ows20:ExceptionReport>
diff --git a/tests/index.html b/tests/index.html
index 8db0d2e..b107ca2 100644
--- a/tests/index.html
+++ b/tests/index.html
@@ -429,6 +429,9 @@
<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRecords&typenames=csw:Record&elementsetname=full&maxrecords=">GetRecords-empty-maxrecords</a></li>
<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRepositoryItem&id=urn:uuid:94bc9c83-97f6-4b40-9eb8-a8e8787a5c63">GetRepositoryItem</a></li>
<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRepositoryItem&id=NOTFOUND">Exception-GetRepositoryItem-notfound</a></li>
+<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW%00&version=2.0.2&request=GetRepositoryItem&id=123">Exception-GetRepositoryItem-service-invalid1</a></li>
+<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW%00'&version=2.0.2&request=GetRepositoryItem&id=123">Exception-GetRepositoryItem-service-invalid2</a></li>
+<li><a href="../csw.py?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2'&request=GetRepositoryItem&id=123">Exception-GetRepositoryItem-version-invalid</a></li>
<li><a href="../csw.py?config=tests/suites/manager/default.cfg&service=CSW&version=2.0.2&request=Harvest">Exception-Harvest-missing-resourcetype</a></li>
<li><a href="../csw.py?config=tests/suites/manager/default.cfg&service=CSW&version=2.0.2&request=Harvest&resourcetype=http://www.opengis.net/wms">Exception-Harvest-missing-source</a></li>
<li><a href="../csw.py?config=tests/suites/manager/default.cfg&service=CSW&version=2.0.2&request=Harvest&resourcetype=http://www.opengis.net/wms1234&source=http://demo.pycsw.org/cite/csw">Exception-Harvest-invalid-resourcetype</a></li>
diff --git a/tests/suites/default/get/requests.txt b/tests/suites/default/get/requests.txt
index 14f72e8..c75edbd 100644
--- a/tests/suites/default/get/requests.txt
+++ b/tests/suites/default/get/requests.txt
@@ -9,3 +9,6 @@ GetRecords-filter,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=C
GetRecords-empty-maxrecords,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRecords&typenames=csw:Record&elementsetname=full&maxrecords=
GetRepositoryItem,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRepositoryItem&id=urn:uuid:94bc9c83-97f6-4b40-9eb8-a8e8787a5c63
Exception-GetRepositoryItem-notfound,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2&request=GetRepositoryItem&id=NOTFOUND
+Exception-GetRepositoryItem-service-invalid1,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW%00&version=2.0.2&request=GetRepositoryItem&id=123
+Exception-GetRepositoryItem-service-invalid2,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW%00'&version=2.0.2&request=GetRepositoryItem&id=123
+Exception-GetRepositoryItem-version-invalid,PYCSW_SERVER?config=tests/suites/default/default.cfg&service=CSW&version=2.0.2'&request=GetRepositoryItem&id=123
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-grass/pycsw.git
More information about the Pkg-grass-devel
mailing list