[Python-modules-commits] [sparqlwrapper] 01/11: New upstream version 1.7.6
chrysn
chrysn-guest at moszumanska.debian.org
Wed Nov 16 11:11:31 UTC 2016
This is an automated email from the git hooks/post-receive script.
chrysn-guest pushed a commit to branch master
in repository sparqlwrapper.
commit ecd5c3dc5869e3bc4f0cc7f8c41a8a93eb8ab291
Author: chrysn <chrysn at fsfe.org>
Date: Tue Nov 8 15:18:51 2016 +0100
New upstream version 1.7.6
---
.gitignore | 3 +
.travis.yml | 17 +++
AUTHORS.md | 26 +++++
AUTHORS.txt | 23 ----
ChangeLog.txt | 31 ++++-
MANAGEMENT.md | 38 ++++++
MANIFEST.in | 5 +-
Makefile | 6 +-
README.md | 9 +-
SPARQLWrapper/SPARQLExceptions.py | 4 +-
SPARQLWrapper/SmartWrapper.py | 4 +
SPARQLWrapper/Wrapper.py | 235 ++++++++++++++++++++++++++-----------
SPARQLWrapper/__init__.py | 28 +++--
SPARQLWrapper/jsonlayer.py | 164 --------------------------
requirements.txt | 2 +
run_tests_py3.sh | 38 ++++++
setup.py | 70 +++++++----
test/dbpedia.py | 14 ++-
test/wrapper_test.py | 238 ++++++++++++++++++++++++++++++++++----
tests.sh | 13 +++
20 files changed, 645 insertions(+), 323 deletions(-)
diff --git a/.gitignore b/.gitignore
index f4e77d5..61757a4 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,3 +3,6 @@
SPARQLWrapper.egg-info
build
dist
+doc
+.idea
+*.iml
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..66f3ae9
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,17 @@
+# http://travis-ci.org/RDFLib/sparqlwrapper
+
+sudo: false
+
+language: python
+
+python:
+ - 2.7
+ - 3.2
+ - 3.3
+ - 3.4
+ - pypy
+
+install:
+ - python setup.py sdist && pip install ./dist/*
+
+script: ./tests.sh
diff --git a/AUTHORS.md b/AUTHORS.md
new file mode 100644
index 0000000..5bcb36e
--- /dev/null
+++ b/AUTHORS.md
@@ -0,0 +1,26 @@
+# Authors
+
+* Ivan Herman ([@iherman](http://github.com/iherman))
+* Sergio Fernández ([@wikier](http://github.com/wikier))
+* Carlos Tejo ([@dayures](http://github.com/dayures))
+* Alexey Zakhlestin ([@indeyets](http://github.com/indeyets))
+
+# Contributors
+
+* Obey Arthur Liu ([@ArthurLiu](http://github.com/ArthurLiu)): different patches
+* Christopher Lenz ([@cmlenz](http://github.com/cmlenz)): feature to allow developers to choose the json module
+* Pēteris Caune ([@cuu508](http://github.com/cuu508)): great feedback and patches
+* Bogdan Benea ([bugdone at users.sourceforge.net](mailto:bugdone at users.sourceforge.net)), patch for the query regular expresion
+* William Waites ([@wwaites](http://github.com/wwaites)): patches for RDFLib3
+* Christoph Burgmer ([@cburgmer](http://github.com/cburgmer)): patches for RDFLib3
+* Thomas Kluyver ([@takluyver](http://github.com/takluyver)): patches for Python 3.x
+* Diego Berrueta ([@berrueta](http://github.com/berrueta)): new function for printing results as table
+* Olivier Berger ([@olberger](http://github.com/olberger)): patch regarding raw response for unknown formats
+* Benjamin Cogrel ([@bcogrel](http://github.com/bcogrel)): standard query types
+* Urs Holzer ([@uholzer](http://github.com/uholzer)): features, patches and testing
+* Alf Lervåg ([@alf](http://github.com/alf)): setup patch
+* Nolan Nichols ([@nicholsn](http://github.com/nicholsn)): http disgest auth support
+* Kevin Turner ([@keturn](https://github.com/keturn)): `SmartWrapper.Value.__repr__()` implementation
+* Marcelo Jorge Vieira ([@marcelometal](https://github.com/marcelometal)): typos
+* Trevor Andersen ([@trevorandersen](https://github.com/trevorandersen): patches for Python 3.x
+
diff --git a/AUTHORS.txt b/AUTHORS.txt
deleted file mode 100644
index 0d6740d..0000000
--- a/AUTHORS.txt
+++ /dev/null
@@ -1,23 +0,0 @@
-
-Authors
--------
-
- Ivan Herman <ivan at w3.org>
- Sergio Fernández <sergio.fernandez at salzburgresearch.at>
- Carlos Tejo <carlos.tejo at fundacionctic.org>
-
-
-Contributors
-------------
-
- Obey Arthur Liu <arthur at milliways.fr>, different patches
- Christopher Lenz <cmlenz at gmx.de>, feature to allow developers to choose the json module
- Pēteris Caune <cuu508 at gmail.com>, great feedback and patches
- Bogdan Benea <bugdone at users.sourceforge.net>, patch for the query regular expresion
- William Waites <ww at styx.org>, patches for RDFLib3
- Christoph Burgmer <cburgmer at users.sourceforge.net>, patches for RDFLib3
- Thomas Kluyver <takowl at gmail.com>, patches for Python 3.x
- Diego Berrueta <diego.berrueta at fundacionctic.org>, new function for printing results as table
- Olivier Berger <olivier.berger at telecom-sudparis.eu>, patch regarding raw response for unknown formats
- Alexey Zakhlestin <indeyets at gmail.com>, several patches
-
diff --git a/ChangeLog.txt b/ChangeLog.txt
index 92cfb36..b45db5e 100644
--- a/ChangeLog.txt
+++ b/ChangeLog.txt
@@ -1,7 +1,36 @@
SPARQLWrapper's changelog:
-------------------------
-YYYY-MM-DD 1.6.1 - ...
+2015-12-18 1.7.6 - Removed wrong response encoding (issue #70)
+ - Authorization header bug when using Python 3 (issue #71)
+
+2015-11-19 1.7.5 - Removed pip dependency on setup (issue #69)
+
+2015-11-05 1.7.4 - Fixed packaging (issue #66)
+
+2015-11-05 1.7.3 - Finally fixed the keepalive issue in all Pyhon versions (issue #65)
+ - Removed old JSON layer in favor of the native json module
+
+2015-11-03 1.7.2 - Moved to the new keepalive package (issues #53 and #61)
+
+2015-10-29 1.7.1 - Fixed build in Python 3.x (issue #57)
+
+2015-10-29 1.7.0 - Added support to HTTP Digest Auth Support (issue #45)
+ - Improved print_results showing language tag (xml:lang) and datatype
+ - Updated to RDFLib 4.x
+
+2014-08-26 1.6.4 - Fixed unicode problems on setup (issue #42)
+
+2014-08-26 1.6.3 - Fixed unicode problems with urllib in Python 3 (issue #35)
+ - Restored SPARQLWrapper2 class (issue #36)
+ - Enhanced warning for missing rdflib-jsonld (issue #38)
+ - Fixed build system (issue #39)
+
+2014-07-24 1.6.2 - Fixed query type detection with comments (issue #32)
+
+2014-07-21 1.6.1 - Added missing query types (issue #17)
+ - Added a new method to the API to select the request method to be fully SPARQL 1.1 Protocol compliant (issue #28)
+ - Improved the test suite coverage, including support to run the tests under Python 3.x (issues #20, #24 and #31)
2014-05-09 1.6.0 - Returning raw response in case of unknown content type returned
- Fixed some issues with the last version of the SPARQL 1.1 Update Protocol
diff --git a/MANAGEMENT.md b/MANAGEMENT.md
new file mode 100644
index 0000000..c315eeb
--- /dev/null
+++ b/MANAGEMENT.md
@@ -0,0 +1,38 @@
+# Management documentation
+
+The project keeps a very low-profile on managing, so we try to keep it simple.
+
+## Contributions
+
+Every contributor (patchs, pull requests, new features, etc) gets part of ownership by be mentioned as contributor to the project (`AUTHORS.md`).
+
+## Release
+
+### Software
+
+The release process is quite simple, just few things have to be done:
+
+First, do not forget to update the changelog (`ChangeLog.txt` file). That information could be added later to the release at github.
+
+Then you have to [create a release](https://github.com/blog/1547-release-your-software) by tagging the master branch:
+
+ git tag x.y.z
+ git push --tags
+
+And then upload the release to pypi:
+
+ python setup.py register sdist --formats=gztar,zip bdist_egg upload
+
+Please, don't forget to increment to the next module (`SPARQLWrapper/__init__.py` file).
+
+### Documentation
+
+In order to provide online documentation, some steps need to be accomplished:
+
+1. First, generate the documentation using [epydoc](http://epydoc.sourceforge.net/) using the makefile
+
+ make doc
+
+2. And then upload the documentation generated (`doc` folder) to GitHub Pages (`gh-pages` branch).
+
+3. After that, the online version of the documentation would be available on [GitHub Pages](http://rdflib.github.io/sparqlwrapper/resources/doc).
diff --git a/MANIFEST.in b/MANIFEST.in
index 3ee7608..f0581d1 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,2 +1,5 @@
-include *.txt
+include *.md
exclude Makefile
+include test/*.py
+include scripts/*.py
+include requirements.txt
diff --git a/Makefile b/Makefile
index b43d36d..4447be3 100644
--- a/Makefile
+++ b/Makefile
@@ -1,13 +1,13 @@
# SPARQLWrapper Makefile
NAME=SPARQLWrapper
+VERSION=`python -c "import SPARQLWrapper,sys;sys.stdout.write(SPARQLWrapper.__version__)";`
DESTDIR =
DOCDIR=doc
-PYTHON=python
doc: clean
mkdir -p $(DOCDIR)
- epydoc -v -n $(NAME) -o $(DOCDIR) --html SPARQLWrapper
+ epydoc -v -n "$(NAME) $(VERSION)" -o $(DOCDIR) --html SPARQLWrapper
clean:
- rm -rf $(DOCDIR)
\ No newline at end of file
+ rm -rf $(DOCDIR)
diff --git a/README.md b/README.md
index 30a14ce..0c9084c 100644
--- a/README.md
+++ b/README.md
@@ -1,12 +1,15 @@
# SPARQL Endpoint interface to Python
+[](https://travis-ci.org/RDFLib/sparqlwrapper)
+[](https://pypi.python.org/pypi/SPARQLWrapper)
+
The distribution contains:
-* `SPARQLWrapper`: the Python library. You should copy the directory somewhere into your PYTHONPATH. Alternatively, you can also run the disutils scrips:
+* `SPARQLWrapper`: the Python library. You should copy the directory somewhere into your `PYTHONPATH`. Alternatively, you can also run the distutils scripts:
python setup.py install
-* `test`: some unit tests
+* `test`: some unit and integrations tests
-* `script`: some scripts to run the library against some SPARQL end-points.
+* `script`: some scripts to run the library against some SPARQL endpoints.
diff --git a/SPARQLWrapper/SPARQLExceptions.py b/SPARQLWrapper/SPARQLExceptions.py
index a0b0e18..a5c4dc1 100644
--- a/SPARQLWrapper/SPARQLExceptions.py
+++ b/SPARQLWrapper/SPARQLExceptions.py
@@ -36,7 +36,7 @@ class EndPointInternalError(SPARQLWrapperException):
class QueryBadFormed(SPARQLWrapperException):
"""
- Query Bad Formed exceptions
+ Query Bad Formed exception
"""
msg = "a bad request has been sent to the endpoint, probably the sparql query is bad formed"
@@ -44,7 +44,7 @@ class QueryBadFormed(SPARQLWrapperException):
class EndPointNotFound(SPARQLWrapperException):
"""
- End Point Not Found exceptions
+ End Point Not Found exception
"""
msg = "it was impossible to connect with the endpoint in that address, check if it is correct"
diff --git a/SPARQLWrapper/SmartWrapper.py b/SPARQLWrapper/SmartWrapper.py
index 4a7c6b8..ccc2659 100644
--- a/SPARQLWrapper/SmartWrapper.py
+++ b/SPARQLWrapper/SmartWrapper.py
@@ -61,6 +61,10 @@ class Value(object):
except :
pass
+ def __repr__(self):
+ cls = self.__class__.__name__
+ return "%s(%s:%r)" % (cls, self.type, self.value)
+
######################################################################################
diff --git a/SPARQLWrapper/Wrapper.py b/SPARQLWrapper/Wrapper.py
index ccde1a7..b0a0817 100644
--- a/SPARQLWrapper/Wrapper.py
+++ b/SPARQLWrapper/Wrapper.py
@@ -25,12 +25,14 @@
import urllib
import urllib2
+from urllib2 import urlopen as urlopener # don't change the name: tests override it
import socket
import base64
import re
+import sys
import warnings
-import jsonlayer
+import json
from KeyCaseInsensitiveDict import KeyCaseInsensitiveDict
from SPARQLExceptions import QueryBadFormed, EndPointNotFound, EndPointInternalError
from SPARQLUtils import deprecated
@@ -40,7 +42,7 @@ from SPARQLWrapper import __agent__
JSON = "json"
JSONLD = "json-ld"
XML = "xml"
-TURTLE = "n3"
+TURTLE = "turtle"
N3 = "n3"
RDF = "rdf"
_allowedFormats = [JSON, XML, TURTLE, N3, RDF]
@@ -50,6 +52,11 @@ POST = "POST"
GET = "GET"
_allowedRequests = [POST, GET]
+# Possible HTTP Authentication methods
+BASIC = "BASIC"
+DIGEST = "DIGEST"
+_allowedAuth = [BASIC, DIGEST]
+
# Possible SPARQL/SPARUL query type
SELECT = "SELECT"
CONSTRUCT = "CONSTRUCT"
@@ -57,8 +64,20 @@ ASK = "ASK"
DESCRIBE = "DESCRIBE"
INSERT = "INSERT"
DELETE = "DELETE"
-MODIFY = "MODIFY"
-_allowedQueryTypes = [SELECT, CONSTRUCT, ASK, DESCRIBE, INSERT, DELETE, MODIFY]
+CREATE = "CREATE"
+CLEAR = "CLEAR"
+DROP = "DROP"
+LOAD = "LOAD"
+COPY = "COPY"
+MOVE = "MOVE"
+ADD = "ADD"
+_allowedQueryTypes = [SELECT, CONSTRUCT, ASK, DESCRIBE, INSERT, DELETE, CREATE, CLEAR, DROP,
+ LOAD, COPY, MOVE, ADD]
+
+# Possible methods to perform requests
+URLENCODED = "urlencoded"
+POSTDIRECTLY = "postdirectly"
+_REQUEST_METHODS = [URLENCODED, POSTDIRECTLY]
# Possible output format (mime types) that can be converted by the local script. Unfortunately,
# it does not work by simply setting the return format, because there is still a certain level of confusion
@@ -79,13 +98,13 @@ _RDF_POSSIBLE = _RDF_XML + _RDF_N3
_SPARQL_POSSIBLE = _SPARQL_XML + _SPARQL_JSON + _RDF_XML + _RDF_N3
_SPARQL_PARAMS = ["query"]
-
try:
import rdflib_jsonld
_allowedFormats.append(JSONLD)
_RDF_POSSIBLE = _RDF_POSSIBLE + _RDF_JSONLD
except ImportError:
- warnings.warn("JSON-LD disabled because no suitable support has been found", RuntimeWarning)
+ #warnings.warn("JSON-LD disabled because no suitable support has been found", RuntimeWarning)
+ pass
# This is very ugly. The fact is that the key for the choice of the output format is not defined.
# Virtuoso uses 'format', joseki uses 'output', rasqual seems to use "results", etc. Lee Feigenbaum
@@ -111,8 +130,8 @@ class SPARQLWrapper(object):
@ivar baseURI: the URI of the SPARQL service
"""
pattern = re.compile(r"""
- ((?P<base>(\s*BASE\s*<.*?>)\s*)|(?P<prefixes>(\s*PREFIX\s+.+:\s*<.*?>)\s*))*
- (?P<queryType>(CONSTRUCT|SELECT|ASK|DESCRIBE|INSERT|DELETE|MODIFY))
+ ((?P<base>(\s*BASE\s*<.*?>)\s*)|(?P<prefixes>(\s*PREFIX\s+.+:\s*<.*?>)\s*))*
+ (?P<queryType>(CONSTRUCT|SELECT|ASK|DESCRIBE|INSERT|DELETE|CREATE|CLEAR|DROP|LOAD|COPY|MOVE|ADD))
""", re.VERBOSE | re.IGNORECASE)
def __init__(self, endpoint, updateEndpoint=None, returnFormat=XML, defaultGraph=None, agent=__agent__):
@@ -141,6 +160,7 @@ class SPARQLWrapper(object):
self.agent = agent
self.user = None
self.passwd = None
+ self.http_auth = BASIC
self._defaultGraph = defaultGraph
if returnFormat in _allowedFormats:
@@ -158,18 +178,30 @@ class SPARQLWrapper(object):
self.addParameter("default-graph-uri", self._defaultGraph)
self.returnFormat = self._defaultReturnFormat
self.method = GET
- self.queryType = SELECT
- self.queryString = """SELECT * WHERE{ ?s ?p ?o }"""
+ self.setQuery("""SELECT * WHERE{ ?s ?p ?o }""")
self.timeout = None
+ self.requestMethod = URLENCODED
def setReturnFormat(self, format):
"""Set the return format. If not an allowed value, the setting is ignored.
@param format: Possible values: are L{JSON}, L{XML}, L{TURTLE}, L{N3}, L{RDF} (constants in this module). All other cases are ignored.
- @type format: string
+ @type format: str
"""
if format in _allowedFormats :
self.returnFormat = format
+ elif format == JSONLD:
+ raise ValueError("Current instance does not support JSON-LD; you might want to install the rdflib-json package.")
+ else:
+ raise ValueError("Invalid format '%s'; current instance supports: %s.", (format, ", ".join(_allowedFormats)))
+
+ def supportsReturnFormat(self, format):
+ """Check if a return format is supported.
+
+ @param format: Possible values: are L{JSON}, L{XML}, L{TURTLE}, L{N3}, L{RDF} (constants in this module). All other cases are ignored.
+ @type format: bool
+ """
+ return (format in _allowedFormats)
def setTimeout(self, timeout):
"""Set the timeout (in seconds) to use for querying the endpoint.
@@ -179,6 +211,21 @@ class SPARQLWrapper(object):
"""
self.timeout = int(timeout)
+ def setRequestMethod(self, method):
+ """Set the internal method to use to perform the request for query or
+ update operations, either URL-encoded (C{SPARQLWrapper.URLENCODED}) or
+ POST directly (C{SPARQLWrapper.POSTDIRECTLY}).
+ Further details at U{http://www.w3.org/TR/sparql11-protocol/#query-operation}
+ and U{http://www.w3.org/TR/sparql11-protocol/#update-operation}.
+
+ @param method: method
+ @type method: str
+ """
+ if method in _REQUEST_METHODS:
+ self.requestMethod = method
+ else:
+ warnings.warn("invalid update method '%s'" % method, RuntimeWarning)
+
@deprecated
def addDefaultGraph(self, uri):
"""
@@ -262,7 +309,7 @@ class SPARQLWrapper(object):
except KeyError:
return False
- def setCredentials(self,user,passwd):
+ def setCredentials(self, user, passwd):
"""
Set the credentials for querying the current endpoint
@param user: username
@@ -273,6 +320,20 @@ class SPARQLWrapper(object):
self.user = user
self.passwd = passwd
+ def setHTTPAuth(self, auth):
+ """
+ Set the HTTP Authentication type (Basic or Digest)
+ @param auth: auth type
+ @type auth: string
+ """
+ if not isinstance(auth, str):
+ raise TypeError('setHTTPAuth takes a string')
+ elif auth.upper() in _allowedAuth:
+ self.http_auth = auth.upper()
+ else:
+ valid_types = ", ".join(_allowedAuth)
+ raise ValueError("Value should be one of {0}".format(valid_types))
+
def setQuery(self, query):
"""
Set the SPARQL query text. Note: no check is done on the validity of the query
@@ -282,6 +343,21 @@ class SPARQLWrapper(object):
@type query: string
@bug: #2320024
"""
+ if sys.version < '3': # have to write it like this, for 2to3 compatibility
+ if isinstance(query, unicode):
+ pass
+ elif isinstance(query, str):
+ query = query.decode('utf-8')
+ else:
+ raise TypeError('setQuery takes either unicode-strings or utf-8 encoded byte-strings')
+ else:
+ if isinstance(query, str):
+ pass
+ elif isinstance(query, bytes):
+ query = query.decode('utf-8')
+ else:
+ raise TypeError('setQuery takes either unicode-strings or utf-8 encoded byte-strings')
+
self.queryString = query
self.queryType = self._parseQueryType(query)
@@ -299,6 +375,8 @@ class SPARQLWrapper(object):
@rtype: string
"""
try:
+ query = query if type(query)==str else query.encode('ascii', 'ignore')
+ query = re.sub(re.compile("#.*?\n" ), "" , query) # remove all occurance singleline comments (issue #32)
r_queryType = self.pattern.search(query).group("queryType").upper()
except AttributeError:
warnings.warn("not detected query type for query '%s'" % query.replace("\n", " "), RuntimeWarning)
@@ -319,21 +397,21 @@ class SPARQLWrapper(object):
def setUseKeepAlive(self):
"""Make urllib2 use keep-alive.
- @raise ImportError: when could not be imported urlgrabber.keepalive.HTTPHandler
+ @raise ImportError: when could not be imported keepalive.HTTPHandler
"""
try:
- from urlgrabber.keepalive import HTTPHandler
+ from keepalive import HTTPHandler
keepalive_handler = HTTPHandler()
opener = urllib2.build_opener(keepalive_handler)
urllib2.install_opener(opener)
except ImportError:
- warnings.warn("urlgrabber not installed in the system. The execution of this method has no effect.")
+ warnings.warn("keepalive support not available, so the execution of this method has no effect")
def isSparqlUpdateRequest(self):
""" Returns TRUE if SPARQLWrapper is configured for executing SPARQL Update request
@return: bool
"""
- return self.queryType in [INSERT, DELETE, MODIFY]
+ return self.queryType in [INSERT, DELETE, CREATE, CLEAR, DROP, LOAD, COPY, MOVE, ADD]
def isSparqlQueryRequest(self):
""" Returns TRUE if SPARQLWrapper is configured for executing SPARQL Query request
@@ -341,35 +419,29 @@ class SPARQLWrapper(object):
"""
return not self.isSparqlUpdateRequest()
- def _getRequestParameters(self):
- queryParameters = self.parameters.copy()
+ def _getRequestEncodedParameters(self, query=None):
+ query_parameters = self.parameters.copy()
- if self.isSparqlUpdateRequest():
- queryParameters["update"] = [self.queryString]
- else:
- queryParameters["query"] = [self.queryString]
+ if query and type(query) == tuple and len(query) == 2:
+ #tuple ("query"/"update", queryString)
+ query_parameters[query[0]] = [query[1]]
# This is very ugly. The fact is that the key for the choice of the output format is not defined.
# Virtuoso uses 'format',sparqler uses 'output'
# However, these processors are (hopefully) oblivious to the parameters they do not understand.
# So: just repeat all possibilities in the final URI. UGLY!!!!!!!
for f in _returnFormatSetting:
- queryParameters[f] = [self.returnFormat]
+ query_parameters[f] = [self.returnFormat]
- utfQueryParameters = {}
+ pairs = (
+ "%s=%s" % (
+ urllib.quote_plus(param.encode('UTF-8'), safe='/'),
+ urllib.quote_plus(value.encode('UTF-8'), safe='/')
+ )
+ for param, values in query_parameters.items() for value in values
+ )
- for k, vs in queryParameters.items():
- encodedValues = []
-
- for v in vs:
- if isinstance(v, unicode):
- encodedValues.append(v.encode('utf-8'))
- else:
- encodedValues.append(v)
-
- utfQueryParameters[k] = encodedValues
-
- return utfQueryParameters
+ return '&'.join(pairs)
def _getAcceptHeader(self):
if self.queryType in [SELECT, ASK]:
@@ -379,7 +451,7 @@ class SPARQLWrapper(object):
acceptHeader = ",".join(_SPARQL_JSON)
else:
acceptHeader = ",".join(_ALL)
- elif self.queryType in [INSERT, DELETE, MODIFY]:
+ elif self.queryType in [INSERT, DELETE]:
acceptHeader = "*/*"
else:
if self.returnFormat == N3 or self.returnFormat == TURTLE:
@@ -397,24 +469,56 @@ class SPARQLWrapper(object):
C{urllib2.Request} object of the urllib2 Python library
@return: request
"""
+ request = None
+
if self.isSparqlUpdateRequest():
+ #protocol details at http://www.w3.org/TR/sparql11-protocol/#update-operation
uri = self.updateEndpoint
+
+ if self.method != POST:
+ warnings.warn("update operations MUST be done by POST")
+
+ if self.requestMethod == POSTDIRECTLY:
+ request = urllib2.Request(uri + "?" + self._getRequestEncodedParameters())
+ request.add_header("Content-Type", "application/sparql-update")
+ request.data = self.queryString.encode('UTF-8')
+ else: # URL-encoded
+ request = urllib2.Request(uri)
+ request.add_header("Content-Type", "application/x-www-form-urlencoded")
+ request.data = self._getRequestEncodedParameters(("update", self.queryString)).encode('ascii')
else:
+ #protocol details at http://www.w3.org/TR/sparql11-protocol/#query-operation
uri = self.endpoint
- encodedParameters = urllib.urlencode(self._getRequestParameters(), True)
-
- if self.method == POST:
- request = urllib2.Request(uri)
- request.add_header("Content-Type", "application/x-www-form-urlencoded")
- request.add_data(encodedParameters)
- else: # GET
- request = urllib2.Request(uri + "?" + encodedParameters)
+ if self.method == POST:
+ if self.requestMethod == POSTDIRECTLY:
+ request = urllib2.Request(uri + "?" + self._getRequestEncodedParameters())
+ request.add_header("Content-Type", "application/sparql-query")
+ request.data = self.queryString.encode('UTF-8')
+ else: # URL-encoded
+ request = urllib2.Request(uri)
+ request.add_header("Content-Type", "application/x-www-form-urlencoded")
+ request.data = self._getRequestEncodedParameters(("query", self.queryString)).encode('ascii')
+ else: # GET
+ request = urllib2.Request(uri + "?" + self._getRequestEncodedParameters(("query", self.queryString)))
request.add_header("User-Agent", self.agent)
request.add_header("Accept", self._getAcceptHeader())
if self.user and self.passwd:
- request.add_header("Authorization", "Basic " + base64.encodestring("%s:%s" % (self.user, self.passwd)))
+ if self.http_auth == BASIC:
+ credentials = "%s:%s" % (self.user, self.passwd)
+ request.add_header("Authorization", "Basic %s" % base64.b64encode(credentials.encode('utf-8')).decode('utf-8'))
+ elif self.http_auth == DIGEST:
+ realm = "SPARQL"
+ pwd_mgr = urllib2.HTTPPasswordMgr()
+ pwd_mgr.add_password(realm, uri, self.user, self.passwd)
+ opener = urllib2.build_opener()
+ opener.add_handler(urllib2.HTTPDigestAuthHandler(pwd_mgr))
+ urllib2.install_opener(opener)
+ else:
+ valid_types = ", ".join(_allowedAuth)
+ raise NotImplementedError("Expecting one of: {0}, but received: {1}".format(valid_types,
+ self.http_auth))
return request
@@ -424,10 +528,13 @@ class SPARQLWrapper(object):
@return: tuples with the raw request plus the expected format
"""
- if (self.timeout): socket.setdefaulttimeout(self.timeout)
+ if self.timeout:
+ socket.setdefaulttimeout(self.timeout)
+
request = self._createRequest()
+
try:
- response = urllib2.urlopen(request)
+ response = urlopener(request)
return response, self.returnFormat
except urllib2.HTTPError, e:
if e.code == 400:
@@ -524,20 +631,6 @@ class QueryResult(object):
"""Method for the standard iterator."""
return self.response.next()
- @staticmethod
- def setJSONModule(module):
- """Set the Python module for encoding JSON data. If not an allowed value, the setting is ignored.
- JSON modules supported:
- - ``simplejson``: http://code.google.com/p/simplejson/
- - ``cjson``: http://pypi.python.org/pypi/python-cjson
- - ``json``: This is the version of ``simplejson`` that is bundled with the
- Python standard library since version 2.6
- (see http://docs.python.org/library/json.html)
- @param module: Possible values: are L{simplejson}, L{cjson}, L{json}. All other cases raise a ValueError exception.
- @type module: string
- """
- jsonlayer.use(module)
-
def _convertJSON(self):
"""
Convert a JSON result into a Python dict. This method can be overwritten in a subclass
@@ -545,7 +638,7 @@ class QueryResult(object):
@return: converted result
@rtype: Python dictionary
"""
- return jsonlayer.decode(self.response.read().decode("utf-8"))
+ return json.loads(self.response.read().decode("utf-8"))
def _convertXML(self):
"""
@@ -650,7 +743,8 @@ class QueryResult(object):
for result in results["results"]["bindings"] :
index = 0
for var in results["head"]["vars"] :
- print result[var]["value"].ljust(width[index]),"|",
+ result = self.__get_prettyprint_string_sparql_var_result(result[var])
+ print result.ljust(width[index]),"|",
index += 1
print
@@ -661,6 +755,17 @@ class QueryResult(object):
for result in results["results"]["bindings"] :
index = 0
for var in results["head"]["vars"] :
- width[index] = max(width[index], len(result[var]["value"]))
+ result = self.__get_prettyprint_string_sparql_var_result(result[var])
+ width[index] = max(width[index], len(result))
index =+ 1
return width
+
+ def __get_prettyprint_string_sparql_var_result(self, result):
+ value = result["value"]
+ lang = result.get("xml:lang", None)
+ datatype = result.get("datatype",None)
+ if lang is not None:
+ value+="@"+lang
+ if datatype is not None:
+ value+=" ["+datatype+"]"
+ return value
diff --git a/SPARQLWrapper/__init__.py b/SPARQLWrapper/__init__.py
index 92637f9..a83b26a 100644
--- a/SPARQLWrapper/__init__.py
+++ b/SPARQLWrapper/__init__.py
@@ -1,6 +1,6 @@
# -*- coding: utf8 -*-
-"""
+u"""
This is a wrapper around a SPARQL service. It helps in creating the query URI and,
possibly, convert the result into a more managable format.
@@ -163,28 +163,32 @@ The package was greatly inspired by U{Lee Feigenbaum's similar package for Javas
@requires: U{RDFLib<http://rdflib.net>} package.
"""
-__version__ = "1.6.0"
+__version__ = "1.7.6"
"""The version of SPARQLWrapper"""
-__authors__ = u"Ivan Herman, Sergio Fernández, Carlos Tejo Alonso"
+__authors__ = "Ivan Herman, Sergio Fernández, Carlos Tejo Alonso, Alexey Zakhlestin"
"""The primary authors of SPARQLWrapper"""
-__license__ = u'W3C® SOFTWARE NOTICE AND LICENSE, http://www.w3.org/Consortium/Legal/copyright-software'
+__license__ = "W3C® SOFTWARE NOTICE AND LICENSE, http://www.w3.org/Consortium/Legal/copyright-software"
"""The license governing the use and distribution of SPARQLWrapper"""
-__url__ = 'http://sparql-wrapper.sourceforge.net/'
+__url__ = "http://rdflib.github.io/sparqlwrapper"
"""The URL for SPARQLWrapper's homepage"""
-__contact__ = 'sparql-wrapper-devel at lists.sourceforge.net'
-"""Mail list to contact to other people SPARQLWrappers folks and the developers"""
+__contact__ = "rdflib-dev at googlegroups.com"
+"""Mail list to contact to other people RDFLib and SPARQLWrappers folks and developers"""
-__date__ = "2012-08-28"
+__date__ = "2015-12-18"
"""Last update"""
-__agent__ = "sparqlwrapper %s (http://sparql-wrapper.sourceforge.net/)" % __version__
+__agent__ = "sparqlwrapper %s (rdflib.github.io/sparqlwrapper)" % __version__
-from Wrapper import SPARQLWrapper, XML, JSON, TURTLE, N3, JSONLD, RDF, GET, POST, SELECT, CONSTRUCT, ASK, DESCRIBE
-from Wrapper import INSERT, DELETE, MODIFY
-from SmartWrapper import SPARQLWrapper2
+from Wrapper import SPARQLWrapper
+from Wrapper import XML, JSON, TURTLE, N3, JSONLD, RDF
+from Wrapper import GET, POST
+from Wrapper import SELECT, CONSTRUCT, ASK, DESCRIBE, INSERT, DELETE
+from Wrapper import URLENCODED, POSTDIRECTLY
+from Wrapper import BASIC, DIGEST
+from SmartWrapper import SPARQLWrapper2
diff --git a/SPARQLWrapper/jsonlayer.py b/SPARQLWrapper/jsonlayer.py
deleted file mode 100644
index 00664a3..0000000
--- a/SPARQLWrapper/jsonlayer.py
+++ /dev/null
@@ -1,164 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# Copyright (C) 2009 Christopher Lenz
-# All rights reserved.
-#
-
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions
-# are met:
-
-# 1. Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-# 2. Redistributions in binary form must reproduce the above copyright
-# notice, this list of conditions and the following disclaimer in
-# the documentation and/or other materials provided with the
-# distribution.
-# 3. The name of the author may not be used to endorse or promote
-# products derived from this software without specific prior
-# written permission.
-
-# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-# OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-# GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
-# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
-# IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-"""Thin abstraction layer over the different available modules for decoding
-and encoding JSON data.
-
-This module currently supports the following JSON modules:
- - ``simplejson``: http://code.google.com/p/simplejson/
- - ``cjson``: http://pypi.python.org/pypi/python-cjson
- - ``json``: This is the version of ``simplejson`` that is bundled with the
- Python standard library since version 2.6
- (see http://docs.python.org/library/json.html)
-
-The default behavior is to use ``simplejson`` if installed, and otherwise
-fallback to the standard library module. To explicitly tell SPARQLWrapper
-which module to use, invoke the `use()` function with the module name::
-
- import jsonlayer
- jsonlayer.use('cjson')
-
-In addition to choosing one of the above modules, you can also configure
-SPARQLWrapper to use custom decoding and encoding functions::
-
- import jsonlayer
- jsonlayer.use(decode=my_decode, encode=my_encode)
-
-"""
-
-__all__ = ['decode', 'encode', 'use']
-
-_initialized = False
-_using = None
-_decode = None
-_encode = None
-
-
-def decode(string):
- """Decode the given JSON string.
-
- :param string: the JSON string to decode
- :type string: basestring
- :return: the corresponding Python data structure
- :rtype: object
- """
- if not _initialized:
- _initialize()
- return _decode(string)
-
-
-def encode(obj):
- """Encode the given object as a JSON string.
-
- :param obj: the Python data structure to encode
- :type obj: object
- :return: the corresponding JSON string
- :rtype: basestring
- """
- if not _initialized:
- _initialize()
- return _encode(obj)
-
-
-def use(module=None, decode=None, encode=None):
- """Set the JSON library that should be used, either by specifying a known
- module name, or by providing a decode and encode function.
-
- The modules "simplejson", "cjson", and "json" are currently supported for
- the ``module`` parameter.
-
- If provided, the ``decode`` parameter must be a callable that accepts a
- JSON string and returns a corresponding Python data structure. The
- ``encode`` callable must accept a Python data structure and return the
- corresponding JSON string. Exceptions raised by decoding and encoding
- should be propagated up unaltered.
-
- :param module: the name of the JSON library module to use, or the module object itself
- :type module: str or module
- :param decode: a function for decoding JSON strings
- :type decode: callable
- :param encode: a function for encoding objects as JSON strings
- :type encode: callable
- """
- global _decode, _encode, _initialized, _using
- if module is not None:
- if not isinstance(module, basestring):
- module = module.__name__
- if module not in ('cjson', 'json', 'simplejson'):
- raise ValueError('Unsupported JSON module %s' % module)
- _using = module
- _initialized = False
- else:
- assert decode is not None and encode is not None
- _using = 'custom'
- _decode = decode
- _encode = encode
- _initialized = True
-
-
-def _initialize():
- global _initialized
-
- def _init_simplejson():
- global _decode, _encode
- import simplejson
- _decode = lambda string, loads=simplejson.loads: loads(string)
- _encode = lambda obj, dumps=simplejson.dumps: \
- dumps(obj, allow_nan=False, ensure_ascii=False)
-
- def _init_cjson():
- global _decode, _encode
- import cjson
- _decode = lambda string, decode=cjson.decode: decode(string)
- _encode = lambda obj, encode=cjson.encode: encode(obj)
-
- def _init_stdlib():
- global _decode, _encode
- json = __import__('json', {}, {})
- _decode = lambda string, loads=json.loads: loads(string)
- _encode = lambda obj, dumps=json.dumps: \
- dumps(obj, allow_nan=False, ensure_ascii=False)
-
- if _using == 'simplejson':
- _init_simplejson()
- elif _using == 'cjson':
- _init_cjson()
- elif _using == 'json':
- _init_stdlib()
- elif _using != 'custom':
- try:
- _init_simplejson()
- except ImportError:
- _init_stdlib()
- _initialized = True
-
-
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..cdd0693
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,2 @@
+rdflib>=4.0
+keepalive>=0.5
diff --git a/run_tests_py3.sh b/run_tests_py3.sh
new file mode 100755
index 0000000..c0d6ae4
--- /dev/null
+++ b/run_tests_py3.sh
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+function cmdcheck() {
+ command -v $0 >/dev/null 2>&1 || { echo >&2 "ERROR: command $0 required but it's not installed; aborting..."; exit -1; }
+}
+
+cmdcheck python3
+
+PYTHON_VERSION=`python3 -c "import sys;t='{v[0]}.{v[1]}'.format(v=list(sys.version_info[:2]));sys.stdout.write(t)";`
+
+cmdcheck 2to3-$PYTHON_VERSION
+cmdcheck nosetests-$PYTHON_VERSION
+
+python3 setup.py build
+
+if [ -d build/py3_testing ]; then
... 579 lines suppressed ...
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/sparqlwrapper.git
More information about the Python-modules-commits
mailing list