[Git][debian-gis-team/pyshp][master] 5 commits: Update Source URL to use HTTPS.
Bas Couwenberg
gitlab at salsa.debian.org
Thu Jan 14 15:54:04 GMT 2021
Bas Couwenberg pushed to branch master at Debian GIS Project / pyshp
Commits:
46441afa by Bas Couwenberg at 2021-01-14T16:47:20+01:00
Update Source URL to use HTTPS.
- - - - -
a50eae93 by Bas Couwenberg at 2021-01-14T16:47:32+01:00
New upstream version 2.1.3+ds
- - - - -
127bc656 by Bas Couwenberg at 2021-01-14T16:47:33+01:00
Update upstream source from tag 'upstream/2.1.3+ds'
Update to upstream version '2.1.3+ds'
with Debian dir e4f38c056a90cf583753552ca0e91987e905cfea
- - - - -
20582525 by Bas Couwenberg at 2021-01-14T16:48:27+01:00
New upstream release.
- - - - -
91671c51 by Bas Couwenberg at 2021-01-14T16:49:16+01:00
Set distribution to unstable.
- - - - -
20 changed files:
- PKG-INFO
- README.md
- changelog.txt
- debian/changelog
- debian/copyright
- setup.py
- shapefile.py
- shapefiles/test/balancing.dbf
- shapefiles/test/contextwriter.dbf
- shapefiles/test/dtype.dbf
- shapefiles/test/line.dbf
- shapefiles/test/linem.dbf
- shapefiles/test/linez.dbf
- shapefiles/test/multipatch.dbf
- shapefiles/test/multipoint.dbf
- shapefiles/test/onlydbf.dbf
- shapefiles/test/point.dbf
- shapefiles/test/polygon.dbf
- shapefiles/test/shapetype.dbf
- shapefiles/test/testfile.dbf
Changes:
=====================================
PKG-INFO
=====================================
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: pyshp
-Version: 2.1.2
+Version: 2.1.3
Summary: Pure Python read/write support for ESRI Shapefile format
Home-page: https://github.com/GeospatialPython/pyshp
Author: Joel Lawhead
@@ -79,6 +79,22 @@ Description: # PyShp
# Version Changes
+ ## 2.1.3
+
+ ### Bug fixes:
+
+ - Fix recent bug in geojson hole-in-polygon checking (see #205)
+ - Misc fixes to allow geo interface dump to json (eg dates as strings)
+ - Handle additional dbf date null values, and return faulty dates as unicode (see #187)
+ - Add writer target typecheck
+ - Fix bugs to allow reading shp/shx/dbf separately
+ - Allow delayed shapefile loading by passing no args
+ - Fix error with writing empty z/m shapefile (@mcuprjak)
+ - Fix signed_area() so ignores z/m coords
+ - Enforce writing the 11th field name character as null-terminator (only first 10 are used)
+ - Minor README fixes
+ - Added more tests
+
## 2.1.2
### Bug fixes:
@@ -971,7 +987,7 @@ Description: # PyShp
**Shapefiles with elevation (Z) values**
Elevation shape types are shapes that include an elevation value at each vertex, for instance elevation from a GPS device.
- Shapes with elevation (Z) values are added with the following methods: "pointz", "multipointz", "linez", and "polygonz".
+ Shapes with elevation (Z) values are added with the following methods: "pointz", "multipointz", "linez", and "polyz".
The Z-values are specified by adding a third Z value to each XY coordinate. Z-values do not support the concept of missing data,
but if you omit the third Z-coordinate it will default to 0. Note that Z-type shapes also support measurement (M) values added
as a fourth M-coordinate. This too is optional.
@@ -1143,6 +1159,7 @@ Description: # PyShp
Kyle Kelley
Louis Tiao
Marcin Cuprjak
+ mcuprjak
Micah Cochran
Michael Davis
Michal Čihař
@@ -1152,6 +1169,7 @@ Description: # PyShp
Paulo Ernesto
Raynor Vliegendhart
Razzi Abuissa
+ RosBer97
Ross Rogers
Ryan Brideau
Tobias Megies
=====================================
README.md
=====================================
@@ -70,6 +70,22 @@ part of your geospatial project.
# Version Changes
+## 2.1.3
+
+### Bug fixes:
+
+- Fix recent bug in geojson hole-in-polygon checking (see #205)
+- Misc fixes to allow geo interface dump to json (eg dates as strings)
+- Handle additional dbf date null values, and return faulty dates as unicode (see #187)
+- Add writer target typecheck
+- Fix bugs to allow reading shp/shx/dbf separately
+- Allow delayed shapefile loading by passing no args
+- Fix error with writing empty z/m shapefile (@mcuprjak)
+- Fix signed_area() so ignores z/m coords
+- Enforce writing the 11th field name character as null-terminator (only first 10 are used)
+- Minor README fixes
+- Added more tests
+
## 2.1.2
### Bug fixes:
@@ -962,7 +978,7 @@ Shapefiles containing M-values can be examined in several ways:
**Shapefiles with elevation (Z) values**
Elevation shape types are shapes that include an elevation value at each vertex, for instance elevation from a GPS device.
-Shapes with elevation (Z) values are added with the following methods: "pointz", "multipointz", "linez", and "polygonz".
+Shapes with elevation (Z) values are added with the following methods: "pointz", "multipointz", "linez", and "polyz".
The Z-values are specified by adding a third Z value to each XY coordinate. Z-values do not support the concept of missing data,
but if you omit the third Z-coordinate it will default to 0. Note that Z-type shapes also support measurement (M) values added
as a fourth M-coordinate. This too is optional.
@@ -1134,6 +1150,7 @@ Karim Bahgat
Kyle Kelley
Louis Tiao
Marcin Cuprjak
+mcuprjak
Micah Cochran
Michael Davis
Michal Čihař
@@ -1143,6 +1160,7 @@ pakoun
Paulo Ernesto
Raynor Vliegendhart
Razzi Abuissa
+RosBer97
Ross Rogers
Ryan Brideau
Tobias Megies
=====================================
changelog.txt
=====================================
@@ -1,4 +1,20 @@
+VERSION 2.1.3
+
+2021-01-14
+ Bug fixes:
+ * Fix recent bug in geojson hole-in-polygon checking (see #205)
+ * Misc fixes to allow geo interface dump to json (eg dates as strings)
+ * Handle additional dbf date null values, and return faulty dates as unicode (see #187)
+ * Add writer target typecheck
+ * Fix bugs to allow reading shp/shx/dbf separately
+ * Allow delayed shapefile loading by passing no args
+ * Fix error with writing empty z/m shapefile (@mcuprjak)
+ * Fix signed_area() so ignores z/m coords
+ * Enforce writing the 11th field name character as null-terminator (only first 10 are used)
+ * Minor README fixes
+ * Added more tests
+
VERSION 2.1.2
2020-09-10
=====================================
debian/changelog
=====================================
@@ -1,11 +1,13 @@
-pyshp (2.1.2+ds-2) UNRELEASED; urgency=medium
+pyshp (2.1.3+ds-1) unstable; urgency=medium
* Team upload.
+ * New upstream release.
* Bump watch file version to 4.
* Update lintian overrides.
* Bump Standards-Version to 4.5.1, no changes.
+ * Update Source URL to use HTTPS.
- -- Bas Couwenberg <sebastic at debian.org> Fri, 06 Nov 2020 19:49:59 +0100
+ -- Bas Couwenberg <sebastic at debian.org> Thu, 14 Jan 2021 16:49:08 +0100
pyshp (2.1.2+ds-1) unstable; urgency=medium
=====================================
debian/copyright
=====================================
@@ -1,7 +1,7 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: PyShp
Upstream-Contact: Joel Lawhead <jlawhead at geospatialpython.com>
-Source: http://pypi.debian.net/pyshp/
+Source: https://pypi.debian.net/pyshp/
Comment: The upstream sources are repacked to excluded the pyshp.egg-info
directory that is automatically removed by dh_clean.
Files-Excluded: pyshp.egg-info/*
=====================================
setup.py
=====================================
@@ -7,7 +7,7 @@ def read_file(file):
return data.decode('utf-8')
setup(name='pyshp',
- version='2.1.2',
+ version='2.1.3',
description='Pure Python read/write support for ESRI Shapefile format',
long_description=read_file('README.md'),
long_description_content_type='text/markdown',
=====================================
shapefile.py
=====================================
@@ -2,11 +2,11 @@
shapefile.py
Provides read and write support for ESRI Shapefiles.
author: jlawhead<at>geospatialpython.com
-version: 2.1.2
+version: 2.1.3
Compatible with Python versions 2.7-3.x
"""
-__version__ = "2.1.2"
+__version__ = "2.1.3"
from struct import pack, unpack, calcsize, error, Struct
import os
@@ -160,7 +160,7 @@ def signed_area(coords):
"""Return the signed area enclosed by a ring using the linear time
algorithm. A value >= 0 indicates a counter-clockwise oriented ring.
"""
- xs, ys = map(list, zip(*coords))
+ xs, ys = map(list, list(zip(*coords))[:2]) # ignore any z or m values
xs.append(xs[1])
ys.append(ys[1])
return sum(xs[i]*(ys[i+1]-ys[i-1]) for i in range(1, len(coords)))/2.0
@@ -237,9 +237,15 @@ def ring_sample(coords, ccw=False):
The orientation of the ring is assumed to be clockwise, unless ccw
(counter-clockwise) is set to True.
"""
- coords = tuple(coords) + (coords[1],) # add the second coordinate to the end to allow checking the last triplet
triplet = []
- for p in coords:
+ def itercoords():
+ # iterate full closed ring
+ for p in coords:
+ yield p
+ # finally, yield the second coordinate to the end to allow checking the last triplet
+ yield coords[1]
+
+ for p in itercoords():
# add point to triplet (but not if duplicate)
if p not in triplet:
triplet.append(p)
@@ -691,12 +697,17 @@ class _Record(list):
"""The index position of the record in the original shapefile"""
return self.__oid
- def as_dict(self):
+ def as_dict(self, date_strings=False):
"""
Returns this Record as a dictionary using the field names as keys
:return: dict
"""
- return dict((f, self[i]) for f, i in self.__field_positions.items())
+ dct = dict((f, self[i]) for f, i in self.__field_positions.items())
+ if date_strings:
+ for k,v in dct.items():
+ if isinstance(v, date):
+ dct[k] = '{:04d}{:02d}{:02d}'.format(v.year, v.month, v.day)
+ return dct
def __repr__(self):
return 'Record #{}: {}'.format(self.__oid, list(self))
@@ -722,7 +733,7 @@ class ShapeRecord(object):
@property
def __geo_interface__(self):
return {'type': 'Feature',
- 'properties': self.record.as_dict(),
+ 'properties': self.record.as_dict(date_strings=True),
'geometry': None if self.shape.shapeType == NULL else self.shape.__geo_interface__}
class Shapes(list):
@@ -785,16 +796,18 @@ class Reader(object):
self._offsets = []
self.shpLength = None
self.numRecords = None
+ self.numShapes = None
self.fields = []
self.__dbfHdrLength = 0
self.__fieldposition_lookup = {}
self.encoding = kwargs.pop('encoding', 'utf-8')
self.encodingErrors = kwargs.pop('encodingErrors', 'strict')
- # See if a shapefile name was passed as an argument
+ # See if a shapefile name was passed as the first argument
if len(args) > 0:
if is_string(args[0]):
self.load(args[0])
return
+ # Otherwise, load from separate shp/shx/dbf args (must be file-like)
if "shp" in kwargs.keys():
if hasattr(kwargs["shp"], "read"):
self.shp = kwargs["shp"]
@@ -803,6 +816,9 @@ class Reader(object):
self.shp.seek(0)
except (NameError, io.UnsupportedOperation):
self.shp = io.BytesIO(self.shp.read())
+ else:
+ raise ShapefileException('The shp arg must be file-like.')
+
if "shx" in kwargs.keys():
if hasattr(kwargs["shx"], "read"):
self.shx = kwargs["shx"]
@@ -811,6 +827,9 @@ class Reader(object):
self.shx.seek(0)
except (NameError, io.UnsupportedOperation):
self.shx = io.BytesIO(self.shx.read())
+ else:
+ raise ShapefileException('The shx arg must be file-like.')
+
if "dbf" in kwargs.keys():
if hasattr(kwargs["dbf"], "read"):
self.dbf = kwargs["dbf"]
@@ -819,10 +838,12 @@ class Reader(object):
self.dbf.seek(0)
except (NameError, io.UnsupportedOperation):
self.dbf = io.BytesIO(self.dbf.read())
+ else:
+ raise ShapefileException('The dbf arg must be file-like.')
+
+ # Load the files
if self.shp or self.dbf:
self.load()
- else:
- raise ShapefileException("Shapefile Reader requires a shapefile or file-like object.")
def __str__(self):
"""
@@ -851,7 +872,38 @@ class Reader(object):
def __len__(self):
"""Returns the number of shapes/records in the shapefile."""
- return self.numRecords
+ if self.dbf:
+ # Preferably use dbf record count
+ if self.numRecords is None:
+ self.__dbfHeader()
+
+ return self.numRecords
+
+ elif self.shp:
+ # Otherwise use shape count
+ if self.shx:
+ # Use index file to get total count
+ if self.numShapes is None:
+ # File length (16-bit word * 2 = bytes) - header length
+ self.shx.seek(24)
+ shxRecordLength = (unpack(">i", self.shx.read(4))[0] * 2) - 100
+ self.numShapes = shxRecordLength // 8
+
+ return self.numShapes
+
+ else:
+ # Index file not available, iterate all shapes to get total count
+ if self.numShapes is None:
+ i = 0
+ for i,shape in enumerate(self.iterShapes()):
+ i += 1
+ self.numShapes = i
+
+ return self.numShapes
+
+ else:
+ # No file loaded yet, treat as 'empty' shapefile
+ return 0
def __iter__(self):
"""Iterates through the shapes/records in the shapefile."""
@@ -862,7 +914,7 @@ class Reader(object):
def __geo_interface__(self):
shaperecords = self.shapeRecords()
fcollection = shaperecords.__geo_interface__
- fcollection['bbox'] = self.bbox
+ fcollection['bbox'] = list(self.bbox)
return fcollection
@property
@@ -1061,15 +1113,16 @@ class Reader(object):
if not shx:
return None
if not self._offsets:
- # File length (16-bit word * 2 = bytes) - header length
- shx.seek(24)
- shxRecordLength = (unpack(">i", shx.read(4))[0] * 2) - 100
- numRecords = shxRecordLength // 8
+ if self.numShapes is None:
+ # File length (16-bit word * 2 = bytes) - header length
+ shx.seek(24)
+ shxRecordLength = (unpack(">i", shx.read(4))[0] * 2) - 100
+ self.numShapes = shxRecordLength // 8
# Jump to the first record.
shx.seek(100)
shxRecords = _Array('i')
# Each offset consists of two nrs, only the first one matters
- shxRecords.fromfile(shx, 2 * numRecords)
+ shxRecords.fromfile(shx, 2 * self.numShapes)
if sys.byteorder != 'big':
shxRecords.byteswap()
self._offsets = [2 * el for el in shxRecords[::2]]
@@ -1200,14 +1253,18 @@ class Reader(object):
value = None
elif typ == 'D':
# date: 8 bytes - date stored as a string in the format YYYYMMDD.
- if value.count(b'0') == len(value): # QGIS NULL is all '0' chars
+ if not value.replace(b'\x00', b'').replace(b' ', b'').replace(b'0', b''):
+ # dbf date field has no official null value
+ # but can check for all hex null-chars, all spaces, or all 0s (QGIS null)
value = None
else:
try:
+ # return as python date object
y, m, d = int(value[:4]), int(value[4:6]), int(value[6:8])
value = date(y, m, d)
except:
- value = value.strip()
+ # if invalid date, just return as unicode string so user can decide
+ value = u(value.strip())
elif typ == 'L':
# logical: 1 byte - initialized to 0x20 (space) otherwise T or F.
if value == b" ":
@@ -1290,6 +1347,8 @@ class Writer(object):
self.shapeType = shapeType
self.shp = self.shx = self.dbf = None
if target:
+ if not is_string(target):
+ raise Exception('The target filepath {} must be of type str/unicode, not {}.'.format(repr(target), type(target)) )
self.shp = self.__getFileObj(os.path.splitext(target)[0] + '.shp')
self.shx = self.__getFileObj(os.path.splitext(target)[0] + '.shx')
self.dbf = self.__getFileObj(os.path.splitext(target)[0] + '.dbf')
@@ -1520,6 +1579,9 @@ class Writer(object):
if self.shapeType in (11,13,15,18):
# Z values are present in Z type
zbox = self.zbox()
+ if zbox is None:
+ # means we have empty shapefile/only null geoms (see commentary on bbox above)
+ zbox = [0,0]
else:
# As per the ESRI shapefile spec, the zbox for non-Z type shapefiles are set to 0s
zbox = [0,0]
@@ -1527,6 +1589,9 @@ class Writer(object):
if self.shapeType in (11,13,15,18,21,23,25,28,31):
# M values are present in M or Z type
mbox = self.mbox()
+ if mbox is None:
+ # means we have empty shapefile/only null geoms (see commentary on bbox above)
+ mbox = [0,0]
else:
# As per the ESRI shapefile spec, the mbox for non-M type shapefiles are set to 0s
mbox = [0,0]
@@ -1563,7 +1628,7 @@ class Writer(object):
name, fieldType, size, decimal = field
name = b(name, self.encoding, self.encodingErrors)
name = name.replace(b' ', b'_')
- name = name.ljust(11).replace(b' ', b'\x00')
+ name = name[:10].ljust(11).replace(b' ', b'\x00')
fieldType = b(fieldType, 'ascii')
size = int(size)
fld = pack('<11sc4xBB14x', name, fieldType, size, decimal)
@@ -1735,7 +1800,10 @@ class Writer(object):
def __shxRecord(self, offset, length):
"""Writes the shx records."""
f = self.__getFileObj(self.shx)
- f.write(pack(">i", offset // 2))
+ try:
+ f.write(pack(">i", offset // 2))
+ except error:
+ raise ShapefileException('The .shp file has reached its file size limit > 4294967294 bytes (4.29 GB). To fix this, break up your file into multiple smaller ones.')
f.write(pack(">i", length))
def record(self, *recordList, **recordDict):
=====================================
shapefiles/test/balancing.dbf
=====================================
Binary files a/shapefiles/test/balancing.dbf and b/shapefiles/test/balancing.dbf differ
=====================================
shapefiles/test/contextwriter.dbf
=====================================
Binary files a/shapefiles/test/contextwriter.dbf and b/shapefiles/test/contextwriter.dbf differ
=====================================
shapefiles/test/dtype.dbf
=====================================
Binary files a/shapefiles/test/dtype.dbf and b/shapefiles/test/dtype.dbf differ
=====================================
shapefiles/test/line.dbf
=====================================
Binary files a/shapefiles/test/line.dbf and b/shapefiles/test/line.dbf differ
=====================================
shapefiles/test/linem.dbf
=====================================
Binary files a/shapefiles/test/linem.dbf and b/shapefiles/test/linem.dbf differ
=====================================
shapefiles/test/linez.dbf
=====================================
Binary files a/shapefiles/test/linez.dbf and b/shapefiles/test/linez.dbf differ
=====================================
shapefiles/test/multipatch.dbf
=====================================
Binary files a/shapefiles/test/multipatch.dbf and b/shapefiles/test/multipatch.dbf differ
=====================================
shapefiles/test/multipoint.dbf
=====================================
Binary files a/shapefiles/test/multipoint.dbf and b/shapefiles/test/multipoint.dbf differ
=====================================
shapefiles/test/onlydbf.dbf
=====================================
Binary files a/shapefiles/test/onlydbf.dbf and b/shapefiles/test/onlydbf.dbf differ
=====================================
shapefiles/test/point.dbf
=====================================
Binary files a/shapefiles/test/point.dbf and b/shapefiles/test/point.dbf differ
=====================================
shapefiles/test/polygon.dbf
=====================================
Binary files a/shapefiles/test/polygon.dbf and b/shapefiles/test/polygon.dbf differ
=====================================
shapefiles/test/shapetype.dbf
=====================================
Binary files a/shapefiles/test/shapetype.dbf and b/shapefiles/test/shapetype.dbf differ
=====================================
shapefiles/test/testfile.dbf
=====================================
Binary files a/shapefiles/test/testfile.dbf and b/shapefiles/test/testfile.dbf differ
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyshp/-/compare/87d91e14f982fcf759e98fab8e19e124b6b55ad0...91671c51016405ddf9f7e77abe8606eacc490713
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyshp/-/compare/87d91e14f982fcf759e98fab8e19e124b6b55ad0...91671c51016405ddf9f7e77abe8606eacc490713
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20210114/92292a93/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list