[Git][debian-gis-team/rasterio][upstream] New upstream version 1.1.4
Bas Couwenberg
gitlab at salsa.debian.org
Thu May 7 14:33:53 BST 2020
Bas Couwenberg pushed to branch upstream at Debian GIS Project / rasterio
Commits:
aee4b2f3 by Bas Couwenberg at 2020-05-07T15:17:39+02:00
New upstream version 1.1.4
- - - - -
26 changed files:
- .travis.yml
- CHANGES.txt
- docs/topics/reproject.rst
- docs/topics/resampling.rst
- rasterio/__init__.py
- rasterio/_base.pyx
- rasterio/_features.pyx
- rasterio/_io.pyx
- rasterio/_warp.pyx
- rasterio/compat.py
- rasterio/errors.py
- rasterio/features.py
- rasterio/io.py
- rasterio/path.py
- rasterio/rio/clip.py
- rasterio/shutil.pyx
- rasterio/vrt.py
- rasterio/warp.py
- requirements-dev.txt
- requirements.txt
- tests/test_env.py
- tests/test_features.py
- tests/test_path.py
- tests/test_rio_convert.py
- tests/test_rio_rasterize.py
- tests/test_rio_warp.py
Changes:
=====================================
.travis.yml
=====================================
@@ -56,7 +56,7 @@ install:
- "if [ \"$GDALVERSION\" == \"master\" -o $(gdal-config --version) == \"$GDALVERSION\" ]; then echo \"Using gdal $GDALVERSION\"; else echo \"NOT using gdal $GDALVERSION as expected; aborting\"; exit 1; fi"
- "python -m pip wheel -r requirements-dev.txt"
- "python -m pip install -r requirements-dev.txt"
- - "GDAL_CONFIG=$GDALINST/gdal-$GDALVERSION/bin/gdal-config python -m pip install --upgrade --force-reinstall --no-use-pep517 -e .[test,plot]"
+ - "GDAL_CONFIG=$GDALINST/gdal-$GDALVERSION/bin/gdal-config python -m pip install --no-deps --force-reinstall --no-use-pep517 -e ."
- "python -m pip install coveralls>=1.1"
- "rio --version"
- "rio --gdal-version"
=====================================
CHANGES.txt
=====================================
@@ -1,13 +1,30 @@
Changes
=======
+1.1.4 (2020-05-07)
+------------------
+
+Bug fixes:
+
+- Raise a more helpful exception when asked to open a dataset in write mode
+ using a read-only driver (#1919).
+- Missing support for geometry collections in the rasterio.features module
+ (#1914) has been fixed (#1915).
+- Support for the int8 data type has been extended to the InMemoryRaster class
+ (#1880).
+- rasterio.plot.show_hist caused a Python 3.8 syntax warning. This was fixed by
+ #1874.
+- Make the strict intersection in rio-clip optional with a --with-complement
+ option (#1907).
+- Add a new as_vsi() method to the Path class for internal use and improve
+ Windows path handling and tests thereof (#1895).
+
1.1.3 (2020-02-24)
------------------
Bug fixes:
- Raise RasterioIOError when errors occur while creating a dataset (#1796).
-- Ensure the output of read_masks contains only 255 and 0 (#1721).
- Filter more categories of invalid features in rasterize (#1815).
- Fall back to use of a DummySession instead of failing when boto3 isn't
available (#1864).
=====================================
docs/topics/reproject.rst
=====================================
@@ -27,7 +27,7 @@ transform.
# degrees N, each pixel covering 15".
rows, cols = src_shape = (512, 512)
d = 1.0/240 # decimal degrees per pixel
- # The following is equivalent to
+ # The following is equivalent to
# A(d, 0, -cols*d/2, 0, -d, rows*d/2).
src_transform = A.translation(-cols*d/2, rows*d/2) * A.scale(d, -d)
src_crs = {'init': 'EPSG:4326'}
@@ -36,13 +36,13 @@ transform.
# Destination: a 1024 x 1024 dataset in Web Mercator (EPSG:3857)
# with origin at 0.0, 0.0.
dst_shape = (1024, 1024)
- dst_transform = [-237481.5, 425.0, 0.0, 237536.4, 0.0, -425.0]
+ dst_transform = A.translation(-237481.5, 237536.4) * A.scale(425.0, -425.0)
dst_crs = {'init': 'EPSG:3857'}
destination = np.zeros(dst_shape, np.uint8)
reproject(
- source,
- destination,
+ source,
+ destination,
src_transform=src_transform,
src_crs=src_crs,
dst_transform=dst_transform,
@@ -64,7 +64,7 @@ Estimating optimal output shape
Rasterio provides a :func:`rasterio.warp.calculate_default_transform()` function to
determine the optimal resolution and transform for the destination raster.
-Given a source dataset in a known coordinate reference system, this
+Given a source dataset in a known coordinate reference system, this
function will return a ``transform, width, height`` tuple which is calculated
by libgdal.
@@ -120,8 +120,6 @@ See ``rasterio/rio/warp.py`` for more complex examples of reprojection based on
new bounds, dimensions, and resolution (as well as a command-line interface
described :ref:`here <warp>`).
-
-
It is also possible to use :func:`~rasterio.warp.reproject()` to create an output dataset zoomed
out by a factor of 2. Methods of the :class:`rasterio.Affine` class help us generate
the output dataset's transform matrix and, thereby, its spatial extent.
@@ -170,4 +168,3 @@ References
----------
.. [1] https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.geometric_transform.html#scipy.ndimage.geometric_transform
-
=====================================
docs/topics/resampling.rst
=====================================
@@ -1,7 +1,8 @@
Resampling
==========
-For details on changing coordinate reference systems, see `Reprojection`.
+For details on changing coordinate reference systems, see
+:doc:`Reprojection <reproject>`.
Up and downsampling
-------------------
@@ -26,43 +27,42 @@ method.
import rasterio
from rasterio.enums import Resampling
+ upscale_factor = 2
+
with rasterio.open("example.tif") as dataset:
+
+ # resample data to target shape
data = dataset.read(
- out_shape=(dataset.height * 2, dataset.width * 2, dataset.count),
+ out_shape=(
+ dataset.count,
+ int(dataset.height * upscale_factor),
+ int(dataset.width * upscale_factor)
+ ),
resampling=Resampling.bilinear
)
-Here is an example of downsampling by a factor of 2 using the average resampling
-method.
-
-.. code-block:: python
-
- with rasterio.open("example.tif") as dataset:
- data = dataset.read(
- out_shape=(dataset.height / 2, dataset.width / 2, dataset.count),
- resampling=Resampling.average
+ # scale image transform
+ transform = dataset.transform * dataset.transform.scale(
+ (dataset.width / data.shape[-1]),
+ (dataset.height / data.shape[-2])
)
-.. note::
-
- After these resolution changing operations, the dataset's resolution and the
- resolution components of its affine *transform* property no longer apply to
- the new arrays.
+Downsampling to 1/2 of the resolution can be done with ``upscale_factor = 1/2``.
Resampling Methods
------------------
-When you change the raster cell grid, you must recalulate the pixel values.
+When you change the raster cell grid, you must recalculate the pixel values.
There is no "correct" way to do this as all methods involve some interpolation.
-The current resampling methods can be found in the `rasterio.enums`_ source.
+The current resampling methods can be found in the
+:class:`rasterio.enums.Resampling()` class.
-Of note, the default ``Resampling.nearest`` method may not be suitable for
-continuous data. In those cases, ``Resampling.bilinear`` and
-``Resampling.cubic`` are better suited. Some specialized statistical
-resampling method exist, e.g. ``Resampling.average``, which may be useful when
+Of note, the default :attr:`~rasterio.enums.Resampling.nearest` method may not
+be suitable for continuous data. In those cases,
+:attr:`~rasterio.enums.Resampling.bilinear` and
+:attr:`~rasterio.enums.Resampling.cubic` are better suited.
+Some specialized statistical resampling method exist, e.g.
+:attr:`~rasterio.enums.Resampling.average`, which may be useful when
certain numerical properties of the data are to be retained.
-
-
-.. _rasterio.enums: https://github.com/mapbox/rasterio/blob/master/rasterio/enums.py#L28
=====================================
rasterio/__init__.py
=====================================
@@ -25,7 +25,7 @@ from rasterio.dtypes import (
bool_, ubyte, sbyte, uint8, int8, uint16, int16, uint32, int32, float32, float64,
complex_, check_dtype)
from rasterio.env import ensure_env_with_credentials, Env
-from rasterio.errors import RasterioIOError
+from rasterio.errors import RasterioIOError, DriverCapabilityError
from rasterio.compat import string_types
from rasterio.io import (
DatasetReader, get_writer_for_path, get_writer_for_driver, MemoryFile)
@@ -40,9 +40,8 @@ import rasterio.coords
import rasterio.enums
import rasterio.path
-
__all__ = ['band', 'open', 'pad', 'Env']
-__version__ = "1.1.3"
+__version__ = "1.1.4"
__gdal_version__ = gdal_version()
# Rasterio attaches NullHandler to the 'rasterio' logger and its
@@ -92,10 +91,6 @@ def open(fp, mode='r', driver=None, width=None, height=None, count=None,
count : int, optional
The count of dataset bands. Required in 'w' or 'w+' modes, it is
ignored in 'r' or 'r+' modes.
- dtype : str or numpy dtype
- The data type for bands. For example: 'uint8' or
- ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
- ignored in 'r' or 'r+' modes.
crs : str, dict, or CRS; optional
The coordinate reference system. Required in 'w' or 'w+' modes,
it is ignored in 'r' or 'r+' modes.
@@ -103,6 +98,10 @@ def open(fp, mode='r', driver=None, width=None, height=None, count=None,
Affine transformation mapping the pixel space to geographic
space. Required in 'w' or 'w+' modes, it is ignored in 'r' or
'r+' modes.
+ dtype : str or numpy dtype
+ The data type for bands. For example: 'uint8' or
+ ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
+ ignored in 'r' or 'r+' modes.
nodata : int, float, or nan; optional
Defines the pixel value to be interpreted as not valid data.
Required in 'w' or 'w+' modes, it is ignored in 'r' or 'r+'
@@ -217,18 +216,26 @@ def open(fp, mode='r', driver=None, width=None, height=None, count=None,
# None.
if mode == 'r':
s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
- elif mode == 'r+':
- s = get_writer_for_path(path)(path, mode, driver=driver, sharing=sharing, **kwargs)
+ elif mode == "r+":
+ s = get_writer_for_path(path, driver=driver)(
+ path, mode, driver=driver, sharing=sharing, **kwargs
+ )
elif mode.startswith("w"):
- s = get_writer_for_driver(driver)(path, mode, driver=driver,
- width=width, height=height,
- count=count, crs=crs,
- transform=transform,
- dtype=dtype, nodata=nodata,
- sharing=sharing,
- **kwargs)
+ writer = get_writer_for_driver(driver)
+ if writer is not None:
+ s = writer(path, mode, driver=driver,
+ width=width, height=height,
+ count=count, crs=crs,
+ transform=transform,
+ dtype=dtype, nodata=nodata,
+ sharing=sharing,
+ **kwargs)
+ else:
+ raise DriverCapabilityError(
+ "Writer does not exist for driver: %s" % str(driver)
+ )
else:
- raise ValueError(
+ raise DriverCapabilityError(
"mode must be one of 'r', 'r+', or 'w', not %s" % mode)
return s
=====================================
rasterio/_base.pyx
=====================================
@@ -31,7 +31,7 @@ from rasterio.errors import (
RasterBlockError, BandOverviewError)
from rasterio.profiles import Profile
from rasterio.transform import Affine, guard_transform, tastes_like_gdal
-from rasterio.path import parse_path, vsi_path
+from rasterio.path import parse_path
from rasterio import windows
include "gdal.pxi"
@@ -60,7 +60,7 @@ def get_dataset_driver(path):
Parameters
----------
- path : rasterio.path.Path
+ path : rasterio.path.Path or str
A remote or local dataset path.
Returns
@@ -70,7 +70,7 @@ def get_dataset_driver(path):
cdef GDALDatasetH dataset = NULL
cdef GDALDriverH driver = NULL
- path = vsi_path(path)
+ path = parse_path(path).as_vsi()
path = path.encode('utf-8')
try:
@@ -179,7 +179,7 @@ cdef class DatasetBase(object):
Parameters
----------
- path : rasterio.path.Path
+ path : rasterio.path.Path or str
Path of the local or remote dataset.
driver : str or list of str
A single driver name or a list of driver names to consider when
@@ -202,7 +202,7 @@ cdef class DatasetBase(object):
self._hds = NULL
if path is not None:
- filename = vsi_path(path)
+ filename = parse_path(path).as_vsi()
# driver may be a string or list of strings. If the
# former, we put it into a list.
=====================================
rasterio/_features.pyx
=====================================
@@ -380,7 +380,7 @@ def _explode(coords):
def _bounds(geometry, north_up=True, transform=None):
- """Bounding box of a GeoJSON geometry.
+ """Bounding box of a GeoJSON geometry, GeometryCollection, or FeatureCollection.
left, bottom, right, top
*not* xmin, ymin, xmax, ymax
@@ -390,7 +390,9 @@ def _bounds(geometry, north_up=True, transform=None):
From Fiona 1.4.8 with updates here to handle feature collections.
TODO: add to Fiona.
"""
+
if 'features' in geometry:
+ # Input is a FeatureCollection
xmins = []
ymins = []
xmaxs = []
@@ -406,7 +408,25 @@ def _bounds(geometry, north_up=True, transform=None):
else:
return min(xmins), max(ymaxs), max(xmaxs), min(ymins)
- else:
+ elif 'geometries' in geometry:
+ # Input is a geometry collection
+ xmins = []
+ ymins = []
+ xmaxs = []
+ ymaxs = []
+ for geometry in geometry['geometries']:
+ xmin, ymin, xmax, ymax = _bounds(geometry)
+ xmins.append(xmin)
+ ymins.append(ymin)
+ xmaxs.append(xmax)
+ ymaxs.append(ymax)
+ if north_up:
+ return min(xmins), min(ymins), max(xmaxs), max(ymaxs)
+ else:
+ return min(xmins), max(ymaxs), max(xmaxs), min(ymins)
+
+ elif 'coordinates' in geometry:
+ # Input is a singular geometry object
if transform is not None:
xyz = list(_explode(geometry['coordinates']))
xyz_px = [transform * point for point in xyz]
@@ -419,6 +439,11 @@ def _bounds(geometry, north_up=True, transform=None):
else:
return min(xyz[0]), max(xyz[1]), max(xyz[0]), min(xyz[1])
+ # all valid inputs returned above, so whatever falls through is an error
+ raise ValueError(
+ "geometry must be a GeoJSON-like geometry, GeometryCollection, "
+ "or FeatureCollection"
+ )
# Mapping of OGR integer geometry types to GeoJSON type names.
GEOMETRY_TYPES = {
=====================================
rasterio/_io.pyx
=====================================
@@ -1,6 +1,5 @@
"""Rasterio input/output."""
-
# cython: boundscheck=False, c_string_type=unicode, c_string_encoding=utf8
from __future__ import absolute_import
@@ -32,7 +31,7 @@ from rasterio.errors import (
)
from rasterio.sample import sample_gen
from rasterio.transform import Affine
-from rasterio.path import parse_path, vsi_path, UnparsedPath
+from rasterio.path import parse_path, UnparsedPath
from rasterio.vrt import _boundless_vrt_doc
from rasterio.windows import Window, intersection
@@ -1052,10 +1051,6 @@ cdef class DatasetWriterBase(DatasetReaderBase):
count : int, optional
The count of dataset bands. Required in 'w' or 'w+' modes, it is
ignored in 'r' or 'r+' modes.
- dtype : str or numpy dtype
- The data type for bands. For example: 'uint8' or
- ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
- ignored in 'r' or 'r+' modes.
crs : str, dict, or CRS; optional
The coordinate reference system. Required in 'w' or 'w+' modes,
it is ignored in 'r' or 'r+' modes.
@@ -1063,6 +1058,10 @@ cdef class DatasetWriterBase(DatasetReaderBase):
Affine transformation mapping the pixel space to geographic
space. Required in 'w' or 'w+' modes, it is ignored in 'r' or
'r+' modes.
+ dtype : str or numpy dtype
+ The data type for bands. For example: 'uint8' or
+ ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
+ ignored in 'r' or 'r+' modes.
nodata : int, float, or nan; optional
Defines the pixel value to be interpreted as not valid data.
Required in 'w' or 'w+' modes, it is ignored in 'r' or 'r+'
@@ -1114,7 +1113,7 @@ cdef class DatasetWriterBase(DatasetReaderBase):
# Make and store a GDAL dataset handle.
filename = path.name
- path = vsi_path(path)
+ path = path.as_vsi()
name_b = path.encode('utf-8')
fname = name_b
@@ -1136,7 +1135,7 @@ cdef class DatasetWriterBase(DatasetReaderBase):
if k.lower() in ['affine']:
continue
- k, v = k.upper(), str(v).upper()
+ k, v = k.upper(), str(v)
if k in ['BLOCKXSIZE', 'BLOCKYSIZE'] and not tiled:
continue
@@ -1746,6 +1745,7 @@ cdef class InMemoryRaster:
cdef OGRSpatialReferenceH osr = NULL
cdef GDALDriverH mdriver = NULL
cdef GDAL_GCP *gcplist = NULL
+ cdef char **options = NULL
if image is not None:
if image.ndim == 3:
@@ -1773,10 +1773,13 @@ cdef class InMemoryRaster:
"in a `with rasterio.Env()` or `with rasterio.open()` "
"block.")
+ if dtype == 'int8':
+ options = CSLSetNameValue(options, 'PIXELTYPE', 'SIGNEDBYTE')
+
datasetname = str(uuid4()).encode('utf-8')
self._hds = exc_wrap_pointer(
GDALCreate(memdriver, <const char *>datasetname, width, height,
- count, <GDALDataType>dtypes.dtype_rev[dtype], NULL))
+ count, <GDALDataType>dtypes.dtype_rev[dtype], options))
if transform is not None:
self.transform = transform
@@ -1816,6 +1819,9 @@ cdef class InMemoryRaster:
CPLFree(srcwkt)
_safe_osr_release(osr)
+ if options != NULL:
+ CSLDestroy(options)
+
self._image = None
if image is not None:
self.write(image)
@@ -1917,10 +1923,6 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
count : int, optional
The count of dataset bands. Required in 'w' or 'w+' modes, it is
ignored in 'r' or 'r+' modes.
- dtype : str or numpy dtype
- The data type for bands. For example: 'uint8' or
- ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
- ignored in 'r' or 'r+' modes.
crs : str, dict, or CRS; optional
The coordinate reference system. Required in 'w' or 'w+' modes,
it is ignored in 'r' or 'r+' modes.
@@ -1928,6 +1930,10 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
Affine transformation mapping the pixel space to geographic
space. Required in 'w' or 'w+' modes, it is ignored in 'r' or
'r+' modes.
+ dtype : str or numpy dtype
+ The data type for bands. For example: 'uint8' or
+ ``rasterio.uint16``. Required in 'w' or 'w+' modes, it is
+ ignored in 'r' or 'r+' modes.
nodata : int, float, or nan; optional
Defines the pixel value to be interpreted as not valid data.
Required in 'w' or 'w+' modes, it is ignored in 'r' or 'r+'
@@ -2005,7 +2011,7 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
# Parse the path to determine if there is scheme-specific
# configuration to be done.
- path = vsi_path(path)
+ path = path.as_vsi()
name_b = path.encode('utf-8')
memdrv = GDALGetDriverByName("MEM")
@@ -2015,6 +2021,9 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
# We've mapped numpy scalar types to GDAL types so see
# if we can crosswalk those.
if hasattr(self._init_dtype, 'type'):
+ if self._init_dtype == 'int8':
+ options = CSLSetNameValue(options, 'PIXELTYPE', 'SIGNEDBYTE')
+
tp = self._init_dtype.type
if tp not in dtypes.dtype_rev:
raise ValueError(
@@ -2026,7 +2035,7 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
self._hds = exc_wrap_pointer(
GDALCreate(memdrv, "temp", self.width, self.height,
- self._count, gdal_dtype, NULL))
+ self._count, gdal_dtype, options))
if self._init_nodata is not None:
for i in range(self._count):
@@ -2097,7 +2106,7 @@ cdef class BufferedDatasetWriterBase(DatasetWriterBase):
# Skip items that are definitely *not* valid driver options.
if k.lower() in ['affine']:
continue
- k, v = k.upper(), str(v).upper()
+ k, v = k.upper(), str(v)
key_b = k.encode('utf-8')
val_b = v.encode('utf-8')
key_c = key_b
=====================================
rasterio/_warp.pyx
=====================================
@@ -25,7 +25,6 @@ from rasterio.errors import (
GDALOptionNotImplementedError,
DriverRegistrationError, CRSError, RasterioIOError,
RasterioDeprecationWarning, WarpOptionsError)
-from rasterio.path import parse_path, vsi_path
from rasterio.transform import Affine, from_bounds, guard_transform, tastes_like_gdal
cimport numpy as np
@@ -443,7 +442,7 @@ def _reproject(
# Set up GDALCreateGenImgProjTransformer2 keyword arguments.
cdef char **imgProjOptions = NULL
- CSLSetNameValue(imgProjOptions, "GCPS_OK", "TRUE")
+ imgProjOptions = CSLSetNameValue(imgProjOptions, "GCPS_OK", "TRUE")
# See http://www.gdal.org/gdal__alg_8h.html#a94cd172f78dbc41d6f407d662914f2e3
# for a list of supported options. I (Sean) don't see harm in
=====================================
rasterio/compat.py
=====================================
@@ -15,7 +15,7 @@ if sys.version_info[0] >= 3: # pragma: no cover
from collections import UserDict
from collections.abc import Iterable, Mapping
from inspect import getfullargspec as getargspec
- from pathlib import Path
+ import pathlib
else: # pragma: no cover
warnings.warn("Python 2 compatibility will be removed after version 1.1", DeprecationWarning)
@@ -28,5 +28,4 @@ else: # pragma: no cover
from UserDict import UserDict
from inspect import getargspec
from collections import Iterable, Mapping
- class Path(object):
- pass
+ pathlib = None
=====================================
rasterio/errors.py
=====================================
@@ -21,6 +21,10 @@ class EnvError(RasterioError):
or modified."""
+class DriverCapabilityError(RasterioError, ValueError):
+ """Raised when a format driver can't a feature such as writing."""
+
+
class DriverRegistrationError(ValueError):
"""Raised when a format driver is requested but is not registered."""
=====================================
rasterio/features.py
=====================================
@@ -373,6 +373,14 @@ def bounds(geometry, north_up=True, transform=None):
return tuple(geometry['bbox'])
geom = geometry.get('geometry') or geometry
+
+ # geometry must be a geometry, GeometryCollection, or FeatureCollection
+ if not ('coordinates' in geom or 'geometries' in geom or 'features' in geom):
+ raise ValueError(
+ "geometry must be a GeoJSON-like geometry, GeometryCollection, "
+ "or FeatureCollection"
+ )
+
return _bounds(geom, north_up=north_up, transform=transform)
=====================================
rasterio/io.py
=====================================
@@ -121,16 +121,16 @@ class MemoryFile(MemoryFileBase):
Other parameters are optional and have the same semantics as the
parameters of `rasterio.open()`.
"""
- vsi_path = UnparsedPath(self.name)
+ mempath = UnparsedPath(self.name)
if self.closed:
raise IOError("I/O operation on closed file.")
if self.exists():
- log.debug("VSI path: {}".format(vsi_path.path))
- return DatasetReader(vsi_path, driver=driver, sharing=sharing, **kwargs)
+ log.debug("VSI path: {}".format(mempath.path))
+ return DatasetReader(mempath, driver=driver, sharing=sharing, **kwargs)
else:
writer = get_writer_for_driver(driver)
- return writer(vsi_path, 'w+', driver=driver, width=width,
+ return writer(mempath, 'w+', driver=driver, width=width,
height=height, count=count, crs=crs,
transform=transform, dtype=dtype,
nodata=nodata, sharing=sharing, **kwargs)
@@ -172,11 +172,11 @@ class ZipMemoryFile(MemoryFile):
-------
A Rasterio dataset object
"""
- vsi_path = UnparsedPath('/vsizip{0}/{1}'.format(self.name, path.lstrip('/')))
+ zippath = UnparsedPath('/vsizip{0}/{1}'.format(self.name, path.lstrip('/')))
if self.closed:
raise IOError("I/O operation on closed file.")
- return DatasetReader(vsi_path, driver=driver, sharing=sharing, **kwargs)
+ return DatasetReader(zippath, driver=driver, sharing=sharing, **kwargs)
def get_writer_for_driver(driver):
@@ -191,7 +191,8 @@ def get_writer_for_driver(driver):
return cls
-def get_writer_for_path(path):
+def get_writer_for_path(path, driver=None):
"""Return the writer class appropriate for the existing dataset."""
- driver = get_dataset_driver(path)
+ if not driver:
+ driver = get_dataset_driver(path)
return get_writer_for_driver(driver)
=====================================
rasterio/path.py
=====================================
@@ -1,13 +1,11 @@
"""Dataset paths, identifiers, and filenames"""
-import os.path
import re
import sys
import attr
-from rasterio.compat import Path as PathlibPath
-from rasterio.compat import string_types, urlparse
+from rasterio.compat import pathlib, string_types, urlparse
from rasterio.errors import PathError
# Supported URI schemes and their mapping to GDAL's VSI suffix.
@@ -34,6 +32,9 @@ REMOTESCHEMES = set([k for k, v in SCHEMES.items() if v in ('curl', 's3', 'oss',
class Path(object):
"""Base class for dataset paths"""
+ def as_vsi(self):
+ return vsi_path(self)
+
@attr.s(slots=True)
class ParsedPath(Path):
@@ -129,13 +130,16 @@ def parse_path(path):
if isinstance(path, Path):
return path
- elif isinstance(path, PathlibPath):
- return ParsedPath(path, None, None)
+ elif pathlib and isinstance(path, pathlib.PurePath):
+ return ParsedPath(path.as_posix(), None, None)
elif isinstance(path, string_types):
- if sys.platform == "win32" and re.match("^[a-zA-Z]\\:", path):
- return UnparsedPath(path)
+ if sys.platform == "win32" and re.match(r"^[a-zA-Z]\:", path):
+ if pathlib:
+ return ParsedPath(pathlib.Path(path).as_posix(), None, None)
+ else:
+ return UnparsedPath(path)
elif path.startswith('/vsi'):
return UnparsedPath(path)
@@ -167,6 +171,7 @@ def vsi_path(path):
Returns
-------
str
+
"""
if isinstance(path, UnparsedPath):
return path.path
=====================================
rasterio/rio/clip.py
=====================================
@@ -50,9 +50,24 @@ projection_projected_opt = click.option(
@projection_projected_opt
@options.overwrite_opt
@options.creation_options
+ at click.option(
+ "--with-complement/--without-complement",
+ default=False,
+ help="Include the relative complement of the raster in the given bounds (giving a larger result), else return results only from the intersection of the raster and the bounds (the default).",
+)
@click.pass_context
-def clip(ctx, files, output, bounds, like, driver, projection,
- overwrite, creation_options):
+def clip(
+ ctx,
+ files,
+ output,
+ bounds,
+ like,
+ driver,
+ projection,
+ overwrite,
+ creation_options,
+ with_complement,
+):
"""Clips a raster using projected or geographic bounds.
\b
@@ -108,8 +123,11 @@ def clip(ctx, files, output, bounds, like, driver, projection,
raise click.UsageError('--bounds or --like required')
bounds_window = src.window(*bounds)
- bounds_window = bounds_window.intersection(
- Window(0, 0, src.width, src.height))
+
+ if not with_complement:
+ bounds_window = bounds_window.intersection(
+ Window(0, 0, src.width, src.height)
+ )
# Get the window with integer height
# and width that contains the bounds window.
@@ -133,6 +151,11 @@ def clip(ctx, files, output, bounds, like, driver, projection,
del out_kwargs['blockysize']
logger.warning("Blockysize removed from creation options to accomodate small output height")
- with rasterio.open(output, 'w', **out_kwargs) as out:
- out.write(src.read(window=out_window,
- out_shape=(src.count, height, width)))
+ with rasterio.open(output, "w", **out_kwargs) as out:
+ out.write(
+ src.read(
+ window=out_window,
+ out_shape=(src.count, height, width),
+ boundless=True,
+ )
+ )
=====================================
rasterio/shutil.pyx
=====================================
@@ -15,7 +15,7 @@ from rasterio._err cimport exc_wrap_int, exc_wrap_pointer
from rasterio.env import ensure_env_with_credentials
from rasterio._err import CPLE_OpenFailedError
from rasterio.errors import DriverRegistrationError, RasterioIOError
-from rasterio.path import parse_path, vsi_path
+from rasterio.path import parse_path
log = logging.getLogger(__name__)
@@ -34,8 +34,8 @@ def exists(path):
cdef GDALDatasetH h_dataset = NULL
- gdal_path = vsi_path(parse_path(path))
- b_path = gdal_path.encode('utf-8')
+ vsipath = parse_path(path).as_vsi()
+ b_path = vsipath.encode('utf-8')
cdef char* c_path = b_path
with nogil:
@@ -110,7 +110,7 @@ def copy(src, dst, driver='GTiff', strict=True, **creation_options):
# Open a new GDAL dataset if src is a string.
if isinstance(src, str):
- if vsi_path(parse_path(src)) == vsi_path(parse_path(dst)):
+ if parse_path(src).as_vsi() == parse_path(dst).as_vsi():
raise RasterioIOError("{} and {} identify the same dataset.".format(src, dst))
src = src.encode('utf-8')
@@ -123,7 +123,7 @@ def copy(src, dst, driver='GTiff', strict=True, **creation_options):
# Try to use the existing GDAL dataset handle otherwise.
else:
- if src.name == vsi_path(parse_path(dst)):
+ if src.name == parse_path(dst).as_vsi():
raise RasterioIOError("{} and {} identify the same dataset.".format(src.name, dst))
src_dataset = (<DatasetReaderBase?>src).handle()
@@ -177,13 +177,13 @@ def copyfiles(src, dst):
src_path = parse_path(src)
dst_path = parse_path(dst)
- if vsi_path(src_path) == vsi_path(dst_path):
+ if src_path.as_vsi() == dst_path.as_vsi():
raise RasterioIOError("{} and {} identify the same dataset.".format(src, dst))
# VFS paths probabaly don't work, but its hard to be completely certain
# so just attempt to use them.
- gdal_src_path = vsi_path(src_path)
- gdal_dst_path = vsi_path(dst_path)
+ gdal_src_path = src_path.as_vsi()
+ gdal_dst_path = dst_path.as_vsi()
b_gdal_src_path = gdal_src_path.encode('utf-8')
b_gdal_dst_path = gdal_dst_path.encode('utf-8')
cdef char* c_gdal_src_path = b_gdal_src_path
@@ -223,7 +223,7 @@ def delete(path, driver=None):
cdef GDALDatasetH h_dataset = NULL
cdef GDALDriverH h_driver = NULL
- gdal_path = vsi_path(parse_path(path))
+ gdal_path = parse_path(path).as_vsi()
b_path = gdal_path.encode('utf-8')
cdef char* c_path = b_path
=====================================
rasterio/vrt.py
=====================================
@@ -7,7 +7,7 @@ from rasterio._warp import WarpedVRTReaderBase
from rasterio.dtypes import _gdal_typename
from rasterio.enums import MaskFlags
from rasterio.env import env_ctx_if_needed
-from rasterio.path import parse_path, vsi_path
+from rasterio.path import parse_path
from rasterio.transform import TransformMethodsMixin
from rasterio.windows import WindowMethodsMixin
@@ -208,7 +208,7 @@ def _boundless_vrt_doc(
sourcefilename = ET.SubElement(complexsource, 'SourceFilename')
sourcefilename.attrib['relativeToVRT'] = "0"
sourcefilename.attrib["shared"] = "0"
- sourcefilename.text = vsi_path(parse_path(src_dataset.name))
+ sourcefilename.text = parse_path(src_dataset.name).as_vsi()
sourceband = ET.SubElement(complexsource, 'SourceBand')
sourceband.text = str(bidx)
sourceproperties = ET.SubElement(complexsource, 'SourceProperties')
@@ -250,7 +250,7 @@ def _boundless_vrt_doc(
sourcefilename = ET.SubElement(simplesource, 'SourceFilename')
sourcefilename.attrib['relativeToVRT'] = "0"
sourcefilename.attrib["shared"] = "0"
- sourcefilename.text = vsi_path(parse_path(src_dataset.name))
+ sourcefilename.text = parse_path(src_dataset.name).as_vsi()
sourceband = ET.SubElement(simplesource, 'SourceBand')
sourceband.text = 'mask,1'
=====================================
rasterio/warp.py
=====================================
@@ -75,7 +75,7 @@ def transform_geom(
Example: CRS({'init': 'EPSG:4326'})
dst_crs: CRS or dict
Target coordinate reference system.
- geom: GeoJSON like dict object
+ geom: GeoJSON like dict object or iterable of GeoJSON like objects.
antimeridian_cutting: bool, optional
If True, cut geometries at the antimeridian, otherwise geometries
will not be cut (default). If False and GDAL is 2.2.0 or newer
@@ -91,8 +91,8 @@ def transform_geom(
Returns
---------
- out: GeoJSON like dict object
- Transformed geometry in GeoJSON dict format
+ out: GeoJSON like dict object or list of GeoJSON like objects.
+ Transformed geometry(s) in GeoJSON dict format
"""
return _transform_geom(
=====================================
requirements-dev.txt
=====================================
@@ -6,7 +6,7 @@ delocate
hypothesis
numpydoc
packaging
-pytest>=3.1.0
+pytest~=4.6.0
pytest-cov>=2.2.0
sphinx
sphinx-rtd-theme
=====================================
requirements.txt
=====================================
@@ -1,9 +1,11 @@
-affine>=1.3.0
-attrs>=16.0.0
+affine~=2.3.0
+attrs>=17.4.0
boto3>=1.2.4
click==7.0
+click-plugins
cligj>=0.5
-enum34;python_version<"3.4"
+enum34; python_version < "3.4"
+matplotlib
numpy>=1.10
-snuggs>=1.4.1
-setuptools>=0.9.8
+snuggs~=1.4.0
+setuptools>=20.0
=====================================
tests/test_env.py
=====================================
@@ -791,9 +791,13 @@ aws_access_key_id = bar
""")
monkeypatch.setenv('AWS_SHARED_CREDENTIALS_FILE', str(credentials_file))
monkeypatch.delenv('AWS_ACCESS_KEY_ID', raising=False)
- # Assert that we don't have any AWS credentials by accident.
- with pytest.raises(Exception):
- rasterio.open("s3://mapbox/rasterio/RGB.byte.tif")
+
+ # To verify that we're unauthenticated, we make a request for an known existing object that will return 404 Not Found.
+ with pytest.raises(Exception) as exc_info:
+ s3 = boto3.client("s3")
+ s3.head_object(Bucket="landsat-pds", Key="L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B1.TIF")
+
+ assert exc_info.value.response["Error"]["Code"] == "403"
with rasterio.Env() as env:
assert env.drivers()
=====================================
tests/test_features.py
=====================================
@@ -48,17 +48,42 @@ def test_bounds_z():
assert bounds(g) == (10, 10, 10, 10)
assert bounds(MockGeoInterface(g)) == (10, 10, 10, 10)
-
-def test_bounds_invalid_obj():
- with pytest.raises(KeyError):
- bounds({'type': 'bogus', 'not_coordinates': []})
+ at pytest.mark.parametrize('geometry', [
+ {'type': 'Polygon'},
+ {'type': 'Polygon', 'not_coordinates': []},
+ {'type': 'bogus', 'not_coordinates': []},
+ {
+ 'type': 'GeometryCollection',
+ 'geometries': [
+ {'type': 'Point', 'coordinates': [1, 1]},
+ {'type': 'LineString', 'not_coordinates': [[-10, -20], [10, 20]]},
+ ]
+ }
+])
+def test_bounds_invalid_obj(geometry):
+ with pytest.raises(ValueError, match="geometry must be a GeoJSON-like geometry, GeometryCollection, or FeatureCollection"):
+ bounds(geometry)
-def test_feature_collection(basic_featurecollection):
+def test_bounds_feature_collection(basic_featurecollection):
fc = basic_featurecollection
assert bounds(fc) == bounds(fc['features'][0]) == (2, 2, 4.25, 4.25)
+def test_bounds_geometry_collection():
+ gc = {
+ 'type': 'GeometryCollection',
+ 'geometries': [
+ {'type': 'Point', 'coordinates': [1, 1]},
+ {'type': 'LineString', 'coordinates': [[-10, -20], [10, 20]]},
+ {'type': 'Polygon', 'coordinates': [[[5, 5], [25, 50], [25, 5]]]}
+ ]
+ }
+
+ assert bounds(gc) == (-10, -20, 25, 50)
+ assert bounds(MockGeoInterface(gc)) == (-10, -20, 25, 50)
+
+
def test_bounds_existing_bbox(basic_featurecollection):
"""Test with existing bbox in geojson.
=====================================
tests/test_path.py
=====================================
@@ -5,6 +5,7 @@ import sys
import pytest
import rasterio
+import rasterio.compat
from rasterio.errors import PathError
from rasterio.path import parse_path, vsi_path, ParsedPath, UnparsedPath
@@ -92,6 +93,11 @@ def test_vsi_path_scheme():
assert vsi_path(ParsedPath('/foo.tif', 'tests/data/files.zip', 'zip')) == '/vsizip/tests/data/files.zip/foo.tif'
+def test_path_as_vsi_scheme():
+ """Correctly make a vsi path"""
+ assert ParsedPath('/foo.tif', 'tests/data/files.zip', 'zip').as_vsi() == '/vsizip/tests/data/files.zip/foo.tif'
+
+
def test_vsi_path_file():
"""Correctly make and ordinary file path from a file path"""
assert vsi_path(ParsedPath('foo.tif', None, 'file')) == 'foo.tif'
@@ -107,6 +113,11 @@ def test_vsi_path_unparsed():
assert vsi_path(UnparsedPath("foo")) == "foo"
+def test_path_as_vsi_unparsed():
+ """Correctly make GDAL filename from unparsed path"""
+ assert UnparsedPath("foo").as_vsi() == "foo"
+
+
def test_vsi_path_error():
"""Raise ValuError if argument is not a path"""
with pytest.raises(ValueError):
@@ -182,3 +193,14 @@ def test_path_error(path):
def test_parse_path():
pathlib = pytest.importorskip("pathlib")
assert isinstance(parse_path(pathlib.Path("/foo/bar.tif")), ParsedPath)
+
+
+def test_parse_path_win():
+ pathlib = pytest.importorskip("pathlib")
+ assert isinstance(parse_path(pathlib.PureWindowsPath(r"C:\foo\bar.tif")), ParsedPath)
+
+
+def test_parse_path_win_no_pathlib(monkeypatch):
+ monkeypatch.setattr(rasterio.path.sys, "platform", "win32")
+ monkeypatch.setattr(rasterio.path, "pathlib", None)
+ assert isinstance(parse_path(r"C:\foo\bar.tif"), UnparsedPath)
=====================================
tests/test_rio_convert.py
=====================================
@@ -1,15 +1,13 @@
-import sys
import os
-import logging
-import numpy as np
+
from click.testing import CliRunner
+import numpy as np
+import pytest
import rasterio
from rasterio.rio.main import main_group
-logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)
-
TEST_BBOX = [-11850000, 4804000, -11840000, 4808000]
@@ -17,11 +15,12 @@ def bbox(*args):
return ' '.join([str(x) for x in args])
-def test_clip_bounds(runner, tmpdir):
+ at pytest.mark.parametrize("bounds", [bbox(*TEST_BBOX)])
+def test_clip_bounds(runner, tmpdir, bounds):
output = str(tmpdir.join('test.tif'))
result = runner.invoke(
- main_group,
- ['clip', 'tests/data/shade.tif', output, '--bounds', bbox(*TEST_BBOX)])
+ main_group, ["clip", "tests/data/shade.tif", output, "--bounds", bounds]
+ )
assert result.exit_code == 0
assert os.path.exists(output)
@@ -29,6 +28,29 @@ def test_clip_bounds(runner, tmpdir):
assert out.shape == (419, 173)
+ at pytest.mark.parametrize("bounds", [bbox(*TEST_BBOX)])
+def test_clip_bounds_with_complement(runner, tmpdir, bounds):
+ output = str(tmpdir.join("test.tif"))
+ result = runner.invoke(
+ main_group,
+ [
+ "clip",
+ "tests/data/shade.tif",
+ output,
+ "--bounds",
+ bounds,
+ "--with-complement",
+ ],
+ )
+ assert result.exit_code == 0
+ assert os.path.exists(output)
+
+ with rasterio.open(output) as out:
+ assert out.shape == (419, 1047)
+ data = out.read()
+ assert (data[420:, :] == 255).all()
+
+
def test_clip_bounds_geographic(runner, tmpdir):
output = str(tmpdir.join('test.tif'))
result = runner.invoke(
@@ -106,9 +128,16 @@ def test_clip_overwrite_with_option(runner, tmpdir):
assert result.exit_code == 0
result = runner.invoke(
- main_group, [
- 'clip', 'tests/data/shade.tif', output, '--bounds', bbox(*TEST_BBOX),
- '--overwrite'])
+ main_group,
+ [
+ "clip",
+ "tests/data/shade.tif",
+ output,
+ "--bounds",
+ bbox(*TEST_BBOX),
+ "--overwrite",
+ ],
+ )
assert result.exit_code == 0
=====================================
tests/test_rio_rasterize.py
=====================================
@@ -215,7 +215,8 @@ def test_rasterize_invalid_like_raster(tmpdir, runner, basic_feature):
input=json.dumps(basic_feature))
assert result.exit_code == 2
- assert 'Invalid value for "--like":' in result.output
+ assert "Invalid value for" in result.output
+ assert "--like" in result.output
def test_rasterize_like_raster_src_crs_mismatch(tmpdir, runner, basic_feature,
=====================================
tests/test_rio_warp.py
=====================================
@@ -607,4 +607,5 @@ def test_warp_resampling_not_yet_supported(
'--resampling', method.name])
assert result.exit_code == 2
- assert 'Invalid value for "--resampling"' in result.output
+ assert "Invalid value for" in result.output
+ assert "--resampling" in result.output
View it on GitLab: https://salsa.debian.org/debian-gis-team/rasterio/-/commit/aee4b2f34cdf69629d0e900834ddd2704ac4377d
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/rasterio/-/commit/aee4b2f34cdf69629d0e900834ddd2704ac4377d
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20200507/3c835687/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list