[Git][debian-gis-team/trollimage][upstream] New upstream version 1.19.0
Antonio Valentino (@antonio.valentino)
gitlab at salsa.debian.org
Sun Oct 23 15:16:23 BST 2022
Antonio Valentino pushed to branch upstream at Debian GIS Project / trollimage
Commits:
2f35a3f3 by Antonio Valentino at 2022-10-23T13:57:04+00:00
New upstream version 1.19.0
- - - - -
9 changed files:
- CHANGELOG.md
- continuous_integration/environment.yaml
- doc/index.rst
- + doc/installation.rst
- setup.py
- + trollimage/_xrimage_rasterio.py
- trollimage/tests/test_image.py
- trollimage/version.py
- trollimage/xrimage.py
Changes:
=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,29 @@
+## Version 1.19.0 (2022/10/21)
+
+### Issues Closed
+
+* [Issue 72](https://github.com/pytroll/trollimage/issues/72) - Add install instructions ([PR 105](https://github.com/pytroll/trollimage/pull/105) by [@djhoese](https://github.com/djhoese))
+
+In this release 1 issue was closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 109](https://github.com/pytroll/trollimage/pull/109) - Fix XRImage.colorize to mask integer data with _FillValue ([545](https://github.com/ssec/polar2grid/issues/545))
+* [PR 108](https://github.com/pytroll/trollimage/pull/108) - Fix typo in rasterio AreaDefinition handling
+
+#### Features added
+
+* [PR 107](https://github.com/pytroll/trollimage/pull/107) - Refactor rasterio usage to improve import time
+
+#### Documentation changes
+
+* [PR 105](https://github.com/pytroll/trollimage/pull/105) - Add installation instructions ([72](https://github.com/pytroll/trollimage/issues/72))
+
+In this release 4 pull requests were closed.
+
+
## Version 1.18.3 (2022/03/07)
### Pull Requests Merged
=====================================
continuous_integration/environment.yaml
=====================================
@@ -14,6 +14,8 @@ dependencies:
- codecov
- rasterio
- libtiff
+ - pyproj
+ - pyresample
- pytest
- pytest-cov
- fsspec
=====================================
doc/index.rst
=====================================
@@ -1,22 +1,18 @@
-.. TrollImage documentation master file, created by
- sphinx-quickstart on Mon Dec 2 09:40:29 2013.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
-Welcome to TrollImage's documentation!
-======================================
+TrollImage
+==========
.. image:: _static/hayan.png
Get the source code here_ !
-.. _here: http://github.com/mraspaud/trollimage
+.. _here: http://github.com/pytroll/trollimage
Contents:
.. toctree::
:maxdepth: 2
+ installation
image
xrimage
colorspaces
=====================================
doc/installation.rst
=====================================
@@ -0,0 +1,95 @@
+============
+Installation
+============
+
+Trollimage is available from conda-forge (via conda), PyPI (via pip), or from
+source (via pip+git). The below instructions show how to install stable
+versions of trollimage or from the source code.
+
+Conda-based Installation
+========================
+
+Trollimage can be installed into a conda environment by installing the package
+from the conda-forge channel. If you do not already have access to a conda
+installation, we recommend installing
+`miniconda <https://docs.conda.io/en/latest/miniconda.html>`_ for the smallest
+and easiest installation.
+
+The commands below will use ``-c conda-forge`` to make sure packages are
+downloaded from the conda-forge channel. Alternatively, you can tell conda
+to always use conda-forge by running:
+
+.. code-block:: bash
+
+ $ conda config --add channels conda-forge
+
+In a new conda environment
+--------------------------
+
+We recommend creating a separate environment for your work with trollimage. To
+create a new environment and install trollimage all in one command you can
+run:
+
+.. code-block:: bash
+
+ $ conda create -c conda-forge -n my_trollimage_env python trollimage
+
+You must then activate the environment so any future python or
+conda commands will use this environment.
+
+.. code-block::
+
+ $ conda activate my_trollimage_env
+
+This method of creating an environment with trollimage (and optionally other
+packages) installed can generally be created faster than creating an
+environment and then later installing trollimage and other packages (see the
+section below).
+
+In an existing environment
+--------------------------
+
+.. note::
+
+ It is recommended that when first exploring trollimage, you create a new
+ environment specifically for this rather than modifying one used for
+ other work.
+
+If you already have a conda environment, it is activated, and would like to
+install trollimage into it, run the following:
+
+.. code-block:: bash
+
+ $ conda install -c conda-forge trollimage
+
+Pip-based Installation
+======================
+
+Trollimage is available from the Python Packaging Index (PyPI). A sandbox
+environment for ``trollimage`` can be created using
+`Virtualenv <http://pypi.python.org/pypi/virtualenv>`_.
+
+To install the `trollimage` package and the minimum amount of python dependencies:
+
+.. code-block:: bash
+
+ $ pip install trollimage
+
+Install from source
+===================
+
+To install directly from github into an existing environment (pip or conda-based):
+
+.. code-block:: bash
+
+ $ pip install git+https://github.com/pytroll/trollimage.git
+
+If you have the ``git`` command installed this will automatically download the
+source code and install it into the current environment. If you would like to
+modify a local copy of trollimage and see the effects immediately in your
+environment use the below command instead. This command should be run from the
+root your cloned version of the git repository (where the ``setup.py`` is located):
+
+.. code-block::
+
+ $ pip install -e .
=====================================
setup.py
=====================================
@@ -46,11 +46,11 @@ setup(name="trollimage",
url="https://github.com/pytroll/trollimage",
packages=['trollimage'],
zip_safe=False,
- install_requires=['numpy >=1.13', 'pillow'],
+ install_requires=['numpy>=1.13', 'pillow'],
python_requires='>=3.6',
extras_require={
- 'geotiff': ['rasterio'],
+ 'geotiff': ['rasterio>=1.0'],
'xarray': ['xarray', 'dask[array]'],
},
- tests_require=['xarray', 'dask[array]'],
+ tests_require=['xarray', 'dask[array]', 'pyproj', 'pyresample'],
)
=====================================
trollimage/_xrimage_rasterio.py
=====================================
@@ -0,0 +1,272 @@
+# Copyright (c) 2022 trollimage developers
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+"""RasterIO-specific utilities needed by the XRImage class."""
+
+import logging
+import threading
+from contextlib import suppress
+
+import dask.array as da
+
+import rasterio
+from rasterio.enums import Resampling
+from rasterio.windows import Window
+
+logger = logging.getLogger(__name__)
+
+
+def get_data_arr_crs_transform_gcps(data_arr):
+ """Convert DataArray's AreaDefinition or SwathDefinition to rasterio geolocation information.
+
+ If possible, a rasterio geotransform object will be created. If it can't be made
+ then it is assumed the provided geometry object is a SwathDefinition and will be
+ checked for GCP coordinates (``swath_def.lons.attrs['gcps']``).
+
+ Args:
+ data_arr: Xarray DataArray.
+
+ Returns:
+ Tuple of (crs, transform, gcps). Each element defaults to ``None`` if
+ it couldn't be calculated.
+
+ """
+ crs = None
+ transform = None
+ gcps = None
+
+ try:
+ area = data_arr.attrs["area"]
+ if rasterio.__gdal_version__ >= '3':
+ wkt_version = 'WKT2_2018'
+ else:
+ wkt_version = 'WKT1_GDAL'
+ if hasattr(area, 'crs'):
+ crs = rasterio.crs.CRS.from_wkt(area.crs.to_wkt(version=wkt_version))
+ else:
+ crs = rasterio.crs.CRS(area.proj_dict)
+ west, south, east, north = area.area_extent
+ height, width = area.shape
+ transform = rasterio.transform.from_bounds(west, south,
+ east, north,
+ width, height)
+
+ except KeyError: # No area
+ logger.info("Couldn't create geotransform")
+ except AttributeError:
+ try:
+ gcps = data_arr.attrs["area"].lons.attrs['gcps']
+ crs = data_arr.attrs["area"].lons.attrs['crs']
+ except KeyError:
+ logger.info("Couldn't create geotransform")
+ return crs, transform, gcps
+
+
+def split_regular_vs_lazy_tags(tags, r_file):
+ """Split tags into regular vs lazy (dask) tags."""
+ da_tags = []
+ for key, val in list(tags.items()):
+ try:
+ if isinstance(val.data, da.Array):
+ da_tags.append((val.data, RIOTag(r_file, key)))
+ tags.pop(key)
+ else:
+ tags[key] = val.item()
+ except AttributeError:
+ continue
+ return tags, da_tags
+
+
+class RIOFile(object):
+ """Rasterio wrapper to allow da.store to do window saving."""
+
+ def __init__(self, path, mode='w', **kwargs):
+ """Initialize the object."""
+ self.path = path
+ self.mode = mode
+ self.kwargs = kwargs
+ self.rfile = None
+ self.lock = threading.Lock()
+
+ @property
+ def width(self):
+ """Width of the band images."""
+ return self.kwargs['width']
+
+ @property
+ def height(self):
+ """Height of the band images."""
+ return self.kwargs['height']
+
+ @property
+ def closed(self):
+ """Check if the file is closed."""
+ return self.rfile is None or self.rfile.closed
+
+ def open(self, mode=None):
+ """Open the file."""
+ mode = mode or self.mode
+ if self.closed:
+ self.rfile = rasterio.open(self.path, mode, **self.kwargs)
+
+ def close(self):
+ """Close the file."""
+ with self.lock:
+ if not self.closed:
+ self.rfile.close()
+
+ def __enter__(self):
+ """Enter method."""
+ self.open()
+ return self
+
+ def __exit__(self, exc_type, exc_value, traceback):
+ """Exit method."""
+ self.close()
+
+ def __del__(self):
+ """Delete the instance."""
+ with suppress(IOError, OSError):
+ self.close()
+
+ @property
+ def colorinterp(self):
+ """Return the color interpretation of the image."""
+ return self.rfile.colorinterp
+
+ @colorinterp.setter
+ def colorinterp(self, val):
+ if rasterio.__version__.startswith("0."):
+ # not supported in older versions, set by PHOTOMETRIC tag
+ logger.warning("Rasterio 1.0+ required for setting colorinterp")
+ else:
+ self.rfile.colorinterp = val
+
+ def write(self, *args, **kwargs):
+ """Write to the file."""
+ with self.lock:
+ self.open('r+')
+ return self.rfile.write(*args, **kwargs)
+
+ def build_overviews(self, *args, **kwargs):
+ """Write overviews."""
+ with self.lock:
+ self.open('r+')
+ return self.rfile.build_overviews(*args, **kwargs)
+
+ def update_tags(self, *args, **kwargs):
+ """Update tags."""
+ with self.lock:
+ self.open('a')
+ return self.rfile.update_tags(*args, **kwargs)
+
+
+class RIOTag:
+ """Rasterio wrapper to allow da.store on tag."""
+
+ def __init__(self, rfile, name):
+ """Init the rasterio tag."""
+ self.rfile = rfile
+ self.name = name
+
+ def __setitem__(self, key, item):
+ """Put the data in the tag."""
+ kwargs = {self.name: item.item()}
+ self.rfile.update_tags(**kwargs)
+
+ def close(self):
+ """Close the file."""
+ return self.rfile.close()
+
+
+class RIODataset:
+ """A wrapper for a rasterio dataset."""
+
+ def __init__(self, rfile, overviews=None, overviews_resampling=None,
+ overviews_minsize=256):
+ """Init the rasterio dataset."""
+ self.rfile = rfile
+ self.overviews = overviews
+ if overviews_resampling is None:
+ overviews_resampling = 'nearest'
+ self.overviews_resampling = Resampling[overviews_resampling]
+ self.overviews_minsize = overviews_minsize
+
+ def __setitem__(self, key, item):
+ """Put the data chunk in the image."""
+ if len(key) == 3:
+ indexes = list(range(
+ key[0].start + 1,
+ key[0].stop + 1,
+ key[0].step or 1
+ ))
+ y = key[1]
+ x = key[2]
+ else:
+ indexes = 1
+ y = key[0]
+ x = key[1]
+ chy_off = y.start
+ chy = y.stop - y.start
+ chx_off = x.start
+ chx = x.stop - x.start
+
+ # band indexes
+ self.rfile.write(item, window=Window(chx_off, chy_off, chx, chy),
+ indexes=indexes)
+
+ def close(self):
+ """Close the file."""
+ if self.overviews is not None:
+ overviews = self.overviews
+ # it's an empty list
+ if len(overviews) == 0:
+ from rasterio.rio.overview import get_maximum_overview_level
+ width = self.rfile.width
+ height = self.rfile.height
+ max_level = get_maximum_overview_level(
+ width, height, self.overviews_minsize)
+ overviews = [2 ** j for j in range(1, max_level + 1)]
+ logger.debug('Building overviews %s with %s resampling',
+ str(overviews), self.overviews_resampling.name)
+ self.rfile.build_overviews(overviews, resampling=self.overviews_resampling)
+
+ return self.rfile.close()
+
+
+def color_interp(data):
+ """Get the color interpretation for this image."""
+ from rasterio.enums import ColorInterp as ci
+ modes = {'L': [ci.gray],
+ 'LA': [ci.gray, ci.alpha],
+ 'YCbCr': [ci.Y, ci.Cb, ci.Cr],
+ 'YCbCrA': [ci.Y, ci.Cb, ci.Cr, ci.alpha]}
+
+ try:
+ mode = ''.join(data['bands'].values)
+ return modes[mode]
+ except KeyError:
+ colors = {'R': ci.red,
+ 'G': ci.green,
+ 'B': ci.blue,
+ 'A': ci.alpha,
+ 'C': ci.cyan,
+ 'M': ci.magenta,
+ 'Y': ci.yellow,
+ 'H': ci.hue,
+ 'S': ci.saturation,
+ 'L': ci.lightness,
+ 'K': ci.black,
+ }
+ return [colors[band] for band in data['bands'].values]
=====================================
trollimage/tests/test_image.py
=====================================
@@ -34,6 +34,7 @@ import pytest
from trollimage import image, xrimage
from trollimage.colormap import Colormap, brbg
+from trollimage._xrimage_rasterio import RIODataset
from .utils import assert_maximum_dask_computes
@@ -1001,7 +1002,7 @@ class TestXRImage:
delay = img.save(tmp.name, compute=False)
assert isinstance(delay, tuple)
assert isinstance(delay[0], da.Array)
- assert isinstance(delay[1], xrimage.RIODataset)
+ assert isinstance(delay[1], RIODataset)
da.store(*delay)
delay[1].close()
@@ -1046,46 +1047,22 @@ class TestXRImage:
tags = _get_tags_after_writing_to_geotiff(data)
assert "TIFFTAG_DATETIME" in tags
+ @pytest.mark.parametrize("output_ext", [".tif", ".tiff"])
+ @pytest.mark.parametrize("use_dask", [False, True])
@pytest.mark.skipif(sys.platform.startswith('win'),
reason="'NamedTemporaryFile' not supported on Windows")
- def test_save_geotiff_int(self):
+ def test_save_geotiff_int(self, output_ext, use_dask):
"""Test saving geotiffs when input data is int."""
- from rasterio.control import GroundControlPoint
-
- # numpy array image
- data = xr.DataArray(np.arange(75).reshape(5, 5, 3), dims=[
- 'y', 'x', 'bands'], coords={'bands': ['R', 'G', 'B']})
- img = xrimage.XRImage(data)
- assert np.issubdtype(img.data.dtype, np.integer)
- with NamedTemporaryFile(suffix='.tif') as tmp:
- img.save(tmp.name)
- with rio.open(tmp.name) as f:
- file_data = f.read()
- assert file_data.shape == (4, 5, 5) # alpha band added
- exp = np.arange(75).reshape(5, 5, 3)
- np.testing.assert_allclose(file_data[0], exp[:, :, 0])
- np.testing.assert_allclose(file_data[1], exp[:, :, 1])
- np.testing.assert_allclose(file_data[2], exp[:, :, 2])
- np.testing.assert_allclose(file_data[3], 255)
- # test .tiff too
- with NamedTemporaryFile(suffix='.tiff') as tmp:
- img.save(tmp.name)
- with rio.open(tmp.name) as f:
- file_data = f.read()
- assert file_data.shape == (4, 5, 5) # alpha band added
- exp = np.arange(75).reshape(5, 5, 3)
- np.testing.assert_allclose(file_data[0], exp[:, :, 0])
- np.testing.assert_allclose(file_data[1], exp[:, :, 1])
- np.testing.assert_allclose(file_data[2], exp[:, :, 2])
- np.testing.assert_allclose(file_data[3], 255)
+ arr = np.arange(75).reshape(5, 5, 3)
+ if use_dask:
+ arr = da.from_array(arr, chunks=5)
- data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
+ data = xr.DataArray(arr,
dims=['y', 'x', 'bands'],
coords={'bands': ['R', 'G', 'B']})
img = xrimage.XRImage(data)
assert np.issubdtype(img.data.dtype, np.integer)
- # Regular default save
- with NamedTemporaryFile(suffix='.tif') as tmp:
+ with NamedTemporaryFile(suffix=output_ext) as tmp:
img.save(tmp.name)
with rio.open(tmp.name) as f:
file_data = f.read()
@@ -1096,20 +1073,28 @@ class TestXRImage:
np.testing.assert_allclose(file_data[2], exp[:, :, 2])
np.testing.assert_allclose(file_data[3], 255)
- # dask delayed save
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_delayed(self):
+ """Test saving a geotiff but not computing the result immediately."""
+ data = xr.DataArray(np.arange(75).reshape(5, 5, 3), dims=[
+ 'y', 'x', 'bands'], coords={'bands': ['R', 'G', 'B']})
+ img = xrimage.XRImage(data)
+ assert np.issubdtype(img.data.dtype, np.integer)
with NamedTemporaryFile(suffix='.tif') as tmp:
delay = img.save(tmp.name, compute=False)
assert isinstance(delay, tuple)
assert isinstance(delay[0], da.Array)
- assert isinstance(delay[1], xrimage.RIODataset)
+ assert isinstance(delay[1], RIODataset)
da.store(*delay)
delay[1].close()
- # GCPs
- class FakeArea():
- def __init__(self, lons, lats):
- self.lons = lons
- self.lats = lats
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_gcps(self):
+ """Test saving geotiffs when input data is int and has GCPs."""
+ from rasterio.control import GroundControlPoint
+ from pyresample import SwathDefinition
gcps = [GroundControlPoint(1, 1, 100.0, 1000.0, z=0.0),
GroundControlPoint(2, 3, 400.0, 2000.0, z=0.0)]
@@ -1124,11 +1109,12 @@ class TestXRImage:
dims=['y', 'x'],
attrs={'gcps': gcps,
'crs': crs})
+ swath_def = SwathDefinition(lons, lats)
data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
dims=['y', 'x', 'bands'],
coords={'bands': ['R', 'G', 'B']},
- attrs={'area': FakeArea(lons, lats)})
+ attrs={'area': swath_def})
img = xrimage.XRImage(data)
with NamedTemporaryFile(suffix='.tif') as tmp:
img.save(tmp.name)
@@ -1142,7 +1128,41 @@ class TestXRImage:
assert ref.z == val.z
assert crs == fcrs
- # with rasterio colormap provided
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_no_gcp_swath(self):
+ """Test saving geotiffs when input data whose SwathDefinition has no GCPs.
+
+ If shouldn't fail, but it also shouldn't have a non-default transform.
+
+ """
+ from pyresample import SwathDefinition
+
+ lons = xr.DataArray(da.from_array(np.arange(25).reshape(5, 5), chunks=5),
+ dims=['y', 'x'],
+ attrs={})
+
+ lats = xr.DataArray(da.from_array(np.arange(25).reshape(5, 5), chunks=5),
+ dims=['y', 'x'],
+ attrs={})
+ swath_def = SwathDefinition(lons, lats)
+
+ data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
+ dims=['y', 'x', 'bands'],
+ coords={'bands': ['R', 'G', 'B']},
+ attrs={'area': swath_def})
+ img = xrimage.XRImage(data)
+ with NamedTemporaryFile(suffix='.tif') as tmp:
+ img.save(tmp.name)
+ with rio.open(tmp.name) as f:
+ assert f.transform.a == 1.0
+ assert f.transform.b == 0.0
+ assert f.transform.c == 0.0
+
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_rio_colormap(self):
+ """Test saving geotiffs when input data is int and a rasterio colormap is provided."""
exp_cmap = {i: (i, 255 - i, i, 255) for i in range(256)}
data = xr.DataArray(da.from_array(np.arange(81).reshape(9, 9, 1), chunks=9),
dims=['y', 'x', 'bands'],
@@ -1158,7 +1178,10 @@ class TestXRImage:
np.testing.assert_allclose(file_data[0], exp[:, :, 0])
assert cmap == exp_cmap
- # with input fill value
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_with_fill(self):
+ """Test saving geotiffs when input data is int and a fill value is specified."""
data = np.arange(75).reshape(5, 5, 3)
# second pixel is all bad
# pixel [0, 1, 1] is also naturally 5 by arange above
@@ -1181,7 +1204,20 @@ class TestXRImage:
np.testing.assert_allclose(file_data[1], exp[:, :, 1])
np.testing.assert_allclose(file_data[2], exp[:, :, 2])
- # input fill value but alpha on output
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_with_fill_and_alpha(self):
+ """Test saving int geotiffs with a fill value and input alpha band."""
+ data = np.arange(75).reshape(5, 5, 3)
+ # second pixel is all bad
+ # pixel [0, 1, 1] is also naturally 5 by arange above
+ data[0, 1, :] = 5
+ data = xr.DataArray(da.from_array(data, chunks=5),
+ dims=['y', 'x', 'bands'],
+ attrs={'_FillValue': 5},
+ coords={'bands': ['R', 'G', 'B']})
+ img = xrimage.XRImage(data)
+ assert np.issubdtype(img.data.dtype, np.integer)
with NamedTemporaryFile(suffix='.tif') as tmp:
img.save(tmp.name)
with rio.open(tmp.name) as f:
@@ -1197,6 +1233,41 @@ class TestXRImage:
np.testing.assert_allclose(file_data[2], exp[:, :, 2])
np.testing.assert_allclose(file_data[3], exp_alpha)
+ @pytest.mark.skipif(sys.platform.startswith('win'),
+ reason="'NamedTemporaryFile' not supported on Windows")
+ def test_save_geotiff_int_with_area_def(self):
+ """Test saving a integer image with an AreaDefinition."""
+ from pyproj import CRS
+ from pyresample import AreaDefinition
+ crs = CRS.from_user_input(4326)
+ area_def = AreaDefinition(
+ "test", "test", "",
+ crs, 5, 5, [-300, -250, 200, 250],
+ )
+
+ data = xr.DataArray(np.arange(75).reshape(5, 5, 3),
+ dims=['y', 'x', 'bands'],
+ coords={'bands': ['R', 'G', 'B']},
+ attrs={"area": area_def})
+ img = xrimage.XRImage(data)
+ assert np.issubdtype(img.data.dtype, np.integer)
+ with NamedTemporaryFile(suffix='.tif') as tmp:
+ img.save(tmp.name)
+ with rio.open(tmp.name) as f:
+ file_data = f.read()
+ assert f.crs.to_epsg() == 4326
+ geotransform = f.transform
+ assert geotransform.a == 100
+ assert geotransform.c == -300
+ assert geotransform.e == -100
+ assert geotransform.f == 250
+ assert file_data.shape == (4, 5, 5) # alpha band added
+ exp = np.arange(75).reshape(5, 5, 3)
+ np.testing.assert_allclose(file_data[0], exp[:, :, 0])
+ np.testing.assert_allclose(file_data[1], exp[:, :, 1])
+ np.testing.assert_allclose(file_data[2], exp[:, :, 2])
+ np.testing.assert_allclose(file_data[3], 255)
+
@pytest.mark.skipif(sys.platform.startswith('win'),
reason="'NamedTemporaryFile' not supported on Windows")
@pytest.mark.parametrize(
@@ -2068,6 +2139,25 @@ class TestXRImageColorize:
assert img.data.attrs["enhancement_history"][-1]["offset"] == expected_offset
assert isinstance(img.data.attrs["enhancement_history"][-1]["colormap"], Colormap)
+ def test_colorize_int_l_rgb_with_fills(self):
+ """Test integer data with _FillValue is masked (NaN) when colorized."""
+ arr = np.arange(75, dtype=np.uint8).reshape(5, 15)
+ arr[1, :] = 255
+ data = xr.DataArray(arr.copy(), dims=['y', 'x'],
+ attrs={"_FillValue": 255})
+ new_brbg = brbg.set_range(5, 20, inplace=False)
+ img = xrimage.XRImage(data)
+ img.colorize(new_brbg)
+ values = img.data.compute()
+ assert values.shape == (3,) + arr.shape # RGB
+ np.testing.assert_allclose(values[:, 1, :], np.nan)
+ assert np.count_nonzero(np.isnan(values)) == arr.shape[1] * 3
+
+ assert "enhancement_history" in img.data.attrs
+ assert img.data.attrs["enhancement_history"][-1]["scale"] == 1 / (20 - 5)
+ assert img.data.attrs["enhancement_history"][-1]["offset"] == -5 / (20 - 5)
+ assert isinstance(img.data.attrs["enhancement_history"][-1]["colormap"], Colormap)
+
def test_colorize_la_rgb(self):
"""Test colorizing an LA image with an RGB colormap."""
arr = np.arange(75).reshape(5, 15) / 74.
=====================================
trollimage/version.py
=====================================
@@ -23,9 +23,9 @@ def get_keywords():
# setup.py/versioneer.py will grep for the variable names, so they must
# each be defined on a line of their own. _version.py will just call
# get_keywords().
- git_refnames = " (HEAD -> main, tag: v1.18.3)"
- git_full = "baae46cbe74beffb6cd669978ec92a44396bf5f7"
- git_date = "2022-03-07 12:29:07 -0600"
+ git_refnames = " (HEAD -> main, tag: v1.19.0)"
+ git_full = "93e547a45a1c3a13768da4a5169a3f5bf5ebc75a"
+ git_date = "2022-10-21 09:50:00 -0500"
keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
return keywords
=====================================
trollimage/xrimage.py
=====================================
@@ -35,9 +35,7 @@ chunks can be saved in parallel.
import logging
import numbers
import os
-import threading
import warnings
-from contextlib import suppress
import dask
import dask.array as da
@@ -47,209 +45,9 @@ from PIL import Image as PILImage
from dask.delayed import delayed
from trollimage.image import check_image_format
-try:
- import rasterio
- from rasterio.enums import Resampling
-except ImportError:
- rasterio = None
-
-try:
- # rasterio 1.0+
- from rasterio.windows import Window
-except ImportError:
- # raster 0.36.0
- # remove this once rasterio 1.0+ is officially available
- def Window(x_off, y_off, x_size, y_size):
- """Replace the missing Window object in rasterio < 1.0."""
- return (y_off, y_off + y_size), (x_off, x_off + x_size)
-
logger = logging.getLogger(__name__)
-class RIOFile(object):
- """Rasterio wrapper to allow da.store to do window saving."""
-
- def __init__(self, path, mode='w', **kwargs):
- """Initialize the object."""
- self.path = path
- self.mode = mode
- self.kwargs = kwargs
- self.rfile = None
- self.lock = threading.Lock()
-
- @property
- def width(self):
- """Width of the band images."""
- return self.kwargs['width']
-
- @property
- def height(self):
- """Height of the band images."""
- return self.kwargs['height']
-
- @property
- def closed(self):
- """Check if the file is closed."""
- return self.rfile is None or self.rfile.closed
-
- def open(self, mode=None):
- """Open the file."""
- mode = mode or self.mode
- if self.closed:
- self.rfile = rasterio.open(self.path, mode, **self.kwargs)
-
- def close(self):
- """Close the file."""
- with self.lock:
- if not self.closed:
- self.rfile.close()
-
- def __enter__(self):
- """Enter method."""
- self.open()
- return self
-
- def __exit__(self, exc_type, exc_value, traceback):
- """Exit method."""
- self.close()
-
- def __del__(self):
- """Delete the instance."""
- with suppress(IOError, OSError):
- self.close()
-
- @property
- def colorinterp(self):
- """Return the color interpretation of the image."""
- return self.rfile.colorinterp
-
- @colorinterp.setter
- def colorinterp(self, val):
- if rasterio.__version__.startswith("0."):
- # not supported in older versions, set by PHOTOMETRIC tag
- logger.warning("Rasterio 1.0+ required for setting colorinterp")
- else:
- self.rfile.colorinterp = val
-
- def write(self, *args, **kwargs):
- """Write to the file."""
- with self.lock:
- self.open('r+')
- return self.rfile.write(*args, **kwargs)
-
- def build_overviews(self, *args, **kwargs):
- """Write overviews."""
- with self.lock:
- self.open('r+')
- return self.rfile.build_overviews(*args, **kwargs)
-
- def update_tags(self, *args, **kwargs):
- """Update tags."""
- with self.lock:
- self.open('a')
- return self.rfile.update_tags(*args, **kwargs)
-
-
-class RIOTag:
- """Rasterio wrapper to allow da.store on tag."""
-
- def __init__(self, rfile, name):
- """Init the rasterio tag."""
- self.rfile = rfile
- self.name = name
-
- def __setitem__(self, key, item):
- """Put the data in the tag."""
- kwargs = {self.name: item.item()}
- self.rfile.update_tags(**kwargs)
-
- def close(self):
- """Close the file."""
- return self.rfile.close()
-
-
-class RIODataset:
- """A wrapper for a rasterio dataset."""
-
- def __init__(self, rfile, overviews=None, overviews_resampling=None,
- overviews_minsize=256):
- """Init the rasterio dataset."""
- self.rfile = rfile
- self.overviews = overviews
- if overviews_resampling is None:
- overviews_resampling = 'nearest'
- self.overviews_resampling = Resampling[overviews_resampling]
- self.overviews_minsize = overviews_minsize
-
- def __setitem__(self, key, item):
- """Put the data chunk in the image."""
- if len(key) == 3:
- indexes = list(range(
- key[0].start + 1,
- key[0].stop + 1,
- key[0].step or 1
- ))
- y = key[1]
- x = key[2]
- else:
- indexes = 1
- y = key[0]
- x = key[1]
- chy_off = y.start
- chy = y.stop - y.start
- chx_off = x.start
- chx = x.stop - x.start
-
- # band indexes
- self.rfile.write(item, window=Window(chx_off, chy_off, chx, chy),
- indexes=indexes)
-
- def close(self):
- """Close the file."""
- if self.overviews is not None:
- overviews = self.overviews
- # it's an empty list
- if len(overviews) == 0:
- from rasterio.rio.overview import get_maximum_overview_level
- width = self.rfile.width
- height = self.rfile.height
- max_level = get_maximum_overview_level(
- width, height, self.overviews_minsize)
- overviews = [2 ** j for j in range(1, max_level + 1)]
- logger.debug('Building overviews %s with %s resampling',
- str(overviews), self.overviews_resampling.name)
- self.rfile.build_overviews(overviews, resampling=self.overviews_resampling)
-
- return self.rfile.close()
-
-
-def color_interp(data):
- """Get the color interpretation for this image."""
- from rasterio.enums import ColorInterp as ci
- modes = {'L': [ci.gray],
- 'LA': [ci.gray, ci.alpha],
- 'YCbCr': [ci.Y, ci.Cb, ci.Cr],
- 'YCbCrA': [ci.Y, ci.Cb, ci.Cr, ci.alpha]}
-
- try:
- mode = ''.join(data['bands'].values)
- return modes[mode]
- except KeyError:
- colors = {'R': ci.red,
- 'G': ci.green,
- 'B': ci.blue,
- 'A': ci.alpha,
- 'C': ci.cyan,
- 'M': ci.magenta,
- 'Y': ci.yellow,
- 'H': ci.hue,
- 'S': ci.saturation,
- 'L': ci.lightness,
- 'K': ci.black,
- }
- return [colors[band] for band in data['bands'].values]
-
-
def combine_scales_offsets(*args):
"""Combine ``(scale, offset)`` tuples in one, considering they are applied from left to right.
@@ -445,12 +243,15 @@ class XRImage:
"""
kwformat = format_kwargs.pop('format', None)
fformat = fformat or kwformat or os.path.splitext(filename)[1][1:]
- if fformat in ('tif', 'tiff', 'jp2') and rasterio:
-
- return self.rio_save(filename, fformat=fformat, driver=driver,
- fill_value=fill_value, compute=compute,
- keep_palette=keep_palette, cmap=cmap,
- **format_kwargs)
+ if fformat in ('tif', 'tiff', 'jp2'):
+ try:
+ return self.rio_save(filename, fformat=fformat, driver=driver,
+ fill_value=fill_value, compute=compute,
+ keep_palette=keep_palette, cmap=cmap,
+ **format_kwargs)
+ except ImportError:
+ logger.warning("Missing 'rasterio' dependency to save GeoTIFF "
+ "image. Will try using PIL...")
return self.pil_save(filename, fformat, fill_value,
compute=compute, **format_kwargs)
@@ -525,6 +326,7 @@ class XRImage:
The delayed or computed result of the saving.
"""
+ from ._xrimage_rasterio import RIOFile, RIODataset, split_regular_vs_lazy_tags
fformat = fformat or os.path.splitext(filename)[1][1:]
drivers = {'jpg': 'JPEG',
'png': 'PNG',
@@ -570,36 +372,14 @@ class XRImage:
format_kwargs.setdefault('photometric',
photometric_map[mode.upper()])
- try:
- area = data.attrs['area']
- if rasterio.__gdal_version__ >= '3':
- wkt_version = 'WKT2_2018'
- else:
- wkt_version = 'WKT1_GDAL'
- if hasattr(area, 'crs'):
- crs = rasterio.crs.CRS.from_wkt(area.crs.to_wkt(version=wkt_version))
- else:
- crs = rasterio.crs.CRS(data.attrs['area'].proj_dict)
- west, south, east, north = data.attrs['area'].area_extent
- height, width = data.sizes['y'], data.sizes['x']
- transform = rasterio.transform.from_bounds(west, south,
- east, north,
- width, height)
-
- except KeyError: # No area
- logger.info("Couldn't create geotransform")
- except AttributeError:
- try:
- gcps = data.attrs['area'].lons.attrs['gcps']
- crs = data.attrs['area'].lons.attrs['crs']
- except KeyError:
- logger.info("Couldn't create geotransform")
+ from ._xrimage_rasterio import get_data_arr_crs_transform_gcps
+ crs, transform, gcps = get_data_arr_crs_transform_gcps(data)
stime = data.attrs.get("start_time")
if stime:
stime_str = stime.strftime("%Y:%m:%d %H:%M:%S")
tags.setdefault('TIFFTAG_DATETIME', stime_str)
- elif driver == 'JPEG' and 'A' in mode:
+ if driver == 'JPEG' and 'A' in mode:
raise ValueError('JPEG does not support alpha')
enhancement_colormap = self._get_colormap_from_enhancement_history(data)
@@ -624,6 +404,7 @@ class XRImage:
**format_kwargs)
r_file.open()
if not keep_palette:
+ from ._xrimage_rasterio import color_interp
r_file.colorinterp = color_interp(data)
if keep_palette and cmap is not None:
@@ -636,8 +417,7 @@ class XRImage:
except AttributeError:
raise ValueError("Colormap is not formatted correctly")
- tags, da_tags = self._split_regular_vs_lazy_tags(tags, r_file)
-
+ tags, da_tags = split_regular_vs_lazy_tags(tags, r_file)
r_file.rfile.update_tags(**tags)
r_dataset = RIODataset(r_file, overviews,
overviews_resampling=overviews_resampling,
@@ -668,21 +448,6 @@ class XRImage:
return enhance_dict["colormap"]
return None
- @staticmethod
- def _split_regular_vs_lazy_tags(tags, r_file):
- """Split tags into regular vs lazy (dask) tags."""
- da_tags = []
- for key, val in list(tags.items()):
- try:
- if isinstance(val.data, da.Array):
- da_tags.append((val.data, RIOTag(r_file, key)))
- tags.pop(key)
- else:
- tags[key] = val.item()
- except AttributeError:
- continue
- return tags, da_tags
-
def pil_save(self, filename, fformat=None, fill_value=None,
compute=True, **format_kwargs):
"""Save the image to the given *filename* using PIL.
@@ -1497,7 +1262,12 @@ class XRImage:
"""Colorize the current image using ``colormap``.
Convert a greyscale image (mode "L" or "LA") to a color image (mode
- "RGB" or "RGBA") by applying a colormap.
+ "RGB" or "RGBA") by applying a colormap. If floating point data being
+ colorized contains NaNs then the result will also contain NaNs instead
+ of a color from the colormap. Integer data that includes
+ a ``.attrs['_FillValue']`` will be converted to a floating point array
+ and values equal to ``_FillValue`` replaced with NaN before being
+ colorized.
To create a color image in mode "P" or "PA", use
:meth:`~XRImage.palettize`.
@@ -1514,13 +1284,8 @@ class XRImage:
if self.mode not in ("L", "LA"):
raise ValueError("Image should be grayscale to colorize")
- if self.mode == "LA":
- alpha = self.data.sel(bands=['A'])
- else:
- alpha = None
-
- l_data = self.data.sel(bands='L')
-
+ l_data = self._get_masked_floating_luminance_data()
+ alpha = self.data.sel(bands=['A']) if self.mode == "LA" else None
new_data = colormap.colorize(l_data.data)
if colormap.colors.shape[1] == 4:
@@ -1537,16 +1302,23 @@ class XRImage:
attrs = self.data.attrs
dims = self.data.dims
self.data = xr.DataArray(new_data, coords=coords, attrs=attrs, dims=dims)
- cmap_min = colormap.values[0]
- cmap_max = colormap.values[-1]
- scale_factor = 1.0 / (cmap_max - cmap_min)
- offset = -cmap_min * scale_factor
+ scale_factor, offset = self._get_colormap_scale_offset(colormap)
self.data.attrs.setdefault('enhancement_history', []).append({
'scale': scale_factor,
'offset': offset,
'colormap': colormap,
})
+ def _get_masked_floating_luminance_data(self):
+ l_data = self.data.sel(bands='L')
+ # mask any integer fields with _FillValue
+ # assume NaN is used otherwise
+ if self.mode == "L" and np.issubdtype(self.data.dtype, np.integer):
+ fill_value = self._get_input_fill_value(self.data)
+ if fill_value is not None:
+ l_data = l_data.where(l_data != fill_value)
+ return l_data
+
def palettize(self, colormap):
"""Palettize the current image using ``colormap``.
@@ -1616,17 +1388,21 @@ class XRImage:
self.data.data = new_data
self.data.coords['bands'] = list(mode)
# See docstring notes above for how scale/offset should be used
- cmap_min = colormap.values[0]
- cmap_max = colormap.values[-1]
- scale_factor = 1.0 / (cmap_max - cmap_min)
- offset = -cmap_min * scale_factor
-
+ scale_factor, offset = self._get_colormap_scale_offset(colormap)
self.data.attrs.setdefault('enhancement_history', []).append({
'scale': scale_factor,
'offset': offset,
'colormap': colormap,
})
+ @staticmethod
+ def _get_colormap_scale_offset(colormap):
+ cmap_min = colormap.values[0]
+ cmap_max = colormap.values[-1]
+ scale_factor = 1.0 / (cmap_max - cmap_min)
+ offset = -cmap_min * scale_factor
+ return scale_factor, offset
+
def blend(self, src):
r"""Alpha blend *src* on top of the current image.
View it on GitLab: https://salsa.debian.org/debian-gis-team/trollimage/-/commit/2f35a3f3555d5f029e7053f6cb309414c5f0ed46
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/trollimage/-/commit/2f35a3f3555d5f029e7053f6cb309414c5f0ed46
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20221023/13986bb2/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list