[Git][debian-gis-team/trollimage][upstream] New upstream version 1.19.0

Antonio Valentino (@antonio.valentino) gitlab at salsa.debian.org
Sun Oct 23 15:16:23 BST 2022



Antonio Valentino pushed to branch upstream at Debian GIS Project / trollimage


Commits:
2f35a3f3 by Antonio Valentino at 2022-10-23T13:57:04+00:00
New upstream version 1.19.0
- - - - -


9 changed files:

- CHANGELOG.md
- continuous_integration/environment.yaml
- doc/index.rst
- + doc/installation.rst
- setup.py
- + trollimage/_xrimage_rasterio.py
- trollimage/tests/test_image.py
- trollimage/version.py
- trollimage/xrimage.py


Changes:

=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,29 @@
+## Version 1.19.0 (2022/10/21)
+
+### Issues Closed
+
+* [Issue 72](https://github.com/pytroll/trollimage/issues/72) - Add install instructions ([PR 105](https://github.com/pytroll/trollimage/pull/105) by [@djhoese](https://github.com/djhoese))
+
+In this release 1 issue was closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 109](https://github.com/pytroll/trollimage/pull/109) - Fix XRImage.colorize to mask integer data with _FillValue ([545](https://github.com/ssec/polar2grid/issues/545))
+* [PR 108](https://github.com/pytroll/trollimage/pull/108) - Fix typo in rasterio AreaDefinition handling
+
+#### Features added
+
+* [PR 107](https://github.com/pytroll/trollimage/pull/107) - Refactor rasterio usage to improve import time
+
+#### Documentation changes
+
+* [PR 105](https://github.com/pytroll/trollimage/pull/105) - Add installation instructions ([72](https://github.com/pytroll/trollimage/issues/72))
+
+In this release 4 pull requests were closed.
+
+
 ## Version 1.18.3 (2022/03/07)
 
 ### Pull Requests Merged


=====================================
continuous_integration/environment.yaml
=====================================
@@ -14,6 +14,8 @@ dependencies:
   - codecov
   - rasterio
   - libtiff
+  - pyproj
+  - pyresample
   - pytest
   - pytest-cov
   - fsspec


=====================================
doc/index.rst
=====================================
@@ -1,22 +1,18 @@
-.. TrollImage documentation master file, created by
-   sphinx-quickstart on Mon Dec  2 09:40:29 2013.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-Welcome to TrollImage's documentation!
-======================================
+TrollImage
+==========
 
 .. image:: _static/hayan.png
 
 Get the source code here_ !
 
-.. _here: http://github.com/mraspaud/trollimage
+.. _here: http://github.com/pytroll/trollimage
 
 Contents:
 
 .. toctree::
    :maxdepth: 2
 
+   installation
    image
    xrimage
    colorspaces


=====================================
doc/installation.rst
=====================================
@@ -0,0 +1,95 @@
+============
+Installation
+============
+
+Trollimage is available from conda-forge (via conda), PyPI (via pip), or from
+source (via pip+git). The below instructions show how to install stable
+versions of trollimage or from the source code.
+
+Conda-based Installation
+========================
+
+Trollimage can be installed into a conda environment by installing the package
+from the conda-forge channel. If you do not already have access to a conda
+installation, we recommend installing
+`miniconda <https://docs.conda.io/en/latest/miniconda.html>`_ for the smallest
+and easiest installation.
+
+The commands below will use ``-c conda-forge`` to make sure packages are
+downloaded from the conda-forge channel. Alternatively, you can tell conda
+to always use conda-forge by running:
+
+.. code-block:: bash
+
+    $ conda config --add channels conda-forge
+
+In a new conda environment
+--------------------------
+
+We recommend creating a separate environment for your work with trollimage. To
+create a new environment and install trollimage all in one command you can
+run:
+
+.. code-block:: bash
+
+    $ conda create -c conda-forge -n my_trollimage_env python trollimage
+
+You must then activate the environment so any future python or
+conda commands will use this environment.
+
+.. code-block::
+
+    $ conda activate my_trollimage_env
+
+This method of creating an environment with trollimage (and optionally other
+packages) installed can generally be created faster than creating an
+environment and then later installing trollimage and other packages (see the
+section below).
+
+In an existing environment
+--------------------------
+
+.. note::
+
+    It is recommended that when first exploring trollimage, you create a new
+    environment specifically for this rather than modifying one used for
+    other work.
+
+If you already have a conda environment, it is activated, and would like to
+install trollimage into it, run the following:
+
+.. code-block:: bash
+
+    $ conda install -c conda-forge trollimage
+
+Pip-based Installation
+======================
+
+Trollimage is available from the Python Packaging Index (PyPI). A sandbox
+environment for ``trollimage`` can be created using
+`Virtualenv <http://pypi.python.org/pypi/virtualenv>`_.
+
+To install the `trollimage` package and the minimum amount of python dependencies:
+
+.. code-block:: bash
+
+    $ pip install trollimage
+
+Install from source
+===================
+
+To install directly from github into an existing environment (pip or conda-based):
+
+.. code-block:: bash
+
+    $ pip install git+https://github.com/pytroll/trollimage.git
+
+If you have the ``git`` command installed this will automatically download the
+source code and install it into the current environment. If you would like to
+modify a local copy of trollimage and see the effects immediately in your
+environment use the below command instead. This command should be run from the
+root your cloned version of the git repository (where the ``setup.py`` is located):
+
+.. code-block::
+
+    $ pip install -e .


=====================================
setup.py
=====================================
@@ -46,11 +46,11 @@ setup(name="trollimage",
       url="https://github.com/pytroll/trollimage",
       packages=['trollimage'],
       zip_safe=False,
-      install_requires=['numpy >=1.13', 'pillow'],
+      install_requires=['numpy>=1.13', 'pillow'],
       python_requires='>=3.6',
       extras_require={
-          'geotiff': ['rasterio'],
+          'geotiff': ['rasterio>=1.0'],
           'xarray': ['xarray', 'dask[array]'],
       },
-      tests_require=['xarray', 'dask[array]'],
+      tests_require=['xarray', 'dask[array]', 'pyproj', 'pyresample'],
       )


=====================================
trollimage/_xrimage_rasterio.py
=====================================
@@ -0,0 +1,272 @@
+# Copyright (c) 2022 trollimage developers
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""RasterIO-specific utilities needed by the XRImage class."""
+
+import logging
+import threading
+from contextlib import suppress
+
+import dask.array as da
+
+import rasterio
+from rasterio.enums import Resampling
+from rasterio.windows import Window
+
+logger = logging.getLogger(__name__)
+
+
+def get_data_arr_crs_transform_gcps(data_arr):
+    """Convert DataArray's AreaDefinition or SwathDefinition to rasterio geolocation information.
+
+    If possible, a rasterio geotransform object will be created. If it can't be made
+    then it is assumed the provided geometry object is a SwathDefinition and will be
+    checked for GCP coordinates (``swath_def.lons.attrs['gcps']``).
+
+    Args:
+        data_arr: Xarray DataArray.
+
+    Returns:
+        Tuple of (crs, transform, gcps). Each element defaults to ``None`` if
+        it couldn't be calculated.
+
+    """
+    crs = None
+    transform = None
+    gcps = None
+
+    try:
+        area = data_arr.attrs["area"]
+        if rasterio.__gdal_version__ >= '3':
+            wkt_version = 'WKT2_2018'
+        else:
+            wkt_version = 'WKT1_GDAL'
+        if hasattr(area, 'crs'):
+            crs = rasterio.crs.CRS.from_wkt(area.crs.to_wkt(version=wkt_version))
+        else:
+            crs = rasterio.crs.CRS(area.proj_dict)
+        west, south, east, north = area.area_extent
+        height, width = area.shape
+        transform = rasterio.transform.from_bounds(west, south,
+                                                   east, north,
+                                                   width, height)
+
+    except KeyError:  # No area
+        logger.info("Couldn't create geotransform")
+    except AttributeError:
+        try:
+            gcps = data_arr.attrs["area"].lons.attrs['gcps']
+            crs = data_arr.attrs["area"].lons.attrs['crs']
+        except KeyError:
+            logger.info("Couldn't create geotransform")
+    return crs, transform, gcps
+
+
+def split_regular_vs_lazy_tags(tags, r_file):
+    """Split tags into regular vs lazy (dask) tags."""
+    da_tags = []
+    for key, val in list(tags.items()):
+        try:
+            if isinstance(val.data, da.Array):
+                da_tags.append((val.data, RIOTag(r_file, key)))
+                tags.pop(key)
+            else:
+                tags[key] = val.item()
+        except AttributeError:
+            continue
+    return tags, da_tags
+
+
+class RIOFile(object):
+    """Rasterio wrapper to allow da.store to do window saving."""
+
+    def __init__(self, path, mode='w', **kwargs):
+        """Initialize the object."""
+        self.path = path
+        self.mode = mode
+        self.kwargs = kwargs
+        self.rfile = None
+        self.lock = threading.Lock()
+
+    @property
+    def width(self):
+        """Width of the band images."""
+        return self.kwargs['width']
+
+    @property
+    def height(self):
+        """Height of the band images."""
+        return self.kwargs['height']
+
+    @property
+    def closed(self):
+        """Check if the file is closed."""
+        return self.rfile is None or self.rfile.closed
+
+    def open(self, mode=None):
+        """Open the file."""
+        mode = mode or self.mode
+        if self.closed:
+            self.rfile = rasterio.open(self.path, mode, **self.kwargs)
+
+    def close(self):
+        """Close the file."""
+        with self.lock:
+            if not self.closed:
+                self.rfile.close()
+
+    def __enter__(self):
+        """Enter method."""
+        self.open()
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        """Exit method."""
+        self.close()
+
+    def __del__(self):
+        """Delete the instance."""
+        with suppress(IOError, OSError):
+            self.close()
+
+    @property
+    def colorinterp(self):
+        """Return the color interpretation of the image."""
+        return self.rfile.colorinterp
+
+    @colorinterp.setter
+    def colorinterp(self, val):
+        if rasterio.__version__.startswith("0."):
+            # not supported in older versions, set by PHOTOMETRIC tag
+            logger.warning("Rasterio 1.0+ required for setting colorinterp")
+        else:
+            self.rfile.colorinterp = val
+
+    def write(self, *args, **kwargs):
+        """Write to the file."""
+        with self.lock:
+            self.open('r+')
+            return self.rfile.write(*args, **kwargs)
+
+    def build_overviews(self, *args, **kwargs):
+        """Write overviews."""
+        with self.lock:
+            self.open('r+')
+            return self.rfile.build_overviews(*args, **kwargs)
+
+    def update_tags(self, *args, **kwargs):
+        """Update tags."""
+        with self.lock:
+            self.open('a')
+            return self.rfile.update_tags(*args, **kwargs)
+
+
+class RIOTag:
+    """Rasterio wrapper to allow da.store on tag."""
+
+    def __init__(self, rfile, name):
+        """Init the rasterio tag."""
+        self.rfile = rfile
+        self.name = name
+
+    def __setitem__(self, key, item):
+        """Put the data in the tag."""
+        kwargs = {self.name: item.item()}
+        self.rfile.update_tags(**kwargs)
+
+    def close(self):
+        """Close the file."""
+        return self.rfile.close()
+
+
+class RIODataset:
+    """A wrapper for a rasterio dataset."""
+
+    def __init__(self, rfile, overviews=None, overviews_resampling=None,
+                 overviews_minsize=256):
+        """Init the rasterio dataset."""
+        self.rfile = rfile
+        self.overviews = overviews
+        if overviews_resampling is None:
+            overviews_resampling = 'nearest'
+        self.overviews_resampling = Resampling[overviews_resampling]
+        self.overviews_minsize = overviews_minsize
+
+    def __setitem__(self, key, item):
+        """Put the data chunk in the image."""
+        if len(key) == 3:
+            indexes = list(range(
+                key[0].start + 1,
+                key[0].stop + 1,
+                key[0].step or 1
+            ))
+            y = key[1]
+            x = key[2]
+        else:
+            indexes = 1
+            y = key[0]
+            x = key[1]
+        chy_off = y.start
+        chy = y.stop - y.start
+        chx_off = x.start
+        chx = x.stop - x.start
+
+        # band indexes
+        self.rfile.write(item, window=Window(chx_off, chy_off, chx, chy),
+                         indexes=indexes)
+
+    def close(self):
+        """Close the file."""
+        if self.overviews is not None:
+            overviews = self.overviews
+            # it's an empty list
+            if len(overviews) == 0:
+                from rasterio.rio.overview import get_maximum_overview_level
+                width = self.rfile.width
+                height = self.rfile.height
+                max_level = get_maximum_overview_level(
+                    width, height, self.overviews_minsize)
+                overviews = [2 ** j for j in range(1, max_level + 1)]
+            logger.debug('Building overviews %s with %s resampling',
+                         str(overviews), self.overviews_resampling.name)
+            self.rfile.build_overviews(overviews, resampling=self.overviews_resampling)
+
+        return self.rfile.close()
+
+
+def color_interp(data):
+    """Get the color interpretation for this image."""
+    from rasterio.enums import ColorInterp as ci
+    modes = {'L': [ci.gray],
+             'LA': [ci.gray, ci.alpha],
+             'YCbCr': [ci.Y, ci.Cb, ci.Cr],
+             'YCbCrA': [ci.Y, ci.Cb, ci.Cr, ci.alpha]}
+
+    try:
+        mode = ''.join(data['bands'].values)
+        return modes[mode]
+    except KeyError:
+        colors = {'R': ci.red,
+                  'G': ci.green,
+                  'B': ci.blue,
+                  'A': ci.alpha,
+                  'C': ci.cyan,
+                  'M': ci.magenta,
+                  'Y': ci.yellow,
+                  'H': ci.hue,
+                  'S': ci.saturation,
+                  'L': ci.lightness,
+                  'K': ci.black,
+                  }
+        return [colors[band] for band in data['bands'].values]


=====================================
trollimage/tests/test_image.py
=====================================
@@ -34,6 +34,7 @@ import pytest
 
 from trollimage import image, xrimage
 from trollimage.colormap import Colormap, brbg
+from trollimage._xrimage_rasterio import RIODataset
 from .utils import assert_maximum_dask_computes
 
 
@@ -1001,7 +1002,7 @@ class TestXRImage:
             delay = img.save(tmp.name, compute=False)
             assert isinstance(delay, tuple)
             assert isinstance(delay[0], da.Array)
-            assert isinstance(delay[1], xrimage.RIODataset)
+            assert isinstance(delay[1], RIODataset)
             da.store(*delay)
             delay[1].close()
 
@@ -1046,46 +1047,22 @@ class TestXRImage:
         tags = _get_tags_after_writing_to_geotiff(data)
         assert "TIFFTAG_DATETIME" in tags
 
+    @pytest.mark.parametrize("output_ext", [".tif", ".tiff"])
+    @pytest.mark.parametrize("use_dask", [False, True])
     @pytest.mark.skipif(sys.platform.startswith('win'),
                         reason="'NamedTemporaryFile' not supported on Windows")
-    def test_save_geotiff_int(self):
+    def test_save_geotiff_int(self, output_ext, use_dask):
         """Test saving geotiffs when input data is int."""
-        from rasterio.control import GroundControlPoint
-
-        # numpy array image
-        data = xr.DataArray(np.arange(75).reshape(5, 5, 3), dims=[
-            'y', 'x', 'bands'], coords={'bands': ['R', 'G', 'B']})
-        img = xrimage.XRImage(data)
-        assert np.issubdtype(img.data.dtype, np.integer)
-        with NamedTemporaryFile(suffix='.tif') as tmp:
-            img.save(tmp.name)
-            with rio.open(tmp.name) as f:
-                file_data = f.read()
-            assert file_data.shape == (4, 5, 5)  # alpha band added
-            exp = np.arange(75).reshape(5, 5, 3)
-            np.testing.assert_allclose(file_data[0], exp[:, :, 0])
-            np.testing.assert_allclose(file_data[1], exp[:, :, 1])
-            np.testing.assert_allclose(file_data[2], exp[:, :, 2])
-            np.testing.assert_allclose(file_data[3], 255)
-        # test .tiff too
-        with NamedTemporaryFile(suffix='.tiff') as tmp:
-            img.save(tmp.name)
-            with rio.open(tmp.name) as f:
-                file_data = f.read()
-            assert file_data.shape == (4, 5, 5)  # alpha band added
-            exp = np.arange(75).reshape(5, 5, 3)
-            np.testing.assert_allclose(file_data[0], exp[:, :, 0])
-            np.testing.assert_allclose(file_data[1], exp[:, :, 1])
-            np.testing.assert_allclose(file_data[2], exp[:, :, 2])
-            np.testing.assert_allclose(file_data[3], 255)
+        arr = np.arange(75).reshape(5, 5, 3)
+        if use_dask:
+            arr = da.from_array(arr, chunks=5)
 
-        data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
+        data = xr.DataArray(arr,
                             dims=['y', 'x', 'bands'],
                             coords={'bands': ['R', 'G', 'B']})
         img = xrimage.XRImage(data)
         assert np.issubdtype(img.data.dtype, np.integer)
-        # Regular default save
-        with NamedTemporaryFile(suffix='.tif') as tmp:
+        with NamedTemporaryFile(suffix=output_ext) as tmp:
             img.save(tmp.name)
             with rio.open(tmp.name) as f:
                 file_data = f.read()
@@ -1096,20 +1073,28 @@ class TestXRImage:
             np.testing.assert_allclose(file_data[2], exp[:, :, 2])
             np.testing.assert_allclose(file_data[3], 255)
 
-        # dask delayed save
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_delayed(self):
+        """Test saving a geotiff but not computing the result immediately."""
+        data = xr.DataArray(np.arange(75).reshape(5, 5, 3), dims=[
+            'y', 'x', 'bands'], coords={'bands': ['R', 'G', 'B']})
+        img = xrimage.XRImage(data)
+        assert np.issubdtype(img.data.dtype, np.integer)
         with NamedTemporaryFile(suffix='.tif') as tmp:
             delay = img.save(tmp.name, compute=False)
             assert isinstance(delay, tuple)
             assert isinstance(delay[0], da.Array)
-            assert isinstance(delay[1], xrimage.RIODataset)
+            assert isinstance(delay[1], RIODataset)
             da.store(*delay)
             delay[1].close()
 
-        # GCPs
-        class FakeArea():
-            def __init__(self, lons, lats):
-                self.lons = lons
-                self.lats = lats
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_gcps(self):
+        """Test saving geotiffs when input data is int and has GCPs."""
+        from rasterio.control import GroundControlPoint
+        from pyresample import SwathDefinition
 
         gcps = [GroundControlPoint(1, 1, 100.0, 1000.0, z=0.0),
                 GroundControlPoint(2, 3, 400.0, 2000.0, z=0.0)]
@@ -1124,11 +1109,12 @@ class TestXRImage:
                             dims=['y', 'x'],
                             attrs={'gcps': gcps,
                                    'crs': crs})
+        swath_def = SwathDefinition(lons, lats)
 
         data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
                             dims=['y', 'x', 'bands'],
                             coords={'bands': ['R', 'G', 'B']},
-                            attrs={'area': FakeArea(lons, lats)})
+                            attrs={'area': swath_def})
         img = xrimage.XRImage(data)
         with NamedTemporaryFile(suffix='.tif') as tmp:
             img.save(tmp.name)
@@ -1142,7 +1128,41 @@ class TestXRImage:
                 assert ref.z == val.z
             assert crs == fcrs
 
-        # with rasterio colormap provided
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_no_gcp_swath(self):
+        """Test saving geotiffs when input data whose SwathDefinition has no GCPs.
+
+        If shouldn't fail, but it also shouldn't have a non-default transform.
+
+        """
+        from pyresample import SwathDefinition
+
+        lons = xr.DataArray(da.from_array(np.arange(25).reshape(5, 5), chunks=5),
+                            dims=['y', 'x'],
+                            attrs={})
+
+        lats = xr.DataArray(da.from_array(np.arange(25).reshape(5, 5), chunks=5),
+                            dims=['y', 'x'],
+                            attrs={})
+        swath_def = SwathDefinition(lons, lats)
+
+        data = xr.DataArray(da.from_array(np.arange(75).reshape(5, 5, 3), chunks=5),
+                            dims=['y', 'x', 'bands'],
+                            coords={'bands': ['R', 'G', 'B']},
+                            attrs={'area': swath_def})
+        img = xrimage.XRImage(data)
+        with NamedTemporaryFile(suffix='.tif') as tmp:
+            img.save(tmp.name)
+            with rio.open(tmp.name) as f:
+                assert f.transform.a == 1.0
+                assert f.transform.b == 0.0
+                assert f.transform.c == 0.0
+
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_rio_colormap(self):
+        """Test saving geotiffs when input data is int and a rasterio colormap is provided."""
         exp_cmap = {i: (i, 255 - i, i, 255) for i in range(256)}
         data = xr.DataArray(da.from_array(np.arange(81).reshape(9, 9, 1), chunks=9),
                             dims=['y', 'x', 'bands'],
@@ -1158,7 +1178,10 @@ class TestXRImage:
             np.testing.assert_allclose(file_data[0], exp[:, :, 0])
             assert cmap == exp_cmap
 
-        # with input fill value
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_with_fill(self):
+        """Test saving geotiffs when input data is int and a fill value is specified."""
         data = np.arange(75).reshape(5, 5, 3)
         # second pixel is all bad
         # pixel [0, 1, 1] is also naturally 5 by arange above
@@ -1181,7 +1204,20 @@ class TestXRImage:
             np.testing.assert_allclose(file_data[1], exp[:, :, 1])
             np.testing.assert_allclose(file_data[2], exp[:, :, 2])
 
-        # input fill value but alpha on output
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_with_fill_and_alpha(self):
+        """Test saving int geotiffs with a fill value and input alpha band."""
+        data = np.arange(75).reshape(5, 5, 3)
+        # second pixel is all bad
+        # pixel [0, 1, 1] is also naturally 5 by arange above
+        data[0, 1, :] = 5
+        data = xr.DataArray(da.from_array(data, chunks=5),
+                            dims=['y', 'x', 'bands'],
+                            attrs={'_FillValue': 5},
+                            coords={'bands': ['R', 'G', 'B']})
+        img = xrimage.XRImage(data)
+        assert np.issubdtype(img.data.dtype, np.integer)
         with NamedTemporaryFile(suffix='.tif') as tmp:
             img.save(tmp.name)
             with rio.open(tmp.name) as f:
@@ -1197,6 +1233,41 @@ class TestXRImage:
             np.testing.assert_allclose(file_data[2], exp[:, :, 2])
             np.testing.assert_allclose(file_data[3], exp_alpha)
 
+    @pytest.mark.skipif(sys.platform.startswith('win'),
+                        reason="'NamedTemporaryFile' not supported on Windows")
+    def test_save_geotiff_int_with_area_def(self):
+        """Test saving a integer image with an AreaDefinition."""
+        from pyproj import CRS
+        from pyresample import AreaDefinition
+        crs = CRS.from_user_input(4326)
+        area_def = AreaDefinition(
+            "test", "test", "",
+            crs, 5, 5, [-300, -250, 200, 250],
+        )
+
+        data = xr.DataArray(np.arange(75).reshape(5, 5, 3),
+                            dims=['y', 'x', 'bands'],
+                            coords={'bands': ['R', 'G', 'B']},
+                            attrs={"area": area_def})
+        img = xrimage.XRImage(data)
+        assert np.issubdtype(img.data.dtype, np.integer)
+        with NamedTemporaryFile(suffix='.tif') as tmp:
+            img.save(tmp.name)
+            with rio.open(tmp.name) as f:
+                file_data = f.read()
+                assert f.crs.to_epsg() == 4326
+                geotransform = f.transform
+                assert geotransform.a == 100
+                assert geotransform.c == -300
+                assert geotransform.e == -100
+                assert geotransform.f == 250
+            assert file_data.shape == (4, 5, 5)  # alpha band added
+            exp = np.arange(75).reshape(5, 5, 3)
+            np.testing.assert_allclose(file_data[0], exp[:, :, 0])
+            np.testing.assert_allclose(file_data[1], exp[:, :, 1])
+            np.testing.assert_allclose(file_data[2], exp[:, :, 2])
+            np.testing.assert_allclose(file_data[3], 255)
+
     @pytest.mark.skipif(sys.platform.startswith('win'),
                         reason="'NamedTemporaryFile' not supported on Windows")
     @pytest.mark.parametrize(
@@ -2068,6 +2139,25 @@ class TestXRImageColorize:
         assert img.data.attrs["enhancement_history"][-1]["offset"] == expected_offset
         assert isinstance(img.data.attrs["enhancement_history"][-1]["colormap"], Colormap)
 
+    def test_colorize_int_l_rgb_with_fills(self):
+        """Test integer data with _FillValue is masked (NaN) when colorized."""
+        arr = np.arange(75, dtype=np.uint8).reshape(5, 15)
+        arr[1, :] = 255
+        data = xr.DataArray(arr.copy(), dims=['y', 'x'],
+                            attrs={"_FillValue": 255})
+        new_brbg = brbg.set_range(5, 20, inplace=False)
+        img = xrimage.XRImage(data)
+        img.colorize(new_brbg)
+        values = img.data.compute()
+        assert values.shape == (3,) + arr.shape  # RGB
+        np.testing.assert_allclose(values[:, 1, :], np.nan)
+        assert np.count_nonzero(np.isnan(values)) == arr.shape[1] * 3
+
+        assert "enhancement_history" in img.data.attrs
+        assert img.data.attrs["enhancement_history"][-1]["scale"] == 1 / (20 - 5)
+        assert img.data.attrs["enhancement_history"][-1]["offset"] == -5 / (20 - 5)
+        assert isinstance(img.data.attrs["enhancement_history"][-1]["colormap"], Colormap)
+
     def test_colorize_la_rgb(self):
         """Test colorizing an LA image with an RGB colormap."""
         arr = np.arange(75).reshape(5, 15) / 74.


=====================================
trollimage/version.py
=====================================
@@ -23,9 +23,9 @@ def get_keywords():
     # setup.py/versioneer.py will grep for the variable names, so they must
     # each be defined on a line of their own. _version.py will just call
     # get_keywords().
-    git_refnames = " (HEAD -> main, tag: v1.18.3)"
-    git_full = "baae46cbe74beffb6cd669978ec92a44396bf5f7"
-    git_date = "2022-03-07 12:29:07 -0600"
+    git_refnames = " (HEAD -> main, tag: v1.19.0)"
+    git_full = "93e547a45a1c3a13768da4a5169a3f5bf5ebc75a"
+    git_date = "2022-10-21 09:50:00 -0500"
     keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
     return keywords
 


=====================================
trollimage/xrimage.py
=====================================
@@ -35,9 +35,7 @@ chunks can be saved in parallel.
 import logging
 import numbers
 import os
-import threading
 import warnings
-from contextlib import suppress
 
 import dask
 import dask.array as da
@@ -47,209 +45,9 @@ from PIL import Image as PILImage
 from dask.delayed import delayed
 from trollimage.image import check_image_format
 
-try:
-    import rasterio
-    from rasterio.enums import Resampling
-except ImportError:
-    rasterio = None
-
-try:
-    # rasterio 1.0+
-    from rasterio.windows import Window
-except ImportError:
-    # raster 0.36.0
-    # remove this once rasterio 1.0+ is officially available
-    def Window(x_off, y_off, x_size, y_size):
-        """Replace the missing Window object in rasterio < 1.0."""
-        return (y_off, y_off + y_size), (x_off, x_off + x_size)
-
 logger = logging.getLogger(__name__)
 
 
-class RIOFile(object):
-    """Rasterio wrapper to allow da.store to do window saving."""
-
-    def __init__(self, path, mode='w', **kwargs):
-        """Initialize the object."""
-        self.path = path
-        self.mode = mode
-        self.kwargs = kwargs
-        self.rfile = None
-        self.lock = threading.Lock()
-
-    @property
-    def width(self):
-        """Width of the band images."""
-        return self.kwargs['width']
-
-    @property
-    def height(self):
-        """Height of the band images."""
-        return self.kwargs['height']
-
-    @property
-    def closed(self):
-        """Check if the file is closed."""
-        return self.rfile is None or self.rfile.closed
-
-    def open(self, mode=None):
-        """Open the file."""
-        mode = mode or self.mode
-        if self.closed:
-            self.rfile = rasterio.open(self.path, mode, **self.kwargs)
-
-    def close(self):
-        """Close the file."""
-        with self.lock:
-            if not self.closed:
-                self.rfile.close()
-
-    def __enter__(self):
-        """Enter method."""
-        self.open()
-        return self
-
-    def __exit__(self, exc_type, exc_value, traceback):
-        """Exit method."""
-        self.close()
-
-    def __del__(self):
-        """Delete the instance."""
-        with suppress(IOError, OSError):
-            self.close()
-
-    @property
-    def colorinterp(self):
-        """Return the color interpretation of the image."""
-        return self.rfile.colorinterp
-
-    @colorinterp.setter
-    def colorinterp(self, val):
-        if rasterio.__version__.startswith("0."):
-            # not supported in older versions, set by PHOTOMETRIC tag
-            logger.warning("Rasterio 1.0+ required for setting colorinterp")
-        else:
-            self.rfile.colorinterp = val
-
-    def write(self, *args, **kwargs):
-        """Write to the file."""
-        with self.lock:
-            self.open('r+')
-            return self.rfile.write(*args, **kwargs)
-
-    def build_overviews(self, *args, **kwargs):
-        """Write overviews."""
-        with self.lock:
-            self.open('r+')
-            return self.rfile.build_overviews(*args, **kwargs)
-
-    def update_tags(self, *args, **kwargs):
-        """Update tags."""
-        with self.lock:
-            self.open('a')
-            return self.rfile.update_tags(*args, **kwargs)
-
-
-class RIOTag:
-    """Rasterio wrapper to allow da.store on tag."""
-
-    def __init__(self, rfile, name):
-        """Init the rasterio tag."""
-        self.rfile = rfile
-        self.name = name
-
-    def __setitem__(self, key, item):
-        """Put the data in the tag."""
-        kwargs = {self.name: item.item()}
-        self.rfile.update_tags(**kwargs)
-
-    def close(self):
-        """Close the file."""
-        return self.rfile.close()
-
-
-class RIODataset:
-    """A wrapper for a rasterio dataset."""
-
-    def __init__(self, rfile, overviews=None, overviews_resampling=None,
-                 overviews_minsize=256):
-        """Init the rasterio dataset."""
-        self.rfile = rfile
-        self.overviews = overviews
-        if overviews_resampling is None:
-            overviews_resampling = 'nearest'
-        self.overviews_resampling = Resampling[overviews_resampling]
-        self.overviews_minsize = overviews_minsize
-
-    def __setitem__(self, key, item):
-        """Put the data chunk in the image."""
-        if len(key) == 3:
-            indexes = list(range(
-                key[0].start + 1,
-                key[0].stop + 1,
-                key[0].step or 1
-            ))
-            y = key[1]
-            x = key[2]
-        else:
-            indexes = 1
-            y = key[0]
-            x = key[1]
-        chy_off = y.start
-        chy = y.stop - y.start
-        chx_off = x.start
-        chx = x.stop - x.start
-
-        # band indexes
-        self.rfile.write(item, window=Window(chx_off, chy_off, chx, chy),
-                         indexes=indexes)
-
-    def close(self):
-        """Close the file."""
-        if self.overviews is not None:
-            overviews = self.overviews
-            # it's an empty list
-            if len(overviews) == 0:
-                from rasterio.rio.overview import get_maximum_overview_level
-                width = self.rfile.width
-                height = self.rfile.height
-                max_level = get_maximum_overview_level(
-                    width, height, self.overviews_minsize)
-                overviews = [2 ** j for j in range(1, max_level + 1)]
-            logger.debug('Building overviews %s with %s resampling',
-                         str(overviews), self.overviews_resampling.name)
-            self.rfile.build_overviews(overviews, resampling=self.overviews_resampling)
-
-        return self.rfile.close()
-
-
-def color_interp(data):
-    """Get the color interpretation for this image."""
-    from rasterio.enums import ColorInterp as ci
-    modes = {'L': [ci.gray],
-             'LA': [ci.gray, ci.alpha],
-             'YCbCr': [ci.Y, ci.Cb, ci.Cr],
-             'YCbCrA': [ci.Y, ci.Cb, ci.Cr, ci.alpha]}
-
-    try:
-        mode = ''.join(data['bands'].values)
-        return modes[mode]
-    except KeyError:
-        colors = {'R': ci.red,
-                  'G': ci.green,
-                  'B': ci.blue,
-                  'A': ci.alpha,
-                  'C': ci.cyan,
-                  'M': ci.magenta,
-                  'Y': ci.yellow,
-                  'H': ci.hue,
-                  'S': ci.saturation,
-                  'L': ci.lightness,
-                  'K': ci.black,
-                  }
-        return [colors[band] for band in data['bands'].values]
-
-
 def combine_scales_offsets(*args):
     """Combine ``(scale, offset)`` tuples in one, considering they are applied from left to right.
 
@@ -445,12 +243,15 @@ class XRImage:
         """
         kwformat = format_kwargs.pop('format', None)
         fformat = fformat or kwformat or os.path.splitext(filename)[1][1:]
-        if fformat in ('tif', 'tiff', 'jp2') and rasterio:
-
-            return self.rio_save(filename, fformat=fformat, driver=driver,
-                                 fill_value=fill_value, compute=compute,
-                                 keep_palette=keep_palette, cmap=cmap,
-                                 **format_kwargs)
+        if fformat in ('tif', 'tiff', 'jp2'):
+            try:
+                return self.rio_save(filename, fformat=fformat, driver=driver,
+                                     fill_value=fill_value, compute=compute,
+                                     keep_palette=keep_palette, cmap=cmap,
+                                     **format_kwargs)
+            except ImportError:
+                logger.warning("Missing 'rasterio' dependency to save GeoTIFF "
+                               "image. Will try using PIL...")
         return self.pil_save(filename, fformat, fill_value,
                              compute=compute, **format_kwargs)
 
@@ -525,6 +326,7 @@ class XRImage:
             The delayed or computed result of the saving.
 
         """
+        from ._xrimage_rasterio import RIOFile, RIODataset, split_regular_vs_lazy_tags
         fformat = fformat or os.path.splitext(filename)[1][1:]
         drivers = {'jpg': 'JPEG',
                    'png': 'PNG',
@@ -570,36 +372,14 @@ class XRImage:
                 format_kwargs.setdefault('photometric',
                                          photometric_map[mode.upper()])
 
-            try:
-                area = data.attrs['area']
-                if rasterio.__gdal_version__ >= '3':
-                    wkt_version = 'WKT2_2018'
-                else:
-                    wkt_version = 'WKT1_GDAL'
-                if hasattr(area, 'crs'):
-                    crs = rasterio.crs.CRS.from_wkt(area.crs.to_wkt(version=wkt_version))
-                else:
-                    crs = rasterio.crs.CRS(data.attrs['area'].proj_dict)
-                west, south, east, north = data.attrs['area'].area_extent
-                height, width = data.sizes['y'], data.sizes['x']
-                transform = rasterio.transform.from_bounds(west, south,
-                                                           east, north,
-                                                           width, height)
-
-            except KeyError:  # No area
-                logger.info("Couldn't create geotransform")
-            except AttributeError:
-                try:
-                    gcps = data.attrs['area'].lons.attrs['gcps']
-                    crs = data.attrs['area'].lons.attrs['crs']
-                except KeyError:
-                    logger.info("Couldn't create geotransform")
+            from ._xrimage_rasterio import get_data_arr_crs_transform_gcps
+            crs, transform, gcps = get_data_arr_crs_transform_gcps(data)
 
             stime = data.attrs.get("start_time")
             if stime:
                 stime_str = stime.strftime("%Y:%m:%d %H:%M:%S")
                 tags.setdefault('TIFFTAG_DATETIME', stime_str)
-        elif driver == 'JPEG' and 'A' in mode:
+        if driver == 'JPEG' and 'A' in mode:
             raise ValueError('JPEG does not support alpha')
 
         enhancement_colormap = self._get_colormap_from_enhancement_history(data)
@@ -624,6 +404,7 @@ class XRImage:
                          **format_kwargs)
         r_file.open()
         if not keep_palette:
+            from ._xrimage_rasterio import color_interp
             r_file.colorinterp = color_interp(data)
 
         if keep_palette and cmap is not None:
@@ -636,8 +417,7 @@ class XRImage:
             except AttributeError:
                 raise ValueError("Colormap is not formatted correctly")
 
-        tags, da_tags = self._split_regular_vs_lazy_tags(tags, r_file)
-
+        tags, da_tags = split_regular_vs_lazy_tags(tags, r_file)
         r_file.rfile.update_tags(**tags)
         r_dataset = RIODataset(r_file, overviews,
                                overviews_resampling=overviews_resampling,
@@ -668,21 +448,6 @@ class XRImage:
                 return enhance_dict["colormap"]
         return None
 
-    @staticmethod
-    def _split_regular_vs_lazy_tags(tags, r_file):
-        """Split tags into regular vs lazy (dask) tags."""
-        da_tags = []
-        for key, val in list(tags.items()):
-            try:
-                if isinstance(val.data, da.Array):
-                    da_tags.append((val.data, RIOTag(r_file, key)))
-                    tags.pop(key)
-                else:
-                    tags[key] = val.item()
-            except AttributeError:
-                continue
-        return tags, da_tags
-
     def pil_save(self, filename, fformat=None, fill_value=None,
                  compute=True, **format_kwargs):
         """Save the image to the given *filename* using PIL.
@@ -1497,7 +1262,12 @@ class XRImage:
         """Colorize the current image using ``colormap``.
 
         Convert a greyscale image (mode "L" or "LA") to a color image (mode
-        "RGB" or "RGBA") by applying a colormap.
+        "RGB" or "RGBA") by applying a colormap. If floating point data being
+        colorized contains NaNs then the result will also contain NaNs instead
+        of a color from the colormap. Integer data that includes
+        a ``.attrs['_FillValue']`` will be converted to a floating point array
+        and values equal to ``_FillValue`` replaced with NaN before being
+        colorized.
 
         To create a color image in mode "P" or "PA", use
         :meth:`~XRImage.palettize`.
@@ -1514,13 +1284,8 @@ class XRImage:
         if self.mode not in ("L", "LA"):
             raise ValueError("Image should be grayscale to colorize")
 
-        if self.mode == "LA":
-            alpha = self.data.sel(bands=['A'])
-        else:
-            alpha = None
-
-        l_data = self.data.sel(bands='L')
-
+        l_data = self._get_masked_floating_luminance_data()
+        alpha = self.data.sel(bands=['A']) if self.mode == "LA" else None
         new_data = colormap.colorize(l_data.data)
 
         if colormap.colors.shape[1] == 4:
@@ -1537,16 +1302,23 @@ class XRImage:
         attrs = self.data.attrs
         dims = self.data.dims
         self.data = xr.DataArray(new_data, coords=coords, attrs=attrs, dims=dims)
-        cmap_min = colormap.values[0]
-        cmap_max = colormap.values[-1]
-        scale_factor = 1.0 / (cmap_max - cmap_min)
-        offset = -cmap_min * scale_factor
+        scale_factor, offset = self._get_colormap_scale_offset(colormap)
         self.data.attrs.setdefault('enhancement_history', []).append({
             'scale': scale_factor,
             'offset': offset,
             'colormap': colormap,
         })
 
+    def _get_masked_floating_luminance_data(self):
+        l_data = self.data.sel(bands='L')
+        # mask any integer fields with _FillValue
+        # assume NaN is used otherwise
+        if self.mode == "L" and np.issubdtype(self.data.dtype, np.integer):
+            fill_value = self._get_input_fill_value(self.data)
+            if fill_value is not None:
+                l_data = l_data.where(l_data != fill_value)
+        return l_data
+
     def palettize(self, colormap):
         """Palettize the current image using ``colormap``.
 
@@ -1616,17 +1388,21 @@ class XRImage:
         self.data.data = new_data
         self.data.coords['bands'] = list(mode)
         # See docstring notes above for how scale/offset should be used
-        cmap_min = colormap.values[0]
-        cmap_max = colormap.values[-1]
-        scale_factor = 1.0 / (cmap_max - cmap_min)
-        offset = -cmap_min * scale_factor
-
+        scale_factor, offset = self._get_colormap_scale_offset(colormap)
         self.data.attrs.setdefault('enhancement_history', []).append({
             'scale': scale_factor,
             'offset': offset,
             'colormap': colormap,
         })
 
+    @staticmethod
+    def _get_colormap_scale_offset(colormap):
+        cmap_min = colormap.values[0]
+        cmap_max = colormap.values[-1]
+        scale_factor = 1.0 / (cmap_max - cmap_min)
+        offset = -cmap_min * scale_factor
+        return scale_factor, offset
+
     def blend(self, src):
         r"""Alpha blend *src* on top of the current image.
 



View it on GitLab: https://salsa.debian.org/debian-gis-team/trollimage/-/commit/2f35a3f3555d5f029e7053f6cb309414c5f0ed46

-- 
View it on GitLab: https://salsa.debian.org/debian-gis-team/trollimage/-/commit/2f35a3f3555d5f029e7053f6cb309414c5f0ed46
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20221023/13986bb2/attachment-0001.htm>


More information about the Pkg-grass-devel mailing list