[Git][debian-gis-team/satpy][upstream] New upstream version 0.53.0

Antonio Valentino (@antonio.valentino) gitlab at salsa.debian.org
Sat Nov 23 19:14:47 GMT 2024



Antonio Valentino pushed to branch upstream at Debian GIS Project / satpy


Commits:
270deb06 by Antonio Valentino at 2024-11-09T16:55:13+00:00
New upstream version 0.53.0
- - - - -


27 changed files:

- .git_archival.txt
- .github/workflows/ci.yaml
- .github/workflows/deploy-sdist.yaml
- .pre-commit-config.yaml
- AUTHORS.md
- CHANGELOG.md
- satpy/enhancements/__init__.py
- satpy/etc/composites/abi.yaml
- satpy/etc/composites/ahi.yaml
- satpy/etc/composites/ami.yaml
- satpy/etc/composites/vii.yaml
- satpy/etc/composites/viirs.yaml
- satpy/etc/readers/mersi_ll_l1b.yaml
- satpy/etc/readers/modis_l1b.yaml
- satpy/modifiers/angles.py
- satpy/modifiers/atmosphere.py
- satpy/readers/hdfeos_base.py
- satpy/readers/hrit_jma.py
- satpy/readers/sar_c_safe.py
- satpy/readers/viirs_edr.py
- satpy/tests/enhancement_tests/test_enhancements.py
- satpy/tests/reader_tests/test_ahi_hrit.py
- satpy/tests/reader_tests/test_viirs_edr.py
- satpy/tests/test_composites.py
- satpy/tests/test_modifiers.py
- satpy/tests/writer_tests/test_cf.py
- satpy/writers/cf_writer.py


Changes:

=====================================
.git_archival.txt
=====================================
@@ -1,4 +1,4 @@
-node: 6814f7531746651537797f6c5dd9315c5d55e0fe
-node-date: 2024-10-23T12:46:22+02:00
-describe-name: v0.52.1
-ref-names: tag: v0.52.1
+node: bd29bb18a0ecf28dc49c7851057b8a67bccae03f
+node-date: 2024-11-08T16:40:56+01:00
+describe-name: v0.53.0
+ref-names: HEAD -> main, tag: v0.53.0


=====================================
.github/workflows/ci.yaml
=====================================
@@ -42,6 +42,8 @@ jobs:
           python-version: ${{ matrix.python-version }}
           activate-environment: test-environment
           channels: conda-forge
+          conda-remove-defaults: true
+          channel-priority: strict
 
       - name: Set cache environment variables
         shell: bash -l {0}


=====================================
.github/workflows/deploy-sdist.yaml
=====================================
@@ -23,7 +23,7 @@ jobs:
 
       - name: Publish package to PyPI
         if: github.event.action == 'published'
-        uses: pypa/gh-action-pypi-publish at v1.10.2
+        uses: pypa/gh-action-pypi-publish at v1.11.0
         with:
           user: __token__
           password: ${{ secrets.pypi_password }}


=====================================
.pre-commit-config.yaml
=====================================
@@ -3,7 +3,7 @@ fail_fast: false
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: 'v0.6.9'
+    rev: 'v0.7.2'
     hooks:
       - id: ruff
   - repo: https://github.com/pre-commit/pre-commit-hooks
@@ -19,7 +19,7 @@ repos:
       - id: bandit
         args: [--ini, .bandit]
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: 'v1.11.2'  # Use the sha / tag you want to point at
+    rev: 'v1.13.0'  # Use the sha / tag you want to point at
     hooks:
       - id: mypy
         additional_dependencies:


=====================================
AUTHORS.md
=====================================
@@ -38,6 +38,8 @@ The following people have made contributions to this project:
 - [Gerrit Holl (gerritholl)](https://github.com/gerritholl) - Deutscher Wetterdienst
 - [David Hoese (djhoese)](https://github.com/djhoese)
 - [Marc Honnorat (honnorat)](https://github.com/honnorat)
+- [Chung-Hsiang Horng(chorng)](https://github.com/chorng)
+- [Mario Hros (k3a)](https://github.com/k3a)
 - [Lloyd Hughes (system123)](https://github.com/system123)
 - [Sara Hörnquist (shornqui)](https://github.com/shornqui)
 - [Mikhail Itkin (mitkin)](https://github.com/mitkin)


=====================================
CHANGELOG.md
=====================================
@@ -1,3 +1,38 @@
+## Version 0.53.0 (2024/11/08)
+
+### Issues Closed
+
+* [Issue 2960](https://github.com/pytroll/satpy/issues/2960) - netcdf4 version causes error ([PR 2961](https://github.com/pytroll/satpy/pull/2961) by [@sfinkens](https://github.com/sfinkens))
+* [Issue 2952](https://github.com/pytroll/satpy/issues/2952) - Altitude, LandCover, and LandSeaMask are missing in the `mersi_ll_l1b` reader for FY3E L1B ([PR 2953](https://github.com/pytroll/satpy/pull/2953) by [@chorng](https://github.com/chorng))
+* [Issue 2948](https://github.com/pytroll/satpy/issues/2948) - "Missing" platform abbreviation causes unexpected error when loading data array in Scene ([PR 2949](https://github.com/pytroll/satpy/pull/2949) by [@joleenf](https://github.com/joleenf))
+
+In this release 3 issues were closed.
+
+### Pull Requests Merged
+
+#### Bugs fixed
+
+* [PR 2971](https://github.com/pytroll/satpy/pull/2971) - Pin flexparser before it breaks pint
+* [PR 2970](https://github.com/pytroll/satpy/pull/2970) - Remove rayleigh correction on VIIRS false_color for I02 band
+* [PR 2968](https://github.com/pytroll/satpy/pull/2968) - Remove unneeded call to private scipy function in SAR reader
+* [PR 2965](https://github.com/pytroll/satpy/pull/2965) - Fix MODIS readers chunking compatibility with newer dask
+* [PR 2961](https://github.com/pytroll/satpy/pull/2961) - Fix CF writer crashing with netcdf development version ([2960](https://github.com/pytroll/satpy/issues/2960))
+* [PR 2957](https://github.com/pytroll/satpy/pull/2957) - Bugfix the VIIRS lowres version of the day-microphysics.
+* [PR 2956](https://github.com/pytroll/satpy/pull/2956) - Fix cira stretch upcasting the data
+* [PR 2954](https://github.com/pytroll/satpy/pull/2954) - Fix Rayleigh correction to use the same datatype as the input data
+* [PR 2950](https://github.com/pytroll/satpy/pull/2950) - Fix dtype promotion in `SunZenithReduction`
+* [PR 2949](https://github.com/pytroll/satpy/pull/2949) - Add more platforms to VIIRS EDR reader ([2948](https://github.com/pytroll/satpy/issues/2948))
+* [PR 2930](https://github.com/pytroll/satpy/pull/2930) - Fix data type when getting a line offset for a segmented hrit_jma
+
+#### Features added
+
+* [PR 2973](https://github.com/pytroll/satpy/pull/2973) - Remove flexparser pinning
+* [PR 2953](https://github.com/pytroll/satpy/pull/2953) - Add altitude, landcover, and landseamask to mersi_ll_l1b reader ([2952](https://github.com/pytroll/satpy/issues/2952))
+* [PR 2946](https://github.com/pytroll/satpy/pull/2946) - Update MODIS L1b reader with additional geoinfo datasets
+
+In this release 14 pull requests were closed.
+
+
 ## Version 0.52.1 (2024/10/23)
 
 ### Issues Closed


=====================================
satpy/enhancements/__init__.py
=====================================
@@ -219,11 +219,12 @@ def cira_stretch(img, **kwargs):
 
 @exclude_alpha
 def _cira_stretch(band_data):
-    log_root = np.log10(0.0223)
+    dtype = band_data.dtype
+    log_root = np.log10(0.0223, dtype=dtype)
     denom = (1.0 - log_root) * 0.75
     band_data *= 0.01
     band_data = band_data.clip(np.finfo(float).eps)
-    band_data = np.log10(band_data)
+    band_data = np.log10(band_data, dtype=dtype)
     band_data -= log_root
     band_data /= denom
     return band_data


=====================================
satpy/etc/composites/abi.yaml
=====================================
@@ -36,7 +36,7 @@ composites:
       - name: C02
         modifiers: [sunz_corrected, rayleigh_corrected_crefl]
       - name: C03
-        modifiers: [sunz_corrected, rayleigh_corrected_crefl]
+        modifiers: [sunz_corrected]
     standard_name: toa_bidirectional_reflectance
 
   green_raw:
@@ -710,7 +710,7 @@ composites:
     compositor: !!python/name:satpy.composites.SelfSharpenedRGB
     prerequisites:
       - name: C03
-        modifiers: [sunz_corrected, rayleigh_corrected]
+        modifiers: [sunz_corrected]
       - name: C02
         modifiers: [sunz_corrected, rayleigh_corrected]
       - name: green


=====================================
satpy/etc/composites/ahi.yaml
=====================================
@@ -214,11 +214,11 @@ composites:
     compositor: !!python/name:satpy.composites.SelfSharpenedRGB
     prerequisites:
     - wavelength: 1.63
-      modifiers: [sunz_corrected] #, rayleigh_corrected]
+      modifiers: [sunz_corrected]
     - wavelength: 0.85
-      modifiers: [sunz_corrected] #, rayleigh_corrected]
+      modifiers: [sunz_corrected]
     - wavelength: 0.635
-      modifiers: [sunz_corrected] #, rayleigh_corrected]
+      modifiers: [sunz_corrected]
     high_resolution_band: blue
     standard_name: natural_color
 
@@ -303,34 +303,6 @@ composites:
       - name: B01
     standard_name: true_color_reproduction_color_stretch
 
-#  true_color_reducedsize_land:
-#    compositor: !!python/name:satpy.composites.GenericCompositor
-#    prerequisites:
-#    - wavelength: 0.65
-#      modifiers: [reducer4, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_land]
-#    - wavelength: 0.51
-#      modifiers: [reducer2, vegetation_corrected_reduced, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_land]
-#    - wavelength: 0.46
-#      modifiers: [reducer2, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_land]
-#    standard_name: true_color
-#
-#  true_color_reducedsize_marine_tropical:
-#    compositor: !!python/name:satpy.composites.GenericCompositor
-#    prerequisites:
-#    - wavelength: 0.65
-#      modifiers: [reducer4, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_marine_tropical]
-#    - wavelength: 0.51
-#      modifiers: [reducer2, vegetation_corrected_reduced, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_marine_tropical]
-#    - wavelength: 0.46
-#      modifiers: [reducer2, effective_solar_pathlength_corrected,
-#                  rayleigh_corrected_reducedsize_marine_tropical]
-#    standard_name: true_color
-
   day_microphysics_eum:
     compositor: !!python/name:satpy.composites.GenericCompositor
     prerequisites:


=====================================
satpy/etc/composites/ami.yaml
=====================================
@@ -137,11 +137,11 @@ composites:
     compositor: !!python/name:satpy.composites.SelfSharpenedRGB
     prerequisites:
       - name: NR016
-        modifiers: [sunz_corrected] #, rayleigh_corrected]
+        modifiers: [sunz_corrected]
       - name: VI008
-        modifiers: [sunz_corrected] #, rayleigh_corrected]
+        modifiers: [sunz_corrected]
       - name: VI006
-        modifiers: [sunz_corrected] #, rayleigh_corrected]
+        modifiers: [sunz_corrected]
     high_resolution_band: blue
     standard_name: natural_color
 


=====================================
satpy/etc/composites/vii.yaml
=====================================
@@ -87,7 +87,6 @@ composites:
       - compositor: !!python/name:satpy.composites.DifferenceCompositor
         prerequisites:
           - name: 'vii_3740'
-            modifiers: [ co2_corrected ]
           - name: 'vii_10690'
       - compositor: !!python/name:satpy.composites.DifferenceCompositor
         prerequisites:


=====================================
satpy/etc/composites/viirs.yaml
=====================================
@@ -232,7 +232,7 @@ composites:
       modifiers: [sunz_corrected, rayleigh_corrected]
     optional_prerequisites:
     - name: I02
-      modifiers: [sunz_corrected_iband, rayleigh_corrected_iband]
+      modifiers: [sunz_corrected_iband]
     standard_name: false_color
     high_resolution_band: green
 
@@ -405,7 +405,7 @@ composites:
     - name: M07
       modifiers: [sunz_corrected]
     - name: M12
-      modifiers: [nir_reflectance]
+      modifiers: [nir_reflectance_lowres]
     - M15
     standard_name: day_microphysics
 


=====================================
satpy/etc/readers/mersi_ll_l1b.yaml
=====================================
@@ -272,3 +272,27 @@ datasets:
     coordinates: [longitude, latitude]
     file_type: mersi_ll_l1b_1000_geo
     file_key: Geolocation/MoonAzimuth
+  altitude:
+    name: altitude
+    units: degree
+    standard_name: altitude
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: mersi_ll_l1b_1000_geo
+    file_key: Geolocation/Altitude
+  landcover:
+    name: landcover
+    units: degree
+    standard_name: landcover
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: mersi_ll_l1b_1000_geo
+    file_key: Geolocation/LandCover
+  landseamask:
+    name: landseamask
+    units: degree
+    standard_name: landseamask
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: mersi_ll_l1b_1000_geo
+    file_key: Geolocation/LandSeaMask


=====================================
satpy/etc/readers/modis_l1b.yaml
=====================================
@@ -486,6 +486,34 @@ datasets:
     coordinates: [longitude, latitude]
     file_type: [hdf_eos_geo, hdf_eos_data_1000m]
 
+  landsea_mask:
+    name: landsea_mask
+    sensor: modis
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: [hdf_eos_geo]
+
+  height:
+    name: height
+    sensor: modis
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: [hdf_eos_geo]
+
+  range:
+    name: range
+    sensor: modis
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: [hdf_eos_geo]
+
+  waterpresent:
+    name: waterpresent
+    sensor: modis
+    resolution: 1000
+    coordinates: [longitude, latitude]
+    file_type: [hdf_eos_geo]
+
 
 file_types:
   hdf_eos_data_250m:


=====================================
satpy/modifiers/angles.py
=====================================
@@ -572,7 +572,6 @@ def sunzen_reduction(data: da.Array,
     return da.map_blocks(_sunzen_reduction_ndarray, data, sunz, limit, max_sza, strength,
                          meta=np.array((), dtype=data.dtype), chunks=data.chunks)
 
-
 def _sunzen_reduction_ndarray(data: np.ndarray,
                               sunz: np.ndarray,
                               limit: float,
@@ -584,7 +583,7 @@ def _sunzen_reduction_ndarray(data: np.ndarray,
 
     # invert the reduction factor such that minimum reduction is done at `limit` and gradually increases towards max_sza
     with np.errstate(invalid="ignore"):  # we expect space pixels to be invalid
-        reduction_factor = 1. - np.log(reduction_factor + 1) / np.log(2)
+        reduction_factor = 1. - np.log2(reduction_factor + 1)
 
     # apply non-linearity to the reduction factor for a non-linear reduction of the signal. This can be used for a
     # slower or faster transision to higher/lower fractions at the ndvi extremes. If strength equals 1.0, this


=====================================
satpy/modifiers/atmosphere.py
=====================================
@@ -77,7 +77,9 @@ class PSPRayleighReflectance(ModifierBase):
         projectables = projectables + (optional_datasets or [])
         if len(projectables) != 6:
             vis, red = self.match_data_arrays(projectables)
-            sata, satz, suna, sunz = get_angles(vis)
+            # Adjust the angle data precision to match the data
+            # This does not affect the accuracy visibly
+            sata, satz, suna, sunz = [d.astype(vis.dtype) for d in get_angles(vis)]
         else:
             vis, red, sata, satz, suna, sunz = self.match_data_arrays(projectables)
             # First make sure the two azimuth angles are in the range 0-360:
@@ -97,7 +99,7 @@ class PSPRayleighReflectance(ModifierBase):
         aerosol_type = self.attrs.get("aerosol_type", "marine_clean_aerosol")
         reduce_lim_low = abs(self.attrs.get("reduce_lim_low", 70))
         reduce_lim_high = abs(self.attrs.get("reduce_lim_high", 105))
-        reduce_strength = np.clip(self.attrs.get("reduce_strength", 0), 0, 1)
+        reduce_strength = np.clip(self.attrs.get("reduce_strength", 0), 0, 1).astype(vis.dtype)
 
         logger.info("Removing Rayleigh scattering with atmosphere '%s' and "
                     "aerosol type '%s' for '%s'",


=====================================
satpy/readers/hdfeos_base.py
=====================================
@@ -238,7 +238,7 @@ class HDFEOSBaseFileReader(BaseFileHandler):
         return normalize_low_res_chunks(
             (1,) * num_nonyx_dims + ("auto", -1),
             var_shape,
-            (1,) * num_nonyx_dims + (scan_length_250m, -1),
+            (1,) * num_nonyx_dims + (scan_length_250m, var_shape[-1]),
             (1,) * num_nonyx_dims + (res_multiplier, res_multiplier),
             np.float32,
         )
@@ -333,6 +333,10 @@ class HDFEOSGeoReader(HDFEOSBaseFileReader):
         "satellite_zenith_angle": ("SensorZenith", "Sensor_Zenith"),
         "solar_azimuth_angle": ("SolarAzimuth", "SolarAzimuth"),
         "solar_zenith_angle": ("SolarZenith", "Solar_Zenith"),
+        "water_present": "WaterPresent",
+        "landsea_mask": "Land/SeaMask",
+        "height": "Height",
+        "range": "Range",
     }
 
     def __init__(self, filename, filename_info, filetype_info, **kwargs):


=====================================
satpy/readers/hrit_jma.py
=====================================
@@ -345,8 +345,8 @@ class HRITJMAFileHandler(HRITFileHandler):
         if self.is_segmented:
             # loff in the file specifies the offset of the full disk image
             # centre (1375/2750 for VIS/IR)
-            segment_number = self.mda["segment_sequence_number"] - 1
-            loff -= (self.mda["total_no_image_segm"] - segment_number - 1) * nlines
+            segment_number = int(self.mda["segment_sequence_number"]) - 1
+            loff -= (int(self.mda["total_no_image_segm"]) - segment_number - 1) * nlines
         elif self.area_id in (NORTH_HEMIS, SOUTH_HEMIS):
             # loff in the file specifies the start line of the half disk image
             # in the full disk image


=====================================
satpy/readers/sar_c_safe.py
=====================================
@@ -511,15 +511,14 @@ def intp(grid_x, grid_y, interpolator):
 
 def interpolate_xarray_linear(xpoints, ypoints, values, shape, chunks=CHUNK_SIZE):
     """Interpolate linearly, generating a dask array."""
-    from scipy.interpolate.interpnd import LinearNDInterpolator, _ndim_coords_from_arrays
+    from scipy.interpolate.interpnd import LinearNDInterpolator
 
     if isinstance(chunks, (list, tuple)):
         vchunks, hchunks = chunks
     else:
         vchunks, hchunks = chunks, chunks
-
-    points = _ndim_coords_from_arrays(np.vstack((np.asarray(ypoints, dtype=np.uint16),
-                                                 np.asarray(xpoints, dtype=np.uint16))).T)
+    points = np.vstack((np.asarray(ypoints, dtype=np.uint16),
+                        np.asarray(xpoints, dtype=np.uint16))).T
 
     interpolator = LinearNDInterpolator(points, values)
 


=====================================
satpy/readers/viirs_edr.py
=====================================
@@ -197,9 +197,12 @@ class VIIRSJRRFileHandler(BaseFileHandler):
         platform_path = self.filename_info["platform_shortname"]
         platform_dict = {"NPP": "Suomi-NPP",
                          "JPSS-1": "NOAA-20",
+                         "SNPP": "Suomi-NPP",
                          "J01": "NOAA-20",
+                         "N20": "NOAA-20",
                          "JPSS-2": "NOAA-21",
-                         "J02": "NOAA-21"}
+                         "J02": "NOAA-21",
+                         "N21": "NOAA-21"}
         return platform_dict[platform_path.upper()]
 
     def available_datasets(self, configured_datasets=None):


=====================================
satpy/tests/enhancement_tests/test_enhancements.py
=====================================
@@ -34,30 +34,55 @@ from satpy.enhancements import create_colormap, on_dask_array, on_separate_bands
 
 def run_and_check_enhancement(func, data, expected, **kwargs):
     """Perform basic checks that apply to multiple tests."""
+    pre_attrs = data.attrs
+    img = _get_enhanced_image(func, data, **kwargs)
+
+    _assert_image(img, pre_attrs, func.__name__, "palettes" in kwargs)
+    _assert_image_data(img, expected)
+
+
+def _get_enhanced_image(func, data, **kwargs):
     from trollimage.xrimage import XRImage
 
-    pre_attrs = data.attrs
     img = XRImage(data)
     func(img, **kwargs)
 
+    return img
+
+
+def _assert_image(img, pre_attrs, func_name, has_palette):
+    assert isinstance(img.data, xr.DataArray)
     assert isinstance(img.data.data, da.Array)
+
     old_keys = set(pre_attrs.keys())
     # It is OK to have "enhancement_history" added
     new_keys = set(img.data.attrs.keys()) - {"enhancement_history"}
     # In case of palettes are used, _FillValue is added.
     # Colorize doesn't add the fill value, so ignore that
-    if "palettes" in kwargs and func.__name__ != "colorize":
+    if has_palette and func_name != "colorize":
         assert "_FillValue" in new_keys
         # Remove it from further comparisons
         new_keys = new_keys - {"_FillValue"}
     assert old_keys == new_keys
 
-    res_data_arr = img.data
-    assert isinstance(res_data_arr, xr.DataArray)
-    assert isinstance(res_data_arr.data, da.Array)
-    res_data = res_data_arr.data.compute()  # mimics what xrimage geotiff writing does
+
+def _assert_image_data(img, expected, dtype=None):
+    # Compute the data to mimic what xrimage geotiff writing does
+    res_data = img.data.data.compute()
     assert not isinstance(res_data, da.Array)
     np.testing.assert_allclose(res_data, expected, atol=1.e-6, rtol=0)
+    if dtype:
+        assert img.data.dtype == dtype
+        assert res_data.dtype == dtype
+
+
+def run_and_check_enhancement_with_dtype(func, data, expected, **kwargs):
+    """Perform basic checks that apply to multiple tests."""
+    pre_attrs = data.attrs
+    img = _get_enhanced_image(func, data, **kwargs)
+
+    _assert_image(img, pre_attrs, func.__name__, "palettes" in kwargs)
+    _assert_image_data(img, expected, dtype=data.dtype)
 
 
 def identical_decorator(func):
@@ -109,14 +134,15 @@ class TestEnhancementStretch:
             exp_data = exp_data[np.newaxis, :, :]
         run_and_check_enhancement(_enh_func, in_data, exp_data)
 
-    def test_cira_stretch(self):
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_cira_stretch(self, dtype):
         """Test applying the cira_stretch."""
         from satpy.enhancements import cira_stretch
 
         expected = np.array([[
             [np.nan, -7.04045974, -7.04045974, 0.79630132, 0.95947296],
-            [1.05181359, 1.11651012, 1.16635571, 1.20691137, 1.24110186]]])
-        run_and_check_enhancement(cira_stretch, self.ch1, expected)
+            [1.05181359, 1.11651012, 1.16635571, 1.20691137, 1.24110186]]], dtype=dtype)
+        run_and_check_enhancement_with_dtype(cira_stretch, self.ch1.astype(dtype), expected)
 
     def test_reinhard(self):
         """Test the reinhard algorithm."""


=====================================
satpy/tests/reader_tests/test_ahi_hrit.py
=====================================
@@ -75,8 +75,8 @@ class TestHRITJMAFileHandler(unittest.TestCase):
         proj_h8 = b"GEOS(140.70)                    "
         proj_mtsat2 = b"GEOS(145.00)                    "
         proj_name = proj_h8 if platform == "Himawari-8" else proj_mtsat2
-        return {"image_segm_seq_no": segno,
-                "total_no_image_segm": numseg,
+        return {"image_segm_seq_no": np.uint8(segno),
+                "total_no_image_segm": np.uint8(numseg),
                 "projection_name": proj_name,
                 "projection_parameters": {
                     "a": 6378169.00,
@@ -85,10 +85,10 @@ class TestHRITJMAFileHandler(unittest.TestCase):
                 },
                 "cfac": 10233128,
                 "lfac": 10233128,
-                "coff": coff,
-                "loff": loff,
-                "number_of_columns": ncols,
-                "number_of_lines": nlines,
+                "coff": np.int32(coff),
+                "loff": np.int32(loff),
+                "number_of_columns": np.uint16(ncols),
+                "number_of_lines": np.uint16(nlines),
                 "image_data_function": idf,
                 "image_observation_time": self._get_acq_time(nlines)}
 


=====================================
satpy/tests/reader_tests/test_viirs_edr.py
=====================================
@@ -468,7 +468,8 @@ class TestVIIRSJRRReader:
         [
             ("npp", "Suomi-NPP"),
             ("JPSS-1", "NOAA-20"),
-            ("J01", "NOAA-20")
+            ("J01", "NOAA-20"),
+            ("n21", "NOAA-21")
         ])
     def test_get_platformname(self, surface_reflectance_file, filename_platform, exp_shortname):
         """Test finding start and end times of granules."""


=====================================
satpy/tests/test_composites.py
=====================================
@@ -257,20 +257,28 @@ class TestRatioSharpenedCompositors:
         with pytest.raises(ValueError, match="SelfSharpenedRGB requires at least one high resolution band, not 'None'"):
             comp((self.ds1, self.ds2, self.ds3))
 
-    def test_basic_no_high_res(self):
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_basic_no_high_res(self, dtype):
         """Test that three datasets can be passed without optional high res."""
         from satpy.composites import RatioSharpenedRGB
         comp = RatioSharpenedRGB(name="true_color")
-        res = comp((self.ds1, self.ds2, self.ds3))
+        res = comp((self.ds1.astype(dtype), self.ds2.astype(dtype), self.ds3.astype(dtype)))
         assert res.shape == (3, 2, 2)
+        assert res.dtype == dtype
+        assert res.values.dtype == dtype
 
-    def test_basic_no_sharpen(self):
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_basic_no_sharpen(self, dtype):
         """Test that color None does no sharpening."""
         from satpy.composites import RatioSharpenedRGB
         comp = RatioSharpenedRGB(name="true_color", high_resolution_band=None)
-        res = comp((self.ds1, self.ds2, self.ds3), optional_datasets=(self.ds4,))
+        res = comp((self.ds1.astype(dtype), self.ds2.astype(dtype), self.ds3.astype(dtype)),
+                   optional_datasets=(self.ds4.astype(dtype),))
         assert res.shape == (3, 2, 2)
+        assert res.dtype == dtype
+        assert res.values.dtype == dtype
 
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize(
         ("high_resolution_band", "neutral_resolution_band", "exp_r", "exp_g", "exp_b"),
         [
@@ -300,22 +308,26 @@ class TestRatioSharpenedCompositors:
              np.array([[1.0, 1.0], [np.nan, 1.0]], dtype=np.float64))
         ]
     )
-    def test_ratio_sharpening(self, high_resolution_band, neutral_resolution_band, exp_r, exp_g, exp_b):
+    def test_ratio_sharpening(self, high_resolution_band, neutral_resolution_band, exp_r, exp_g, exp_b, dtype):
         """Test RatioSharpenedRGB by different groups of high_resolution_band and neutral_resolution_band."""
         from satpy.composites import RatioSharpenedRGB
         comp = RatioSharpenedRGB(name="true_color", high_resolution_band=high_resolution_band,
                                  neutral_resolution_band=neutral_resolution_band)
-        res = comp((self.ds1, self.ds2, self.ds3), optional_datasets=(self.ds4,))
+        res = comp((self.ds1.astype(dtype), self.ds2.astype(dtype), self.ds3.astype(dtype)),
+                   optional_datasets=(self.ds4.astype(dtype),))
 
         assert "units" not in res.attrs
         assert isinstance(res, xr.DataArray)
         assert isinstance(res.data, da.Array)
+        assert res.dtype == dtype
 
         data = res.values
         np.testing.assert_allclose(data[0], exp_r, rtol=1e-5)
         np.testing.assert_allclose(data[1], exp_g, rtol=1e-5)
         np.testing.assert_allclose(data[2], exp_b, rtol=1e-5)
+        assert res.dtype == dtype
 
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize(
         ("exp_shape", "exp_r", "exp_g", "exp_b"),
         [
@@ -325,17 +337,19 @@ class TestRatioSharpenedCompositors:
              np.array([[16 / 3, 16 / 3], [16 / 3, 0]], dtype=np.float64))
         ]
     )
-    def test_self_sharpened_basic(self, exp_shape, exp_r, exp_g, exp_b):
+    def test_self_sharpened_basic(self, exp_shape, exp_r, exp_g, exp_b, dtype):
         """Test that three datasets can be passed without optional high res."""
         from satpy.composites import SelfSharpenedRGB
         comp = SelfSharpenedRGB(name="true_color")
-        res = comp((self.ds1, self.ds2, self.ds3))
-        data = res.values
+        res = comp((self.ds1.astype(dtype), self.ds2.astype(dtype), self.ds3.astype(dtype)))
+        assert res.dtype == dtype
 
+        data = res.values
         assert data.shape == exp_shape
         np.testing.assert_allclose(data[0], exp_r, rtol=1e-5)
         np.testing.assert_allclose(data[1], exp_g, rtol=1e-5)
         np.testing.assert_allclose(data[2], exp_b, rtol=1e-5)
+        assert data.dtype == dtype
 
 
 class TestDifferenceCompositor(unittest.TestCase):


=====================================
satpy/tests/test_modifiers.py
=====================================
@@ -135,29 +135,46 @@ class TestSunZenithCorrector:
         assert res.dtype == res_np.dtype
         assert "y" not in res.coords
         assert "x" not in res.coords
+        if as_32bit:
+            assert res.dtype == np.float32
 
-    def test_basic_lims_not_provided(self, sunz_ds1):
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_basic_lims_not_provided(self, sunz_ds1, dtype):
         """Test custom limits when SZA isn't provided."""
         from satpy.modifiers.geometry import SunZenithCorrector
         comp = SunZenithCorrector(name="sza_test", modifiers=tuple(), correction_limit=90)
-        res = comp((sunz_ds1,), test_attr="test")
-        np.testing.assert_allclose(res.values, np.array([[66.853262, 68.168939], [66.30742, 67.601493]]))
-
+        res = comp((sunz_ds1.astype(dtype),), test_attr="test")
+        expected = np.array([[66.853262, 68.168939], [66.30742, 67.601493]], dtype=dtype)
+        values = res.values
+        np.testing.assert_allclose(values, expected, rtol=1e-5)
+        assert res.dtype == dtype
+        assert values.dtype == dtype
+
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize("data_arr", [lazy_fixture("sunz_ds1"), lazy_fixture("sunz_ds1_stacked")])
-    def test_basic_default_provided(self, data_arr, sunz_sza):
+    def test_basic_default_provided(self, data_arr, sunz_sza, dtype):
         """Test default limits when SZA is provided."""
         from satpy.modifiers.geometry import SunZenithCorrector
         comp = SunZenithCorrector(name="sza_test", modifiers=tuple())
-        res = comp((data_arr, sunz_sza), test_attr="test")
-        np.testing.assert_allclose(res.values, np.array([[22.401667, 22.31777], [22.437503, 22.353533]]))
-
+        res = comp((data_arr.astype(dtype), sunz_sza.astype(dtype)), test_attr="test")
+        expected = np.array([[22.401667, 22.31777], [22.437503, 22.353533]], dtype=dtype)
+        values = res.values
+        np.testing.assert_allclose(values, expected)
+        assert res.dtype == dtype
+        assert values.dtype == dtype
+
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize("data_arr", [lazy_fixture("sunz_ds1"), lazy_fixture("sunz_ds1_stacked")])
-    def test_basic_lims_provided(self, data_arr, sunz_sza):
+    def test_basic_lims_provided(self, data_arr, sunz_sza, dtype):
         """Test custom limits when SZA is provided."""
         from satpy.modifiers.geometry import SunZenithCorrector
         comp = SunZenithCorrector(name="sza_test", modifiers=tuple(), correction_limit=90)
-        res = comp((data_arr, sunz_sza), test_attr="test")
-        np.testing.assert_allclose(res.values, np.array([[66.853262, 68.168939], [66.30742, 67.601493]]))
+        res = comp((data_arr.astype(dtype), sunz_sza.astype(dtype)), test_attr="test")
+        expected = np.array([[66.853262, 68.168939], [66.30742, 67.601493]], dtype=dtype)
+        values = res.values
+        np.testing.assert_allclose(values, expected, rtol=1e-5)
+        assert res.dtype == dtype
+        assert values.dtype == dtype
 
     def test_imcompatible_areas(self, sunz_ds2, sunz_sza):
         """Test sunz correction on incompatible areas."""
@@ -179,18 +196,28 @@ class TestSunZenithReducer:
         cls.custom = SunZenithReducer(name="sza_reduction_test_custom", modifiers=tuple(),
                                       correction_limit=70, max_sza=95, strength=3.0)
 
-    def test_default_settings(self, sunz_ds1, sunz_sza):
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_default_settings(self, sunz_ds1, sunz_sza, dtype):
         """Test default settings with sza data available."""
-        res = self.default((sunz_ds1, sunz_sza), test_attr="test")
-        np.testing.assert_allclose(res.values,
-                                   np.array([[0.02916261, 0.02839063], [0.02949383, 0.02871911]]),
-                                   rtol=1e-5)
-
-    def test_custom_settings(self, sunz_ds1, sunz_sza):
+        res = self.default((sunz_ds1.astype(dtype), sunz_sza.astype(dtype)), test_attr="test")
+        expected = np.array([[0.02916261, 0.02839063], [0.02949383, 0.02871911]], dtype=dtype)
+        assert res.dtype == dtype
+        values = res.values
+        assert values.dtype == dtype
+        np.testing.assert_allclose(values,
+                                   expected,
+                                   rtol=2e-5)
+
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
+    def test_custom_settings(self, sunz_ds1, sunz_sza, dtype):
         """Test custom settings with sza data available."""
-        res = self.custom((sunz_ds1, sunz_sza), test_attr="test")
-        np.testing.assert_allclose(res.values,
-                                   np.array([[0.01041319, 0.01030033], [0.01046164, 0.01034834]]),
+        res = self.custom((sunz_ds1.astype(dtype), sunz_sza.astype(dtype)), test_attr="test")
+        expected = np.array([[0.01041319, 0.01030033], [0.01046164, 0.01034834]], dtype=dtype)
+        assert res.dtype == dtype
+        values = res.values
+        assert values.dtype == dtype
+        np.testing.assert_allclose(values,
+                                   expected,
                                    rtol=1e-5)
 
     def test_invalid_max_sza(self, sunz_ds1, sunz_sza):
@@ -502,6 +529,7 @@ class TestPSPRayleighReflectance:
                               })
         return input_band, red_band, angle1, angle1, angle1, angle1
 
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize(
         ("name", "wavelength", "resolution", "aerosol_type", "reduce_lim_low", "reduce_lim_high", "reduce_strength",
          "exp_mean", "exp_unique"),
@@ -521,7 +549,7 @@ class TestPSPRayleighReflectance:
         ]
     )
     def test_rayleigh_corrector(self, name, wavelength, resolution, aerosol_type, reduce_lim_low, reduce_lim_high,
-                                reduce_strength, exp_mean, exp_unique):
+                                reduce_strength, exp_mean, exp_unique, dtype):
         """Test PSPRayleighReflectance with fake data."""
         from satpy.modifiers.atmosphere import PSPRayleighReflectance
         ray_cor = PSPRayleighReflectance(name=name, atmosphere="us-standard", aerosol_types=aerosol_type,
@@ -535,42 +563,48 @@ class TestPSPRayleighReflectance:
         assert ray_cor.attrs["reduce_strength"] == reduce_strength
 
         input_band, red_band, *_ = self._create_test_data(name, wavelength, resolution)
-        res = ray_cor([input_band, red_band])
+        res = ray_cor([input_band.astype(dtype), red_band.astype(dtype)])
 
         assert isinstance(res, xr.DataArray)
         assert isinstance(res.data, da.Array)
+        assert res.dtype == dtype
 
         data = res.values
         unique = np.unique(data[~np.isnan(data)])
         np.testing.assert_allclose(np.nanmean(data), exp_mean, rtol=1e-5)
         assert data.shape == (3, 5)
         np.testing.assert_allclose(unique, exp_unique, rtol=1e-5)
+        assert data.dtype == dtype
 
+    @pytest.mark.parametrize("dtype", [np.float32, np.float64])
     @pytest.mark.parametrize("as_optionals", [False, True])
-    def test_rayleigh_with_angles(self, as_optionals):
+    def test_rayleigh_with_angles(self, as_optionals, dtype):
         """Test PSPRayleighReflectance with angles provided."""
         from satpy.modifiers.atmosphere import PSPRayleighReflectance
         aerosol_type = "rayleigh_only"
         ray_cor = PSPRayleighReflectance(name="B01", atmosphere="us-standard", aerosol_types=aerosol_type)
-        prereqs, opt_prereqs = self._get_angles_prereqs_and_opts(as_optionals)
+        prereqs, opt_prereqs = self._get_angles_prereqs_and_opts(as_optionals, dtype)
         with mock.patch("satpy.modifiers.atmosphere.get_angles") as get_angles:
             res = ray_cor(prereqs, opt_prereqs)
         get_angles.assert_not_called()
 
         assert isinstance(res, xr.DataArray)
         assert isinstance(res.data, da.Array)
+        assert res.dtype == dtype
 
         data = res.values
         unique = np.unique(data[~np.isnan(data)])
         np.testing.assert_allclose(unique, np.array([-75.0, -37.71298492, 31.14350754]), rtol=1e-5)
         assert data.shape == (3, 5)
+        assert data.dtype == dtype
 
-    def _get_angles_prereqs_and_opts(self, as_optionals):
+    def _get_angles_prereqs_and_opts(self, as_optionals, dtype):
         wavelength = (0.45, 0.47, 0.49)
         resolution = 1000
         input_band, red_band, *angles = self._create_test_data("B01", wavelength, resolution)
-        prereqs = [input_band, red_band]
+        prereqs = [input_band.astype(dtype), red_band.astype(dtype)]
         opt_prereqs = []
+        angles = [a.astype(dtype) for a in angles]
         if as_optionals:
             opt_prereqs = angles
         else:


=====================================
satpy/tests/writer_tests/test_cf.py
=====================================
@@ -528,12 +528,19 @@ class TestNetcdfEncodingKwargs:
             assert f["test-array"].dtype == expected["dtype"]
             assert f["test-array"].encoding["complevel"] == expected["complevel"]
 
-    def test_warning_if_backends_dont_match(self, scene, filename, monkeypatch):
+    @pytest.mark.parametrize(
+        "versions",
+        [
+            {"netCDF4": "1.5.0", "libnetcdf": "4.9.1-development"},
+            {"netCDF4": "1.6.0", "libnetcdf": "invalid-version"}
+        ]
+    )
+    def test_warning_if_backends_dont_match(self, scene, filename, monkeypatch, versions):
         """Test warning if backends don't match."""
         import netCDF4
         with monkeypatch.context() as m:
-            m.setattr(netCDF4, "__version__", "1.5.0")
-            m.setattr(netCDF4, "__netcdf4libversion__", "4.9.1")
+            m.setattr(netCDF4, "__version__", versions["netCDF4"])
+            m.setattr(netCDF4, "__netcdf4libversion__", versions["libnetcdf"])
             with pytest.warns(UserWarning, match=r"Backend version mismatch"):
                 scene.save_datasets(filename=filename, writer="cf")
 


=====================================
satpy/writers/cf_writer.py
=====================================
@@ -158,7 +158,7 @@ import warnings
 
 import numpy as np
 import xarray as xr
-from packaging.version import Version
+from packaging.version import InvalidVersion, Version
 
 from satpy.cf.coords import EPOCH  # noqa: F401 (for backward compatibility)
 from satpy.writers import Writer
@@ -390,8 +390,26 @@ def _backend_versions_match():
 
 def _get_backend_versions():
     import netCDF4
+    libnetcdf_version = _parse_libnetcdf_version(
+        netCDF4.__netcdf4libversion__
+    )
     return {
         "netCDF4": Version(netCDF4.__version__),
-        "libnetcdf": Version(netCDF4.__netcdf4libversion__),
+        "libnetcdf": libnetcdf_version,
         "xarray": Version(xr.__version__)
     }
+
+
+def _parse_libnetcdf_version(version_str):
+    # Make libnetcdf development version compatible with PEP440
+    version_str = version_str.replace("development", "dev")
+    try:
+        return Version(version_str)
+    except InvalidVersion:
+        warnings.warn(
+            f"Unable to parse netcdf-c version {version_str}, "
+            f"using 0.0.0 as fallback",
+            UserWarning,
+            stacklevel=3
+        )
+        return Version("0.0.0")



View it on GitLab: https://salsa.debian.org/debian-gis-team/satpy/-/commit/270deb06076375f3ba116d695e033ae480250d36

-- 
View it on GitLab: https://salsa.debian.org/debian-gis-team/satpy/-/commit/270deb06076375f3ba116d695e033ae480250d36
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20241123/96b35af2/attachment-0001.htm>


More information about the Pkg-grass-devel mailing list