[Git][debian-gis-team/xarray-eopf][upstream] New upstream version 0.2.9
Antonio Valentino (@antonio.valentino)
gitlab at salsa.debian.org
Mon Jun 22 07:26:37 BST 2026
Antonio Valentino pushed to branch upstream at Debian GIS Project / xarray-eopf
Commits:
8fecd399 by Antonio Valentino at 2026-06-22T06:06:53+00:00
New upstream version 0.2.9
- - - - -
19 changed files:
- CHANGES.md
- docs/examples/sentinel_1_analysis.ipynb
- docs/guide.md
- environment.yml
- + examples/sentinel_1_analysis.ipynb
- integration/test_sen1_analysis.py
- integration/test_sen2_analysis.py
- pyproject.toml
- tests/amodes/test_sentinel1.py
- tests/amodes/test_sentinel2.py
- tests/amodes/test_sentinel3.py
- tests/helpers/__init__.py
- tests/helpers/sentinel1.py
- xarray_eopf/amodes/sentinel1.py
- xarray_eopf/amodes/sentinel2.py
- xarray_eopf/amodes/sentinel3.py
- xarray_eopf/backend.py
- xarray_eopf/utils.py
- xarray_eopf/version.py
Changes:
=====================================
CHANGES.md
=====================================
@@ -1,10 +1,16 @@
+## Changes in 0.2.9 (from 2026-06-03)
+
+- Added support for Sentinel-1 Level-2 OCN analysis mode.
+- Fixed an issue in Sentinel-1 GRD analysis mode that could produce NaN values along
+ the edges of the bounding box.
+
## Changes in 0.2.8 (from 2026-05-08)
- Fix package discovery in `pyproject.toml` to ensure only `xarray_eopf`
(and its subpackages) is included in the PyPI wheel.
- Remove the `coarsen.py` module, as it has been moved to [xcube-resampling](https://github.com/xcube-dev/xcube-resampling)
and is no longer used internally.
-- Add support for Sentinel-1 GRD analysis mode.
+- Add support for Sentinel-1 Level-1 GRD analysis mode.
- Updated year in the headers.
- Added footprint-based subsetting for Sentinel-3 OLCI and SLSTR LST using STAC
metadata, improving performance by avoiding full latitude/longitude grid downloads
=====================================
docs/examples/sentinel_1_analysis.ipynb
=====================================
The diff for this file was not included because it is too large.
=====================================
docs/guide.md
=====================================
@@ -55,13 +55,17 @@ Additional parameters specific to each Sentinel mission are described below.
#### Remarks on Specific Sentinel Missions
-##### Sentinel-1
+Processing workflows differ significantly across Sentinel-1 product types. Therefore,
+each product family is documented in its own dedicated section.
+
+> **Note:** Support for SLC products is planned for a future release.
+
+##### Sentinel-1 Level-1 GRD
> Note: Support for Sentinel-1 GRD products in analysis mode is
-> currently experimental and undergoing validation. Support for SLC products is
-> planned for a future release.
+> currently experimental and undergoing validation.
-Sentinel-1 GRD data is provided in radar geometry, defined by the coordinates
+Sentinel-1 Level-1 GRD data is provided in radar geometry, defined by the coordinates
(`azimuth_time`, `ground_range`). To transform this data into an
**analysis-ready dataset**, the following processing steps are applied:
@@ -77,17 +81,16 @@ Sentinel-1 GRD data is provided in radar geometry, defined by the coordinates
📖 [D. Small, *Flattening Gamma: Radiometric Terrain Correction for SAR Imagery*](https://ieeexplore.ieee.org/document/5752845)
-
**Supported Products:**
-- [Sentinel-1 GRD](https://stac.browser.user.eopf.eodc.eu/collections/sentinel-1-l1-grd)
+- [Sentinel-1 Level-1 GRD](https://stac.browser.user.eopf.eodc.eu/collections/sentinel-1-l1-grd)
**Supported Variables**
- **Polarization bands**:
`vv`, `vh`, `hh`, `hv` *(each GRD product contains only a subset of these bands)*
-**Specific Sentinel-1 parameters `**kwargs`:**
+**Specific Sentinel-1 Level-1 GRD parameters `**kwargs`:**
- `crs`: Coordinate reference system of the output dataset. Can be provided as a
`str` or a `pyproj.CRS` object. If a string is given, it will be parsed using
@@ -113,6 +116,60 @@ Examples:
- [Docs – Sentinel-1 Analysis Mode](https://eopf-sample-service.github.io/xarray-eopf/examples/sentinel_1_analysis/)
+##### Sentinel-1 Level-2 OCN
+
+Sentinel-1 Level-2 OCN products are geolocated datasets provided on their
+**native grid**, where each pixel is associated with an individual
+latitude/longitude pair. As a result, the spatial coordinates form a
+**2D irregular grid** rather than a regular latitude/longitude raster.
+
+The analysis mode uses the [rectification algorithm from xcube-resampling](https://xcube-dev.github.io/xcube-resampling/guide/#3-rectification)
+to transform the irregular grid into a regular spatial grid with 1D latitude and
+longitude coordinates.
+
+**Supported Products:**
+
+- [Sentinel-1 Level-2 OCN](https://stac.browser.user.eopf.eodc.eu/collections/sentinel-1-l2-ocn)
+
+**Supported Variables**
+
+- **Wind Variables**:
+ `wind_speed`, `wind_direction`
+- **Auxiliary Variables**:
+ `inversion_quality`, `wind_quality`, `percentage_bright_points`
+
+**Specific Sentinel-1 Level-2 OCN parameters `**kwargs`:**
+
+- `crs`: Coordinate reference system of the output dataset. Can be provided as a
+ `str` or a `pyproj.CRS` object. If a string is given, it will be parsed using
+ [`pyproj.crs.CRS.from_string`](https://pyproj4.github.io/pyproj/dev/api/crs/crs.html#pyproj.crs.CRS.from_string).
+ If not specified, [EPSG:4326](https://epsg.io/4326) is used.
+- `resolution`: Target resolution for all spatial variables expressed in the units
+ of the specified `crs`. If not specified, the resolution is derived from the data.
+- `interp_methods`: for upsampling / interpolating
+ spatial data variables. Can be a single interpolation method for all
+ variables or a dictionary mapping variable names or dtypes to
+ interpolation method (for more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms)).
+ Supported methods include:
+
+ - `0` (nearest neighbor, default for integer arrays)
+ - `1` (linear / bilinear, default for float arrays)
+ - `"nearest"`
+ - `"triangular"`
+ - `"bilinear"`
+
+- `agg_methods`: Optional aggregation methods to be used for downsampling
+ spatial data variables / bands. Can be a single method for all variables or
+ a dictionary mapping variable names or dtypes to methods. Supported methods include:
+ `"center"`, `"count"`, `"first"`, `"last"`, `"max"`, `"mean"`, `"median"`,
+ `"mode"`, `"min"`, `"prod"`, `"std"`, `"sum"`, and `"var"`.
+ Defaults to `"center"` for integer arrays, else `"mean"`.
+ For more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms).
+
+Examples:
+
+- [Docs – Sentinel-1 Analysis Mode](https://eopf-sample-service.github.io/xarray-eopf/examples/sentinel_1_analysis/)
+
##### Sentinel-2
@@ -151,16 +208,15 @@ bands from multiple resolutions onto the same grid using [affine transformation
- `interp_methods`: for upsampling / interpolating
spatial data variables. Can be a single interpolation method for all
variables or a dictionary mapping variable names or dtypes to
- interpolation method. Supported methods include:
+ interpolation method (for more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms)).
+ Supported methods include:
- - `0` (nearest neighbor)
- - `1` (linear / bilinear)
+ - `0` (nearest neighbor, default for integer arrays)
+ - `1` (linear / bilinear, default for float arrays)
- `"nearest"`
- `"triangular"`
- `"bilinear"`
- The default is `0` for integer arrays (e.g. Sentinel-2 L2A SCL),
- else `1`. For more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms).
- `agg_methods`: Optional aggregation methods to be used for downsampling
spatial data variables / bands. Can be a single method for all variables or
a dictionary mapping variable names or dtypes to methods. Supported methods include:
@@ -242,25 +298,25 @@ for details.
- Sentinel-3 OLCI Level-2 LFR: 300 meter
- Sentinel-3 SLSTR Level-1 RBT: 500 meter (1000 meter if selected variables come from F- or I-stripe)
- Sentinel-3 SLSTR Level-2 LST: 1000 meter
+
- `interp_methods`: for upsampling / interpolating
spatial data variables. Can be a single interpolation method for all
variables or a dictionary mapping variable names or dtypes to
- interpolation method. Supported methods include:
+ interpolation method (for more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms)).
+ Supported methods include:
- - `0` (nearest neighbor)
- - `1` (linear / bilinear)
+ - `0` (nearest neighbor, default for integer arrays)
+ - `1` (linear / bilinear, default for float arrays)
- `"nearest"`
- `"triangular"`
- `"bilinear"`
- The default is `0` for integer arrays (e.g. Sentinel-2 L2A SCL),
- else `1`. For more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms).
- `agg_methods`: Optional aggregation methods to be used for downsampling
spatial data variables / bands. Can be a single method for all variables or
a dictionary mapping variable names or dtypes to methods. Supported methods include:
`"center"`, `"count"`, `"first"`, `"last"`, `"max"`, `"mean"`, `"median"`,
`"mode"`, `"min"`, `"prod"`, `"std"`, `"sum"`, and `"var"`.
- Defaults to `"center"` for integer arrays (e.g. Sentinel-2 L2A SCL), else `"mean"`.
+ Defaults to `"center"` for integer arrays, else `"mean"`.
For more information view [xcube-resampling Documentation](https://xcube-dev.github.io/xcube-resampling/guide/#spatial-resampling-algorithms).
The spatial resampling of datasets is performed using [xcube-resampling](https://xcube-dev.github.io/xcube-resampling/).
=====================================
environment.yml
=====================================
@@ -15,7 +15,7 @@ dependencies:
- rioxarray
- s3fs
- xarray >=2024.10
- - xcube-resampling >=0.3.2
+ - xcube-resampling >=0.3.4
- zarr >=2.11, <3
# Development Dependencies - Tools
- black
=====================================
examples/sentinel_1_analysis.ipynb
=====================================
The diff for this file was not included because it is too large.
=====================================
integration/test_sen1_analysis.py
=====================================
@@ -49,3 +49,29 @@ class Sentinel2AnalysisTest(TestCase):
assert_dataset_is_chunked(self, ds, verbose=show_chunking)
for var_name in ds.data_vars:
self.assertEqual((541, 1081), ds[var_name].shape, msg=var_name)
+
+ def test_open_datatree_sen1_onc(self):
+ url = (
+ "https://objects.eodc.eu/e05ab01a9d56408d82ac32d69a5aae2a:202507-s01siwocn"
+ "/31/products/cpm_v256/S1A_IW_OCN__2SDV_20250731T213433_20250731T213458_"
+ "060333_077FA7_8163.zarr"
+ )
+ with timeit("open " + url) as result:
+ # noinspection PyTypeChecker
+ ds = xr.open_dataset(
+ url,
+ engine="eopf-zarr",
+ op_mode="analysis",
+ chunks={},
+ )
+ self.assertTrue(result.time_delta < allowed_open_time)
+
+ self.assertIn("wind_direction", ds)
+ self.assertIn("wind_speed", ds)
+ self.assertIn("inversion_quality", ds)
+ self.assertIn("wind_quality", ds)
+ self.assertIn("percentage_bright_points", ds)
+
+ assert_dataset_is_chunked(self, ds, verbose=show_chunking)
+ for var_name in ds.data_vars:
+ self.assertEqual((222, 290), ds[var_name].shape, msg=var_name)
=====================================
integration/test_sen2_analysis.py
=====================================
@@ -16,20 +16,19 @@ show_chunking = False
class Sentinel2AnalysisTest(TestCase):
def test_open_dataset_sen2_l1c(self):
self._test_open_dataset_sen2_l1c(
- "https://objects.eodc.eu:443/e05ab01a9d56408d82ac32d69a5aae2a:202603-"
- "s02msil1c-eu/13/products/cpm_v262/S2A_MSIL1C_20260313T101741_N0512_"
- "R065_T32TLQ_20260313T153853.zarr"
+ "https://objects.eodc.eu/e05ab01a9d56408d82ac32d69a5aae2a:202605-"
+ "s02msil1c-eu/17/products/cpm_v270/S2A_MSIL1C_20260517T125321_"
+ "N0512_R138_T28WET_20260517T194745.zarr"
)
def test_open_dataset_sen2_l2a(self):
self._test_open_dataset_sen2_l2a(
- "https://objects.eodc.eu/e05ab01a9d56408d82ac32d69a5aae2a:202603-"
- "s02msil2a-eu/18/products/cpm_v262/S2A_MSIL2A_20260318T125321_"
- "N0512_R138_T28WDT_20260318T204314.zarr"
+ "https://objects.eodc.eu/e05ab01a9d56408d82ac32d69a5aae2a:202605-"
+ "s02msil2a-eu/17/products/cpm_v270/S2C_MSIL2A_20260517T132721_N0512"
+ "_R024_T26WPU_20260517T181717.zarr"
)
def _test_open_dataset_sen2_l1c(self, url):
- # See https://stac.browser.user.eopf.eodc.eu/collections/sentinel-2-l1c/items/S2B_MSIL1C_20250415T142749_N0511_R139_T25WEV_20250415T180239
with timeit("open " + url) as result:
# noinspection PyTypeChecker
ds = xr.open_dataset(
@@ -46,7 +45,7 @@ class Sentinel2AnalysisTest(TestCase):
assert_dataset_is_chunked(self, ds, verbose=show_chunking)
for var_name in ds.data_vars:
- self.assertEqual((10980, 10980), ds[var_name].shape[-2:], msg=var_name)
+ self.assertEqual((10980, 10980), ds[var_name].shape, msg=var_name)
def _test_open_dataset_sen2_l2a(self, url):
with timeit("open " + url) as result:
@@ -68,4 +67,4 @@ class Sentinel2AnalysisTest(TestCase):
assert_dataset_is_chunked(self, ds, verbose=show_chunking)
for var_name in ds.data_vars:
- self.assertEqual((10980, 10980), ds[var_name].shape[-2:], msg=var_name)
+ self.assertEqual((10980, 10980), ds[var_name].shape, msg=var_name)
=====================================
pyproject.toml
=====================================
@@ -51,7 +51,7 @@ dependencies = [
"rioxarray",
"s3fs",
"xarray>=2024.10",
- "xcube-resampling>=0.3.2",
+ "xcube-resampling>=0.3.4",
"zarr>=2.11,<3",
]
=====================================
tests/amodes/test_sentinel1.py
=====================================
@@ -14,49 +14,30 @@ import pytest
import xarray as xr
from xcube_resampling.gridmapping import GridMapping
-from tests.helpers import make_s1_grd_datatree
+from tests.helpers import make_s1_grd_datatree, make_s1_ocn_datatree
from xarray_eopf.amode import AnalysisModeRegistry
from xarray_eopf.amodes import sentinel1 as sen1
-from xarray_eopf.amodes.sentinel1 import Sen1GRD, register
+from xarray_eopf.amodes.sentinel1 import Sen1GRD, Sen1OCN, register
class Sentinel1AnalysisModeTest(TestCase):
def test_register(self):
registry = AnalysisModeRegistry()
register(registry)
- self.assertEqual(1, len(list(registry.keys())))
- self.assertEqual(Sen1GRD.product_type, registry.keys()[0])
+ self.assertEqual(2, len(list(registry.keys())))
+ self.assertIn(Sen1GRD.product_type, registry.keys())
+ self.assertIn(Sen1OCN.product_type, registry.keys())
# noinspection PyUnresolvedReferences
class Sen1TestMixin:
- def test_get_applicable_params(self: TestCase):
- dem = xr.DataArray(np.ones((2, 2)), dims=("lat", "lon"))
- self.assertEqual({}, self.mode.get_applicable_params())
- self.assertEqual(
- {
- "resolution": 10,
- "bbox": [1, 3, 4, 5],
- "crs": pyproj.CRS.from_string("EPSG:4326"),
- "dem": dem,
- "footprint_scale_factor": (2.0, 3.0),
- "apply_rtc": False,
- },
- self.mode.get_applicable_params(
- resolution=10,
- bbox=[1, 3, 4, 5],
- crs="EPSG:4326",
- dem=dem,
- footprint_scale_factor=(2.0, 3.0),
- apply_rtc=False,
- ),
- )
-
def test_process_metadata(self: TestCase):
self.assertEqual({}, self.mode.process_metadata(xr.DataTree()))
dt = xr.DataTree()
dt.attrs["other_metadata"] = {"test_key": "test_val"}
- self.assertEqual({"test_key": "test_val"}, self.mode.process_metadata(dt))
+ self.assertEqual(
+ {"other_metadata": {"test_key": "test_val"}}, self.mode.process_metadata(dt)
+ )
def test_transform_datatree(self: TestCase):
dt = xr.DataTree()
@@ -70,21 +51,6 @@ class Sen1TestMixin:
out = self.mode.transform_dataset(ds, stac_meta={"k": "v"})
self.assertIs(out, ds)
- def test_get_applicable_params_interp_methods_branch(self: TestCase):
- with pytest.raises(TypeError):
- self.mode.get_applicable_params(interp_methods="cubic")
-
- def test_get_applicable_params_interp_methods_update_line(self: TestCase):
- with patch.object(sen1, "assert_arg_is_instance"):
- params = self.mode.get_applicable_params(interp_methods="nearest")
- self.assertEqual("nearest", params["interp_methods"])
-
- def test_get_applicable_params_footprint_scale_factor_invalid_values(
- self: TestCase,
- ):
- with pytest.raises(TypeError, match="footprint_scale_factor"):
- self.mode.get_applicable_params(footprint_scale_factor=(1.0, "x"))
-
class Sen1GRDTest(Sen1TestMixin, TestCase):
mode = Sen1GRD()
@@ -109,6 +75,34 @@ class Sen1GRDTest(Sen1TestMixin, TestCase):
self.assertEqual(1.0, params["d_az"])
self.assertEqual(40.0, params["spacing_az"])
+ def test_get_applicable_params(self: TestCase):
+ dem = xr.DataArray(np.ones((2, 2)), dims=("lat", "lon"))
+ self.assertEqual({}, self.mode.get_applicable_params())
+ self.assertEqual(
+ {
+ "resolution": 10,
+ "bbox": [1, 3, 4, 5],
+ "crs": pyproj.CRS.from_string("EPSG:4326"),
+ "dem": dem,
+ "interp_methods": "nearest",
+ "footprint_scale_factor": (2.0, 3.0),
+ "apply_rtc": False,
+ },
+ self.mode.get_applicable_params(
+ resolution=10,
+ bbox=[1, 3, 4, 5],
+ crs="EPSG:4326",
+ dem=dem,
+ interp_methods="nearest",
+ footprint_scale_factor=(2.0, 3.0),
+ apply_rtc=False,
+ ),
+ )
+ with pytest.raises(TypeError, match="interp_methods"):
+ self.mode.get_applicable_params(interp_methods="cubic")
+ with pytest.raises(TypeError, match="footprint_scale_factor"):
+ self.mode.get_applicable_params(footprint_scale_factor=(1.0, "x"))
+
def test_convert_datatree(self):
expected = xr.Dataset(
{"vv": xr.DataArray(np.ones((2, 2)), dims=("lat", "lon"))}
@@ -143,6 +137,95 @@ class Sen1GRDTest(Sen1TestMixin, TestCase):
self.mode.convert_datatree(self.dt, includes="bibo", dem=self.dem)
+class Sen1OCNTest(Sen1TestMixin, TestCase):
+ mode = Sen1OCN()
+ dt = make_s1_ocn_datatree()
+
+ def test_is_valid_source_ok(self):
+ self.assertTrue(self.mode.is_valid_source("data/S1A_IW_OCN_20240201.zarr"))
+ self.assertTrue(self.mode.is_valid_source("S1A_IW_OCN_TEST"))
+
+ def test_is_not_valid_source(self):
+ self.assertFalse(self.mode.is_valid_source("data/S1A_IW_SLC_20240201.zarr"))
+ self.assertFalse(self.mode.is_valid_source(dict()))
+
+ def test_get_applicable_params(self: TestCase):
+ self.assertEqual({}, self.mode.get_applicable_params())
+ self.assertEqual(
+ {
+ "resolution": 1,
+ "bbox": [1, 3, 4, 5],
+ "crs": pyproj.CRS.from_string("EPSG:4326"),
+ "interp_methods": "nearest",
+ "agg_methods": "nearest",
+ },
+ self.mode.get_applicable_params(
+ resolution=1,
+ bbox=[1, 3, 4, 5],
+ crs="EPSG:4326",
+ interp_methods="nearest",
+ agg_methods="nearest",
+ ),
+ )
+ with pytest.raises(TypeError):
+ self.mode.get_applicable_params(interp_methods="cubic")
+
+ def test_convert_datatree(self):
+ # with bbox and resolution
+ out = self.mode.convert_datatree(
+ self.dt,
+ includes=["wind_direction", "wind_speed"],
+ resolution=1,
+ bbox=[-1, 1, 2, 4],
+ )
+ self.assertCountEqual(["wind_direction", "wind_speed"], out.keys())
+ self.assertEqual({"lat": 3, "lon": 3}, out.sizes)
+ self.assertListEqual([-0.5, 0.5, 1.5], out.lon.values.tolist())
+ self.assertListEqual([3.5, 2.5, 1.5], out.lat.values.tolist())
+ self.assertTrue(np.all(out.wind_direction.values == 1))
+ self.assertTrue(np.all(out.wind_speed.values == 1))
+
+ # without bbox and resolution
+ out = self.mode.convert_datatree(self.dt)
+ self.assertCountEqual(
+ [
+ "wind_direction",
+ "wind_speed",
+ "inversion_quality",
+ "wind_quality",
+ "percentage_bright_points",
+ ],
+ out.keys(),
+ )
+ self.assertEqual({"lat": 7, "lon": 7}, out.sizes)
+ self.assertListEqual(
+ [-3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0], out.lon.values.tolist()
+ )
+ self.assertListEqual(
+ [6.0, 5.0, 4.0, 3.0, 2.0, 1.0, 0.0], out.lat.values.tolist()
+ )
+
+ # projected crs
+ out = self.mode.convert_datatree(
+ self.dt, crs=pyproj.CRS.from_string("EPSG:32631")
+ )
+ self.assertCountEqual(
+ [
+ "wind_direction",
+ "wind_speed",
+ "inversion_quality",
+ "wind_quality",
+ "percentage_bright_points",
+ ],
+ out.keys(),
+ )
+ self.assertEqual({"y": 8, "x": 8}, out.sizes)
+
+ def test_convert_datatree_fail(self):
+ with pytest.raises(ValueError, match="No valid variable names"):
+ self.mode.convert_datatree(self.dt, includes="invalid_var")
+
+
class Sentinel1FunctionsTest(TestCase):
def test_gridparams_iter(self):
params = sen1.GridParams(
=====================================
tests/amodes/test_sentinel2.py
=====================================
@@ -50,6 +50,11 @@ class MsiTestMixin:
def test_process_metadata(self: TestCase):
self.assertEqual({}, self.mode.process_metadata(xr.DataTree()))
+ dt = xr.DataTree()
+ dt.attrs["other_metadata"] = {"test_key": "test_val"}
+ self.assertEqual(
+ {"other_metadata": {"test_key": "test_val"}}, self.mode.process_metadata(dt)
+ )
def test_assign_grid_mapping(self: TestCase):
def make_band():
=====================================
tests/amodes/test_sentinel3.py
=====================================
@@ -54,7 +54,9 @@ class Sen3TestMixin:
self.assertEqual({}, self.mode.process_metadata(xr.DataTree()))
dt = xr.DataTree()
dt.attrs["other_metadata"] = {"test_key": "test_val"}
- self.assertEqual({"test_key": "test_val"}, self.mode.process_metadata(dt))
+ self.assertEqual(
+ {"other_metadata": {"test_key": "test_val"}}, self.mode.process_metadata(dt)
+ )
@staticmethod
def create_simple_dataset() -> xr.Dataset:
@@ -211,7 +213,7 @@ class OlciEfrTest(Sen3TestMixin, TestCase):
includes=["oa01_radiance"],
resolution=0.1,
)
- self.assertEqual({"test_key": "test_val"}, ds.attrs)
+ self.assertCountEqual(["stac_discovery", "other_metadata"], ds.attrs.keys())
class SlstrRbtTest(Sen3TestMixin, TestCase):
=====================================
tests/helpers/__init__.py
=====================================
@@ -2,16 +2,17 @@
# Permissions are hereby granted under the terms of the Apache 2.0 License:
# https://opensource.org/license/apache-2-0.
-from .sentinel1 import make_s1_grd_datatree
+from .sentinel1 import make_s1_grd_datatree, make_s1_ocn_datatree
from .sentinel2 import make_s2_msi, make_s2_msi_l1c, make_s2_msi_l2a
from .sentinel3 import make_s3_olci_efr, make_s3_slstr_lst, make_s3_slstr_rbt
__all__ = [
+ "make_s1_grd_datatree",
+ "make_s1_ocn_datatree",
"make_s2_msi",
"make_s2_msi_l1c",
"make_s2_msi_l2a",
"make_s3_olci_efr",
"make_s3_slstr_rbt",
"make_s3_slstr_lst",
- "make_s1_grd_datatree",
]
=====================================
tests/helpers/sentinel1.py
=====================================
@@ -84,3 +84,89 @@ def make_s1_grd_datatree() -> xr.DataTree:
}
}
return dt
+
+
+def _make_owi_measurements(height: int, width: int):
+ shape = (height, width)
+
+ return xr.Dataset(
+ {
+ "wind_speed": xr.DataArray(
+ np.ones(shape, dtype="float32"),
+ dims=("height", "width"),
+ ),
+ "wind_direction": xr.DataArray(
+ np.ones(shape, dtype="float32"),
+ dims=("height", "width"),
+ ),
+ },
+ coords=_make_lat_lon_coords(height, width),
+ )
+
+
+def _make_owi_quality(height: int, width: int):
+ shape = (height, width)
+
+ return xr.Dataset(
+ {
+ "calibration_constant": xr.DataArray(
+ np.zeros(shape, dtype="float32"),
+ dims=("height", "width"),
+ ),
+ "inversion_quality": xr.DataArray(
+ np.zeros(shape, dtype="float64"),
+ dims=("height", "width"),
+ attrs={"_eopf_attrs": {"valid_min": 0, "valid_max": 3}},
+ ),
+ "wind_quality": xr.DataArray(
+ np.zeros(shape, dtype="float64"),
+ dims=("height", "width"),
+ attrs={"_eopf_attrs": {"valid_min": 0, "valid_max": 3}},
+ ),
+ "percentage_bright_points": xr.DataArray(
+ np.ones(shape, dtype="float32"),
+ dims=("height", "width"),
+ attrs={"_eopf_attrs": {"valid_min": 0, "valid_max": 100}},
+ ),
+ },
+ coords=_make_lat_lon_coords(height, width),
+ )
+
+
+def _make_lat_lon_coords(height: int, width: int):
+ y = np.arange(height - 1, -1, -1)
+ x = np.arange(width)
+
+ latitude = y[:, None] + x[None, :]
+ longitude = -y[:, None] + x[None, :]
+
+ return {
+ "latitude": (
+ ("height", "width"),
+ latitude,
+ ),
+ "longitude": (
+ ("height", "width"),
+ longitude,
+ ),
+ }
+
+
+def make_s1_ocn_datatree() -> xr.DataTree:
+ height = 4
+ width = 4
+
+ product = "S1A_IW_OCN"
+
+ return xr.DataTree.from_dict(
+ {
+ f"/owi/{product}/measurements": _make_owi_measurements(
+ height,
+ width,
+ ),
+ f"/owi/{product}/quality": _make_owi_quality(
+ height,
+ width,
+ ),
+ }
+ )
=====================================
xarray_eopf/amodes/sentinel1.py
=====================================
@@ -20,8 +20,10 @@ import pystac_client
import rioxarray
import xarray as xr
from xcube_resampling import resample_in_space
+from xcube_resampling.constants import SpatialAggMethods, SpatialInterpMethods
from xcube_resampling.gridmapping import GridMapping
-from xcube_resampling.utils import reproject_bbox
+from xcube_resampling.rectify import rectify_dataset
+from xcube_resampling.utils import reproject_bbox, transform_resolution
from xarray_eopf.amode import AnalysisMode, AnalysisModeRegistry
from xarray_eopf.source import get_source_path
@@ -33,6 +35,7 @@ _ONE_SECOND = np.timedelta64(_S_TO_NS, "ns")
_CRS_ECEF = pyproj.CRS.from_string("EPSG:4978")
_CRS_WGS84 = pyproj.CRS.from_string("EPSG:4326")
_DEM_CHUNKSIZE = dict(lat=1800, lon=1800)
+_CHUNKSIZE = (2048, 2048)
@dataclass(frozen=True)
@@ -102,6 +105,26 @@ class Sen1(AnalysisMode, ABC):
pattern = re.compile(rf"S1[A-D]_[A-Z]{{2}}_{self.product_type}_[^/]+$")
return bool(pattern.search(root_path)) if root_path else False
+ def transform_datatree(self, datatree: xr.DataTree, **params) -> xr.DataTree:
+ warnings.warn(
+ "Analysis mode not implemented for given source, "
+ "returning data tree as-is."
+ )
+ return datatree
+
+ def transform_dataset(
+ self, dataset: xr.Dataset, stac_meta: dict, **params
+ ) -> xr.Dataset:
+ # ToDo: what should be added when opening a subgroup in analysis mode?
+ return dataset
+
+ def process_metadata(self, datatree: xr.DataTree) -> dict:
+ return datatree.attrs
+
+
+class Sen1GRD(Sen1):
+ product_type = "GRDH"
+
def get_applicable_params(self, **kwargs) -> dict[str, Any]:
params = {}
@@ -157,26 +180,6 @@ class Sen1(AnalysisMode, ABC):
return params
- def transform_datatree(self, datatree: xr.DataTree, **params) -> xr.DataTree:
- warnings.warn(
- "Analysis mode not implemented for given source, return data tree as-is."
- )
- return datatree
-
- def transform_dataset(
- self, dataset: xr.Dataset, stac_meta: dict, **params
- ) -> xr.Dataset:
- # ToDo: what should be added when opening a subgroup in analysis mode?
- return dataset
-
- def process_metadata(self, datatree: xr.DataTree) -> dict:
- other_metadata = datatree.attrs.get("other_metadata", {})
- return other_metadata
-
-
-class Sen1GRD(Sen1):
- product_type = "GRDH"
-
def convert_datatree(
self,
datatree: xr.DataTree,
@@ -290,9 +293,139 @@ class Sen1GRD(Sen1):
)
+class Sen1OCN(Sen1):
+ product_type = "OCN"
+
+ def get_applicable_params(self, **kwargs) -> dict[str, Any]:
+ params = {}
+
+ resolution = kwargs.get("resolution")
+ if resolution is not None:
+ assert_arg_is_instance(resolution, "resolution", (float, int))
+ params.update(resolution=resolution)
+
+ bbox = kwargs.get("bbox")
+ if bbox is not None:
+ assert_arg_is_instance(bbox, "bbox", (Sequence,))
+ assert_arg_has_length(bbox, "bbox", 4)
+ params.update(bbox=bbox)
+
+ crs = kwargs.get("crs")
+ if crs is not None:
+ if isinstance(crs, str):
+ crs = pyproj.CRS.from_string(crs)
+ assert_arg_is_instance(crs, "crs", (pyproj.CRS,))
+ params.update(crs=crs)
+
+ interp_methods = kwargs.get("interp_methods")
+ if interp_methods is not None:
+ assert_arg_is_instance(
+ interp_methods, "interp_methods", Literal["nearest", "bilinear"]
+ )
+ params.update(interp_methods=interp_methods)
+
+ agg_methods = kwargs.get("agg_methods")
+ if agg_methods is not None:
+ assert_arg_is_instance(agg_methods, "agg_methods", (str, dict))
+ params.update(agg_methods=agg_methods)
+
+ return params
+
+ def convert_datatree(
+ self,
+ datatree: xr.DataTree,
+ includes: str | Iterable[str] | None = None,
+ excludes: str | Iterable[str] | None = None,
+ resolution: float = None,
+ bbox: Sequence[float | int] | None = None,
+ crs: pyproj.CRS | None = None,
+ interp_methods: SpatialInterpMethods | None = None,
+ agg_methods: SpatialAggMethods | None = None,
+ ) -> xr.Dataset:
+ # load measurement data
+ assert (
+ len(datatree.owi.children) == 1
+ ), "Expected one child in OCN OWI sub data tree"
+ sub_dt = next(iter(datatree.owi.children.values()))
+ dataset = sub_dt.measurements.to_dataset()
+ dataset.update(sub_dt.quality.to_dataset().drop_vars("calibration_constant"))
+
+ # correct attributes and encoding
+ def _apply_valid_range(da, *, dtype=None, fill_value=None):
+ if dtype is not None:
+ da = da.astype(dtype)
+
+ if fill_value is not None:
+ da.encoding["_FillValue"] = fill_value
+
+ eopf_attrs = da.attrs["_eopf_attrs"]
+ da.attrs.update(
+ valid_min=eopf_attrs["valid_min"],
+ valid_max=eopf_attrs["valid_max"],
+ )
+
+ return da
+
+ dataset["inversion_quality"] = _apply_valid_range(
+ dataset.inversion_quality,
+ dtype="uint8",
+ fill_value=255,
+ )
+ dataset["wind_quality"] = _apply_valid_range(
+ dataset.wind_quality,
+ dtype="uint8",
+ fill_value=255,
+ )
+ dataset["percentage_bright_points"] = _apply_valid_range(
+ dataset.percentage_bright_points,
+ )
+
+ # filter dataset by variable names
+ name_filter = NameFilter(includes=includes, excludes=excludes)
+ variable_names = [k for k in dataset.data_vars if name_filter.accept(str(k))]
+ if not variable_names:
+ raise ValueError("No valid variable names found in dataset")
+ dataset = dataset[variable_names]
+
+ # reproject dataset to regular grid
+ source_gm = GridMapping.from_dataset(dataset)
+ if bbox is None:
+ if crs:
+ bbox = reproject_bbox(source_gm.xy_bbox, source_gm.crs, crs)
+ else:
+ bbox = source_gm.xy_bbox
+ if resolution is None:
+ if crs and not crs.is_geographic:
+ center_lat = (
+ (source_gm.xy_bbox[0] + source_gm.xy_bbox[2]) / 2,
+ (source_gm.xy_bbox[1] + source_gm.xy_bbox[3]) / 2,
+ )
+ resolution = transform_resolution(
+ center_lat, source_gm.xy_res, source_gm.crs, crs
+ )
+ else:
+ resolution = source_gm.xy_res
+ if crs is None:
+ crs = source_gm.crs
+ target_gm = GridMapping.regular_from_bbox(
+ bbox=bbox, xy_res=resolution, crs=crs, tile_size=_CHUNKSIZE
+ )
+
+ rectified_dataset = rectify_dataset(
+ dataset,
+ source_gm=source_gm,
+ target_gm=target_gm,
+ interp_methods=interp_methods,
+ agg_methods=agg_methods,
+ )
+ rectified_dataset.attrs = self.process_metadata(datatree)
+ return rectified_dataset
+
+
def register(registry: AnalysisModeRegistry):
"""Register Sentinel-1 analysis modes."""
registry.register(Sen1GRD)
+ registry.register(Sen1OCN)
def get_dem(
@@ -705,7 +838,7 @@ def simulate_acquisition(
return out
-def compute_dem_area(dem_ecef: xr.DataArray, gm_dem: xr.DataArray) -> xr.DataArray:
+def compute_dem_area(dem_ecef: xr.DataArray, gm_dem: GridMapping) -> xr.DataArray:
"""Compute per-pixel surface area on the DEM in ECEF coordinates.
Args:
@@ -738,8 +871,16 @@ def compute_dem_area(dem_ecef: xr.DataArray, gm_dem: xr.DataArray) -> xr.DataArr
# interpolate DEM to pixel corners
chunksizes = {key: val[0] for key, val in dem_ecef.chunksizes.items()}
- xyz_c = dem_ecef.interp({x_dim: x_corner}).chunk({x_dim: chunksizes[x_dim]})
- xyz_c = xyz_c.interp({y_dim: y_corner}).chunk(chunksizes)
+ xyz_c = dem_ecef.interp(
+ {x_dim: x_corner},
+ method="linear",
+ kwargs={"fill_value": "extrapolate"},
+ ).chunk({x_dim: chunksizes[x_dim]})
+ xyz_c = xyz_c.interp(
+ {y_dim: y_corner},
+ method="linear",
+ kwargs={"fill_value": "extrapolate"},
+ ).chunk(chunksizes)
# compute edge vectors
dx = xyz_c.diff(x_dim)
@@ -769,7 +910,7 @@ def compute_dem_area(dem_ecef: xr.DataArray, gm_dem: xr.DataArray) -> xr.DataArr
def compute_gamma_area(
- dem_ecef: xr.DataArray, gm_dem: xr.DataArray, direction: xr.DataArray
+ dem_ecef: xr.DataArray, gm_dem: GridMapping, direction: xr.DataArray
) -> xr.DataArray:
"""Compute gamma area by projecting DEM areas onto look direction.
=====================================
xarray_eopf/amodes/sentinel2.py
=====================================
@@ -143,7 +143,8 @@ class Msi(AnalysisMode, ABC):
def transform_datatree(self, datatree: xr.DataTree, **params) -> xr.DataTree:
warnings.warn(
- "Analysis mode not implemented for given source, return data tree as-is."
+ "Analysis mode not implemented for given source, "
+ "returning data tree as-is."
)
return datatree
@@ -319,8 +320,7 @@ class Msi(AnalysisMode, ABC):
return dataset
def process_metadata(self, datatree: xr.DataTree) -> dict:
- other_metadata = datatree.attrs.get("other_metadata", {})
- return other_metadata
+ return datatree.attrs
class MsiL1c(Msi):
=====================================
xarray_eopf/amodes/sentinel3.py
=====================================
@@ -86,8 +86,8 @@ class Sen3(AnalysisMode, ABC):
def transform_datatree(self, datatree: xr.DataTree, **params) -> xr.DataTree:
warnings.warn(
- "Analysis mode not implemented for given source, return data tree as-is.",
- UserWarning,
+ "Analysis mode not implemented for given source, "
+ "returning data tree as-is."
)
return datatree
@@ -191,10 +191,8 @@ class Sen3(AnalysisMode, ABC):
return dataset
- # noinspection PyMethodMayBeStatic
def process_metadata(self, datatree: xr.DataTree) -> dict:
- other_metadata = datatree.attrs.get("other_metadata", {})
- return other_metadata
+ return datatree.attrs
def _apply_orthorectification(
self,
=====================================
xarray_eopf/backend.py
=====================================
@@ -163,14 +163,15 @@ class EopfBackend(BackendEntrypoint):
using [`pyproj.crs.CRS.from_string`](https://pyproj4.github.io/pyproj/dev/api/crs/crs.html#pyproj.crs.CRS.from_string).
interp_methods: Optional interpolation method to be used if
`op_mode="analysis"`,
- - for Sentinel-1:
+
+ - for Sentinel-1 GRD:
method used during geometric and radiometric terrain correction
(GTC and RTC).
- `"nearest"`
- `"bilinear"`
- - for Sentinel-2 and Sentinel-3:
+ - for Sentinel-1 OCN, Sentinel-2, and Sentinel-3:
for upsampling / interpolating spatial data variables. Can be a
single interpolation method for all variables or a dictionary
mapping variable names or dtypes to interpolation method.
=====================================
xarray_eopf/utils.py
=====================================
@@ -11,8 +11,8 @@ from typing import (
Type,
TypeAlias,
TypeVar,
- get_origin,
get_args,
+ get_origin,
)
import numpy as np
=====================================
xarray_eopf/version.py
=====================================
@@ -2,4 +2,4 @@
# Permissions are hereby granted under the terms of the Apache 2.0 License:
# https://opensource.org/license/apache-2-0.
-version = "0.2.8"
+version = "0.2.9"
View it on GitLab: https://salsa.debian.org/debian-gis-team/xarray-eopf/-/commit/8fecd39957fbbe2e369c0781b073a92e8bdb2f3e
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/xarray-eopf/-/commit/8fecd39957fbbe2e369c0781b073a92e8bdb2f3e
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20260622/12e090e8/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list