[Git][debian-gis-team/eodag][upstream] New upstream version 4.7.1+ds
Antonio Valentino (@antonio.valentino)
gitlab at salsa.debian.org
Thu Aug 20 18:08:12 BST 2026
Antonio Valentino pushed to branch upstream at Debian GIS Project / eodag
Commits:
048acf2a by Antonio Valentino at 2026-08-20T14:29:36+00:00
New upstream version 4.7.1+ds
- - - - -
21 changed files:
- .github/workflows/benchmark.yml
- .github/workflows/changelog.yml
- .github/workflows/deploy.yml
- .github/workflows/fetch.yml
- .github/workflows/publish.yml
- .github/workflows/test.yml
- CHANGES.rst
- docs/providers.rst
- eodag/api/product/_assets.py
- eodag/plugins/search/eumetsat_ds.py
- eodag/plugins/search/qssearch.py
- eodag/resources/ext_collections.json
- eodag/resources/ext_product_types.json
- eodag/utils/logging.py
- eodag/utils/s3.py
- pyproject.toml
- tests/context.py
- tests/test_benchmark.py
- + tests/units/test_assets.py
- tests/units/test_utils.py
- tox.ini
Changes:
=====================================
.github/workflows/benchmark.yml
=====================================
@@ -19,11 +19,13 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Run benchmark on develop and current ref, then compare
shell: bash
run: |
=====================================
.github/workflows/changelog.yml
=====================================
@@ -16,11 +16,10 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
- name: Install python-semantic-release and packaging in venv
run: |
uv venv
=====================================
.github/workflows/deploy.yml
=====================================
@@ -22,11 +22,10 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
- name: Install python-semantic-release in venv
run: |
uv venv
=====================================
.github/workflows/fetch.yml
=====================================
@@ -22,11 +22,10 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
- name: Install eodag in venv
run: |
uv venv
@@ -168,11 +167,10 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
- name: Install eodag v3 in venv
run: |
uv venv
=====================================
.github/workflows/publish.yml
=====================================
@@ -19,11 +19,13 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Check that the current version isn't already on PyPi
run: |
if [ "$(./utils/get_pypi_latest_version.sh)" != "$(git describe --tags)" ]
=====================================
.github/workflows/test.yml
=====================================
@@ -20,11 +20,13 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Run linters with tox
run: uvx --python 3.9 --with tox-uv tox -e linters
@@ -42,11 +44,13 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Test with tox
run: uvx --python ${{ matrix.python-version }} --with tox-uv --with tox-gh-actions tox
- name: Upload Unit Test Results
@@ -193,11 +197,13 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Install pandoc
run: |
curl -s https://api.github.com/repos/jgm/pandoc/releases/latest | grep -o "https.*amd64.deb" | wget -O pandoc.deb -qi -
@@ -215,10 +221,12 @@ jobs:
# Get history and tags for SCM versioning to work
fetch-depth: 0
- name: Install the latest version of uv with cache enabled
- uses: astral-sh/setup-uv at v3
+ uses: astral-sh/setup-uv at v9.0.0
with:
version: "latest"
enable-cache: true
- cache-dependency-glob: ""
+ cache-dependency-glob: |
+ pyproject.toml
+ tox.ini
- name: Testing with tox and sphinx (to have rst2html.py utility available)
run: uvx --with tox-uv --with sphinx tox -e pypi
=====================================
CHANGES.rst
=====================================
@@ -3,6 +3,51 @@ Release history
===============
+v4.7.1 (2026-08-20)
+===================
+
+Performance Improvements
+------------------------
+
+* **core**: Optimize AssetsDict batch updates (`#2314`_, `4783588`_)
+
+.. _#2314: https://github.com/CS-SI/eodag/pull/2314
+.. _4783588: https://github.com/CS-SI/eodag/commit/4783588f865255124671af8e2fec45ffbfaac3d5
+
+
+v4.7.0 (2026-08-19)
+===================
+
+Features
+--------
+
+* **core**: Sort assets and remove private fields (`#2309`_, `3e97a4d`_)
+
+Bug Fixes
+---------
+
+* **logging**: Logs handling through tqdm (`#1640`_, `48c3891`_)
+
+Continuous Integration
+----------------------
+
+* Tox and uv conf cleanup (`#2300`_, `8f6e915`_)
+
+Documentation
+-------------
+
+* Update provider registration introduction (`#2302`_, `cb4e4d6`_)
+
+.. _#1640: https://github.com/CS-SI/eodag/pull/1640
+.. _#2300: https://github.com/CS-SI/eodag/pull/2300
+.. _#2302: https://github.com/CS-SI/eodag/pull/2302
+.. _#2309: https://github.com/CS-SI/eodag/pull/2309
+.. _3e97a4d: https://github.com/CS-SI/eodag/commit/3e97a4d4fb7169392e6892707377d47b7243286d
+.. _48c3891: https://github.com/CS-SI/eodag/commit/48c38912bb5d24cae599769951db7e4550b223b1
+.. _8f6e915: https://github.com/CS-SI/eodag/commit/8f6e915d831f73effabe139ca828f89f718c3008
+.. _cb4e4d6: https://github.com/CS-SI/eodag/commit/cb4e4d64b23173289eefafddaa740bbfaff4a7ee
+
+
v4.6.0 (2026-08-11)
===================
=====================================
docs/providers.rst
=====================================
@@ -10,7 +10,8 @@ This section introduces the featured providers that are already integrated, so y
easily start searching, accessing, and downloading products.
Some providers are completely open, while others require an account to access their data.
-If credentials are needed, check the :doc:`registration guide <providers>` for details.
+Check provider ``Registration info`` in this page to see if credentials are expected and how to get
+them.
.. admonition:: EODAG is not limited to the providers listed here
:class: note
=====================================
eodag/api/product/_assets.py
=====================================
@@ -21,6 +21,8 @@ import re
from collections import UserDict
from typing import TYPE_CHECKING, Any, Optional
+from typing_extensions import override
+
from eodag.utils.exceptions import NotAvailableError
from eodag.utils.repr import dict_to_html_table
@@ -60,12 +62,50 @@ class AssetsDict(UserDict):
self.product = product
super(AssetsDict, self).__init__(*args, **kwargs)
- def update(self, data: dict[str, Any]) -> None: # type: ignore
- """Update assets"""
- super().update(data)
-
def __setitem__(self, key: str, value: dict[str, Any]) -> None:
- super().__setitem__(key, Asset(self.product, key, value))
+ super().__setitem__(key, self._make_asset(key, value))
+ self.sort()
+
+ def _make_asset(self, key: str, value: dict[str, Any]) -> Asset:
+ """Build the :class:`Asset` to store for the given key/value pair.
+
+ Subclasses (e.g. eodag_cube) can override this to use an extended
+ :class:`Asset` subclass without duplicating the sorting and other
+ logic implemented in :meth:`__setitem__`/:meth:`update`.
+
+ :param key: Asset key
+ :param value: Already-validated asset dictionary
+ :returns: The :class:`Asset` instance to store
+ """
+ return Asset(self.product, key, value)
+
+ @override
+ def update(self, *args: Any, **kwargs: Any) -> None:
+ """Used to self update with external value"""
+ incoming = dict(*args, **kwargs)
+
+ if not incoming:
+ return
+
+ if type(self).__setitem__ is not AssetsDict.__setitem__:
+ # Preserve custom __setitem__ behavior implemented by subclasses.
+ super().update(incoming)
+ else:
+ # Avoid sorting after every insertion in the base implementation.
+ for key, value in incoming.items():
+ self.data[key] = self._make_asset(key, value)
+ self.sort()
+
+ def sort(self):
+ """Used to self sort"""
+ sorted_assets = {}
+ for key in sorted(self.data):
+ asset = self.data[key]
+ # delete private asset fields (fields starting with '_')
+ for private_field in [k for k in asset if k[0] == "_"]:
+ del asset[private_field]
+ sorted_assets[key] = asset
+ self.data = sorted_assets
def as_dict(self) -> dict[str, Any]:
"""Builds a representation of AssetsDict to enable its serialization
@@ -171,7 +211,8 @@ class Asset(UserDict):
def __init__(self, product: EOProduct, key: str, *args: Any, **kwargs: Any) -> None:
self.product = product
self.key = key
- super(Asset, self).__init__(*args, **kwargs)
+ # Faster than UserDict.__init__, which goes through MutableMapping.update()
+ self.data = dict(*args, **kwargs)
def as_dict(self) -> dict[str, Any]:
"""Builds a representation of Asset to enable its serialization
=====================================
eodag/plugins/search/eumetsat_ds.py
=====================================
@@ -47,14 +47,13 @@ class EumetsatDsSearch(QueryStringSearch):
for index in range(0, len(processed_results)):
result = processed_results[index]
if hasattr(result, "assets"):
- for name, asset in result.assets.items():
+ for asset in result.assets.values():
if "type" in asset:
asset["eumesat_ds:type"] = asset["type"]
del asset["type"]
if "mediaType" in asset:
asset["type"] = asset["mediaType"]
del asset["mediaType"]
- result.assets[name] = asset
processed_results[index] = result
return processed_results
=====================================
eodag/plugins/search/qssearch.py
=====================================
@@ -1307,9 +1307,9 @@ class QueryStringSearch(Search):
product = EOProduct(self.provider, properties, **product_kwargs)
additional_assets = self.get_assets_from_mapping(result)
- product.assets.update(additional_assets)
# move assets from properties to product's attr, normalize keys & roles
+ normalized_assets: dict[str, Any] = {}
for key, asset in product.properties.pop("assets", {}).items():
url = asset.get("href", "")
norm_key, roles = product.driver.guess_asset_key_and_roles(
@@ -1319,13 +1319,13 @@ class QueryStringSearch(Search):
if norm_key is not None:
asset["title"] = norm_key
asset["roles"] = roles
- product.assets[norm_key] = asset
+ normalized_assets[norm_key] = asset
else:
asset["title"] = asset.get("title", key)
- product.assets[key] = asset
+ normalized_assets[key] = asset
- # sort assets
- product.assets.data = dict(sorted(product.assets.data.items()))
+ # batch-update once to avoid assets sort and clean up on every update
+ product.assets.update({**additional_assets, **normalized_assets})
product._normalize_bands()
products.append(product)
return products
=====================================
eodag/resources/ext_collections.json
=====================================
The diff for this file was not included because it is too large.
=====================================
eodag/resources/ext_product_types.json
=====================================
The diff for this file was not included because it is too large.
=====================================
eodag/utils/logging.py
=====================================
@@ -18,11 +18,31 @@
from __future__ import annotations
import logging.config
-from typing import Optional
+from typing import Any, Optional
disable_tqdm = False
+class TqdmLoggingHandler(logging.StreamHandler):
+ """Logging handler writing through :func:`tqdm.write` to avoid breaking progress bars"""
+
+ def emit(self, record: logging.LogRecord) -> None:
+ """Emit a record using ``tqdm.write()`` if progress bars are enabled
+
+ :param record: The log record to emit
+ """
+ if disable_tqdm:
+ super().emit(record)
+ return
+ try:
+ from tqdm.auto import tqdm
+
+ tqdm.write(self.format(record), file=self.stream, end=self.terminator)
+ self.flush()
+ except Exception:
+ self.handleError(record)
+
+
def setup_logging(verbose: int, no_progress_bar: bool = False) -> None:
"""Define logging level
@@ -45,10 +65,10 @@ def setup_logging(verbose: int, no_progress_bar: bool = False) -> None:
level = "DEBUG" if verbose == 3 else "INFO"
- handlers = {
+ handlers: dict[str, dict[str, Any]] = {
"console": {
"level": level,
- "class": "logging.StreamHandler",
+ "class": "eodag.utils.logging.TqdmLoggingHandler",
"formatter": "standard",
},
"null": {"level": level, "class": "logging.NullHandler"},
=====================================
eodag/utils/s3.py
=====================================
@@ -605,12 +605,12 @@ def update_assets_from_s3(
assets_data[key] = asset_data
- for key in assets_data:
- if key not in product.assets:
- product.assets[key] = assets_data[key]
-
- # sort assets
- product.assets.data = dict(sorted(product.assets.data.items()))
+ new_assets = {
+ key: asset_data
+ for key, asset_data in assets_data.items()
+ if key not in product.assets
+ }
+ product.assets.update(new_assets)
# update driver
product.driver = product.get_driver()
=====================================
pyproject.toml
=====================================
@@ -211,7 +211,7 @@ include = ["eodag*"]
"*" = ["LICENSE", "NOTICE", "py.typed"]
[tool.setuptools_scm]
-fallback_version = "4.6.1.dev0"
+fallback_version = "4.7.2.dev0"
[tool.isort]
multi_line_output = 3
=====================================
tests/context.py
=====================================
@@ -28,6 +28,7 @@ from unittest import mock
from eodag import EODataAccessGateway, api, config, setup_logging
from eodag.api.core import DEFAULT_LIMIT, DEFAULT_MAX_LIMIT
from eodag.api.product import EOProduct
+from eodag.api.product._assets import Asset, AssetsDict
from eodag.api.product.drivers import DRIVERS
from eodag.api.product.drivers.generic import GenericDriver
from eodag.api.product.drivers.sentinel1 import Sentinel1Driver
=====================================
tests/test_benchmark.py
=====================================
@@ -21,7 +21,8 @@ import subprocess
import sys
from tempfile import TemporaryDirectory
-from tests import test_cli
+from tests import EODagTestBase, test_cli
+from tests.context import EOProduct
from tests.integration import test_core_search_results
from tests.units import test_stac_reader
from tests.utils import write_eodag_conf_with_fake_credentials
@@ -144,3 +145,36 @@ def test_benchmark_stac_reader_fetch_recursive(benchmark):
"test_stac_reader_fetch_root_recursive",
)
)
+
+
+def test_benchmark_eoproduct_instantiation(benchmark):
+ test_case = EODagTestBase()
+ test_case.setUp()
+ try:
+ benchmark(
+ lambda: EOProduct(
+ test_case.provider,
+ test_case.eoproduct_props,
+ collection=test_case.collection,
+ )
+ )
+ finally:
+ test_case.tearDown()
+
+
+def test_benchmark_eoproduct_assets_population(benchmark):
+ test_case = EODagTestBase()
+ test_case.setUp()
+ try:
+ product = test_case._dummy_product()
+ assets = {
+ f"band{i}": {"href": f"http://example.com/band{i}.tif"} for i in range(20)
+ }
+
+ def _populate_assets():
+ product.assets.update(assets)
+ return len(product.assets)
+
+ benchmark(_populate_assets)
+ finally:
+ test_case.tearDown()
=====================================
tests/units/test_assets.py
=====================================
@@ -0,0 +1,90 @@
+# -*- coding: utf-8 -*-
+# Copyright 2026, CS GROUP - France, https://www.csgroup.eu/
+#
+# This file is part of EODAG project
+# https://www.github.com/CS-SI/EODAG
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import unittest
+
+from tests.context import Asset, AssetsDict, mock
+
+
+class CustomAsset(Asset):
+ pass
+
+
+class CustomAssetsDict(AssetsDict):
+ def _make_asset(self, key, value):
+ return CustomAsset(self.product, key, value)
+
+
+class HookedAssetsDict(AssetsDict):
+ def __init__(self, product):
+ self.setitem_keys = []
+ super().__init__(product)
+
+ def __setitem__(self, key, value):
+ self.setitem_keys.append(key)
+ super().__setitem__(key, value)
+
+
+class TestAssetsDict(unittest.TestCase):
+ @mock.patch.object(AssetsDict, "sort")
+ def test_update_and_setitem_use_asset_factory_override(self, mock_sort):
+ """Use the overridden asset factory and sort assets after updates."""
+ assets = CustomAssetsDict(object())
+
+ assets.update(
+ {
+ "z": {"href": "z"},
+ "a": {"href": "a"},
+ }
+ )
+ assets["m"] = {"href": "m"}
+
+ mock_sort.assert_called()
+
+ self.assertTrue(
+ all(isinstance(asset, CustomAsset) for asset in assets.values())
+ )
+
+ def test_update_preserves_overridden_setitem(self):
+ """Use an overridden __setitem__ when updating a subclass."""
+ assets = HookedAssetsDict(object())
+
+ assets.update({"z": {"href": "z"}, "a": {"href": "a"}})
+
+ self.assertEqual(assets.setitem_keys, ["z", "a"])
+
+ def test_update_and_setitem_sort_assets_and_remove_private_fields(self):
+ """Sort assets and remove private fields after updates."""
+ assets = AssetsDict(object())
+
+ assets.update(
+ {
+ "z": {"href": "z", "_internal": "remove"},
+ "a": {"href": "a", "_internal": "remove"},
+ }
+ )
+ assets["m"] = {"href": "m", "_internal": "remove"}
+
+ self.assertEqual(list(assets), ["a", "m", "z"])
+ self.assertDictEqual(
+ {key: dict(asset) for key, asset in assets.items()},
+ {
+ "a": {"href": "a"},
+ "m": {"href": "m"},
+ "z": {"href": "z"},
+ },
+ )
=====================================
tests/units/test_utils.py
=====================================
@@ -34,6 +34,7 @@ from requests.exceptions import RequestException
from shapely.geometry import Point, Polygon
from eodag.utils import get_geometry_from_ecmwf_area, get_geometry_from_ecmwf_feature
+from eodag.utils.logging import TqdmLoggingHandler
from tests.context import (
HTTP_REQ_TIMEOUT,
USER_AGENT,
@@ -203,6 +204,56 @@ class TestUtils(unittest.TestCase):
bar(1)
self.assertEqual(tqdm_out.getvalue(), "")
+ @mock.patch("tqdm.auto.tqdm.write")
+ def test_tqdm_logging_handler_uses_tqdm_write(self, tqdm_write):
+ """Test logging through tqdm preserves the progress bar output."""
+ handler = TqdmLoggingHandler(stream=StringIO())
+ handler.setFormatter(logging.Formatter("%(message)s"))
+ record = logging.LogRecord(
+ name="eodag",
+ level=logging.INFO,
+ pathname=__file__,
+ lineno=1,
+ msg="log message",
+ args=(),
+ exc_info=None,
+ )
+
+ handler.emit(record)
+
+ tqdm_write.assert_called_once_with(
+ "log message", file=handler.stream, end=handler.terminator
+ )
+
+ @mock.patch("tqdm.auto.tqdm.write")
+ def test_tqdm_logging_handler_falls_back_to_stream(self, tqdm_write):
+ """Test disabled progress bars use the regular stream handler."""
+ setup_logging(verbose=2, no_progress_bar=True)
+ output = StringIO()
+ handler = TqdmLoggingHandler(stream=output)
+ handler.setFormatter(logging.Formatter("%(message)s"))
+ record = logging.LogRecord(
+ name="eodag",
+ level=logging.INFO,
+ pathname=__file__,
+ lineno=1,
+ msg="log message",
+ args=(),
+ exc_info=None,
+ )
+
+ handler.emit(record)
+
+ self.assertEqual(output.getvalue(), "log message\n")
+ tqdm_write.assert_not_called()
+
+ def test_setup_logging_installs_tqdm_handler(self):
+ """Test console logging uses the handler that cooperates with tqdm."""
+ setup_logging(verbose=2)
+
+ logger = logging.getLogger("eodag")
+ self.assertIsInstance(logger.handlers[0], TqdmLoggingHandler)
+
def test_merge_mappings(self):
"""Test merge_mappings merges configuration mappings correctly."""
=====================================
tox.ini
=====================================
@@ -16,6 +16,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
[tox]
+requires = tox-uv
envlist = py39, py310, py311, py312, py313, py314, benchmark, docs, pypi, linters
skipdist = true
@@ -30,8 +31,6 @@ python =
3.14: py314
[testenv]
-install_command = uv pip install --upgrade {opts} {packages}
-
commands =
python -c "import os; os.makedirs('test-reports', exist_ok=True)"
pytest -v --instafail \
@@ -88,7 +87,6 @@ skip_install = true
deps =
build
twine
- uv
commands =
# build package to be checked
python -m build --installer uv
View it on GitLab: https://salsa.debian.org/debian-gis-team/eodag/-/commit/048acf2aeb85b97005c46dcd076c6f5beefc5a33
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/eodag/-/commit/048acf2aeb85b97005c46dcd076c6f5beefc5a33
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20260820/5a1db2f4/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list