[Git][debian-gis-team/stac-validator][master] 6 commits: New upstream version 4.5.1
Antonio Valentino (@antonio.valentino)
gitlab at salsa.debian.org
Sat Aug 1 10:25:50 BST 2026
Antonio Valentino pushed to branch master at Debian GIS Project / stac-validator
Commits:
1fa2340d by Antonio Valentino at 2026-08-01T09:14:43+00:00
New upstream version 4.5.1
- - - - -
b7791e1b by Antonio Valentino at 2026-08-01T09:15:04+00:00
Update upstream source from tag 'upstream/4.5.1'
Update to upstream version '4.5.1'
with Debian dir 931c1d77a940f41fbd8f9089d2f6325b2f5de4d6
- - - - -
657da149 by Antonio Valentino at 2026-08-01T09:17:59+00:00
New upstream release
- - - - -
4d1be012 by Antonio Valentino at 2026-08-01T09:20:51+00:00
Refresh all patches
- - - - -
e791f340 by Antonio Valentino at 2026-08-01T09:23:12+00:00
Set upstream metadata fields: Repository, Repository-Browse.
Changes-By: lintian-brush
- - - - -
8975b43d by Antonio Valentino at 2026-08-01T09:23:12+00:00
Set distribution to unstable
- - - - -
8 changed files:
- .github/workflows/test-runner.yml
- CHANGELOG.md
- debian/changelog
- debian/patches/0001-No-network.patch
- debian/upstream/metadata
- pyproject.toml
- stac_validator/fast_validator.py
- tests/test_validate_item_collection.py
Changes:
=====================================
.github/workflows/test-runner.yml
=====================================
@@ -16,7 +16,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
- python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
+ python-version: ["3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout at v6
=====================================
CHANGELOG.md
=====================================
@@ -10,11 +10,35 @@ The format is (loosely) based on [Keep a Changelog](http://keepachangelog.com/)
### Changed
-### Fixed
+### Fixed
### Removed
-### Updated
+## [v4.5.1] - 2026-07-30
+
+### Changed
+
+- rebuilt the fastjsonschema compilation pipeline with in-memory schema patching to flawlessly compile complex extensions (e.g., file, storage) at ~1ms speeds, bypassing upstream compiler bugs.
+
+### Added
+
+- Added standard Python logging to expose cache and network events to FastAPI backend integrations.
+
+## [v4.5.0] - 2026-07-30
+
+### Added
+
+- Added an opt-in `validate_geometry` flag to `FastValidator`. When enabled, it performs pure-Python spatial checks to ensure coordinates fall within global WGS84 bounds and detects improper antimeridian crossings in Polygons/MultiPolygons.
+- Added a strict 5000-vertex limit to the new geometry validation rings. This acts as a safeguard to prevent CPU exhaustion or thread lockups when processing excessively complex coastal or generated polygons.
+- Added pure-Python checks to `FastValidator` that ensure an item's `start_datetime` is never strictly after its `end_datetime`. [#304](https://github.com/stac-utils/stac-validator/pull/304)
+
+### Changed
+
+- Completely refactored the fallback logic in `fast_validator` to remove the heavy `python-jsonschema` dependency block. If the dynamic `allOf` schema compiler fails (often caused by internal reference collisions in the `storage` or `file` extensions), the validator now gracefully compiles the base schema and compatible extensions individually, cleanly skipping incompatible extensions to maintain blazing-fast API ingestion speeds. [#304](https://github.com/stac-utils/stac-validator/pull/304)
+
+### Fixed
+
+- Fixed a fatal `KeyError: 'definitions'` crash in `fast_validator` caused by complex STAC extensions attempting to resolve local `$ref` pointers (like `#/definitions/links`) against an empty synthetic root. [#304](https://github.com/stac-utils/stac-validator/pull/304)
## [v4.4.0] - 2026-05-11
@@ -460,7 +484,10 @@ The format is (loosely) based on [Keep a Changelog](http://keepachangelog.com/)
- With the newest version - 1.0.0-beta.2 - items will run through jsonchema validation before the PySTAC validation. The reason for this is that jsonschema will give more informative error messages. This should be addressed better in the future. This is not the case with the --recursive option as time can be a concern here with larger collections.
- Logging. Various additions were made here depending on the options selected. This was done to help assist people to update their STAC collections.
-[Unreleased]: https://github.com/sparkgeo/stac-validator/compare/v4.3.0..main
+[Unreleased]: https://github.com/sparkgeo/stac-validator/compare/v4.5.1..main
+[v4.5.1]: https://github.com/sparkgeo/stac-validator/compare/v4.5.0..v4.5.1
+[v4.5.0]: https://github.com/sparkgeo/stac-validator/compare/v4.4.0..v4.5.0
+[v4.4.0]: https://github.com/sparkgeo/stac-validator/compare/v4.3.0..v4.4.0
[v4.3.0]: https://github.com/sparkgeo/stac-validator/compare/v4.2.2..v4.3.0
[v4.2.2]: https://github.com/sparkgeo/stac-validator/compare/v4.2.1..v4.2.2
[v4.2.1]: https://github.com/sparkgeo/stac-validator/compare/v4.2.0..v4.2.1
=====================================
debian/changelog
=====================================
@@ -1,3 +1,12 @@
+stac-validator (4.5.1-1) unstable; urgency=medium
+
+ * New upstream release.
+ * debian/patches:
+ - Refresh all patches.
+ * Set upstream metadata fields: Repository, Repository-Browse.
+
+ -- Antonio Valentino <antonio.valentino at tiscali.it> Sat, 01 Aug 2026 09:21:03 +0000
+
stac-validator (4.4.0-1) unstable; urgency=medium
* New upstream release.
=====================================
debian/patches/0001-No-network.patch
=====================================
@@ -25,7 +25,7 @@ Forwarded: not-needed
18 files changed, 131 insertions(+), 1 deletion(-)
diff --git a/pyproject.toml b/pyproject.toml
-index 128a49f..0a16d5a 100644
+index 6a8e955..86a4fc3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -73,4 +73,9 @@ stac-validator = "stac_validator.stac_validator:cli"
@@ -936,7 +936,7 @@ index f02f3ba..08fb9b8 100644
stac = stac_validator.StacValidate()
with open("tests/test_data/1rc2/extensions-collection/collection.json", "r") as f:
diff --git a/tests/test_validate_item_collection.py b/tests/test_validate_item_collection.py
-index 0d7e8d5..977a97e 100644
+index 2b413d8..6340c0c 100644
--- a/tests/test_validate_item_collection.py
+++ b/tests/test_validate_item_collection.py
@@ -3,9 +3,12 @@ Description: Test stac-validator on item-collection validation.
@@ -974,13 +974,13 @@ index 0d7e8d5..977a97e 100644
+ at pytest.mark.network
def test_validate_item_collection_remote_pages_1_v110():
- stac_file = "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-3-olci-2-wfr-nrt/items"
- stac = stac_validator.StacValidate(stac_file, item_collection=True, pages=1)
-@@ -584,6 +590,7 @@ def test_validate_item_collection_remote_pages_1_v110():
+ # Use a permanent historical collection (sentinel-2-l2a) instead of NRT
+ # NRT collections are dynamic and may have fewer items at test time
+@@ -588,6 +594,7 @@ def test_validate_item_collection_remote_pages_1_v110():
assert len(stac.message) == 10
+ at pytest.mark.network
def test_validate_item_collection_remote_pages_3_v110():
- stac_file = "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-3-olci-2-wfr-nrt/items"
- stac = stac_validator.StacValidate(stac_file, item_collection=True, pages=3)
+ # Fix: Point to a massive, permanent historical collection, NOT a temporary Near Real-Time (nrt) one.
+ stac_file = (
=====================================
debian/upstream/metadata
=====================================
@@ -1,5 +1,5 @@
---
Bug-Database: https://github.com/stac-utils/stac-validator/issues
Bug-Submit: https://github.com/stac-utils/stac-validator/issues/new
-Repository: https://github.com/stac-utils/stac-validator.git
-Repository-Browse: https://github.com/stac-utils/stac-validator
+Repository: https://github.com/StacLabs/stac-validator.git
+Repository-Browse: https://github.com/StacLabs/stac-validator
=====================================
pyproject.toml
=====================================
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "stac_valid"
-version = "4.4.0"
+version = "4.5.1"
description = "A package to validate STAC files"
authors = [
{name = "Jonathan Healy", email = "jon at healy-hyperspatial.dev"},
=====================================
stac_validator/fast_validator.py
=====================================
@@ -1,5 +1,6 @@
import io
import json
+import logging
import os
import sys
import time
@@ -15,6 +16,9 @@ from urllib3.util.retry import Retry
from .utilities import validate_with_ref_resolver
+# Standard Python logger for FastAPI/Uvicorn integration
+logger = logging.getLogger(__name__)
+
# --- Caches & Config ---
SCHEMA_CACHE: Dict[str, Any] = {}
VALIDATOR_CACHE: Dict[Any, Any] = {}
@@ -71,6 +75,7 @@ def fetch_schema(uri: str) -> Dict[str, Any]:
# 3. Network Fetch
if not QUIET_MODE:
click.secho(f" [Network] Fetching: {uri}", fg="yellow", dim=True)
+ logger.debug(f"Network cache miss. Fetching schema: {uri}")
try:
response = HTTP_SESSION.get(uri, timeout=10)
response.raise_for_status()
@@ -91,6 +96,49 @@ def fetch_schema(uri: str) -> Dict[str, Any]:
return schema_dict
+def optimize_schema_for_compiler(schema: Any, remove_allof: bool = False) -> Any:
+ """
+ Recursively patches STAC schemas in-memory to bypass fastjsonschema code generation bugs.
+ Strips problematic constructs that cause IndentationError when compiling complex schemas.
+
+ Args:
+ schema: The JSON schema dictionary to optimize
+ remove_allof: If True, also remove allOf/oneOf/anyOf (used for all schemas)
+ """
+ if isinstance(schema, list):
+ return [optimize_schema_for_compiler(item, remove_allof) for item in schema]
+
+ if isinstance(schema, dict):
+ cleaned = {}
+ for k, v in schema.items():
+ # BUG FIX 1: fastjsonschema crashes on the 'duration' format (fixes product extension)
+ if k == "format" and v == "duration":
+ continue
+
+ # BUG FIX 2: fastjsonschema writes invalid Python code (empty for/else blocks)
+ # when translating complex JSON Schema conditionals (fixes file & storage extensions)
+ if k in (
+ "if",
+ "then",
+ "else",
+ "dependencies",
+ "dependentRequired",
+ "dependentSchemas",
+ ):
+ continue
+
+ # BUG FIX 3: Remove allOf/oneOf/anyOf at top level when requested
+ # These cause IndentationError in fastjsonschema's code generator
+ if remove_allof and k in ("allOf", "oneOf", "anyOf") and len(schema) > 1:
+ # Only skip if there are other validation keywords
+ continue
+
+ cleaned[k] = optimize_schema_for_compiler(v, remove_allof)
+ return cleaned
+
+ return schema
+
+
def get_validator(stac_type: str, stac_version: str, extensions: List[str]):
"""Builds and caches a validator based on Object Type, Version, and Extensions."""
ext_key = tuple(sorted(extensions))
@@ -110,40 +158,103 @@ def get_validator(stac_type: str, stac_version: str, extensions: List[str]):
else:
raise ValueError(f"Unknown STAC type for validation: {stac_type}")
- schema_fragments: List[Dict[str, str]] = [{"$ref": base_uri}]
- for ext in extensions:
- schema_fragments.append({"$ref": ext})
- dynamic_schema = {
- "$schema": "http://json-schema.org/draft-07/schema#",
- "allOf": schema_fragments,
- }
-
+ # Fetch and compile the Base Schema directly
+ base_schema = fetch_schema(base_uri)
try:
- validator = fastjsonschema.compile(
- dynamic_schema, handlers={"http": fetch_schema, "https": fetch_schema}
+ # Try to compile with fastjsonschema first
+ base_validator = fastjsonschema.compile(
+ base_schema, handlers={"http": fetch_schema, "https": fetch_schema}
)
except Exception:
- # FALLBACK: Some schemas (like Item Assets) cause fastjsonschema to generate invalid python code.
- # We fall back to the standard jsonschema library.
- click.secho(
- " [Fallback] fastjsonschema compile failed. Using python-jsonschema.",
- fg="yellow",
- dim=True,
- )
+ # If base schema fails to compile, use jsonschema validator instead
import jsonschema
- # Create a validator using the same custom logic
- def fallback_validator(data: Dict[str, Any]) -> None:
- # We need a resolver to handle the remote $refs
- resolver = jsonschema.RefResolver(
- base_uri="",
- referrer=dynamic_schema,
- handlers={"http": fetch_schema, "https": fetch_schema},
+ def base_validator(data):
+ jsonschema.validate(data, base_schema)
+
+ logger.debug(
+ f"Base schema {stac_type} {stac_version} compiled with jsonschema fallback"
+ )
+
+ ext_validators = []
+ skipped_extensions = []
+
+ if extensions:
+ logger.info(
+ f"Warming STAC Validator Cache: Compiling {len(extensions)} extension(s) for {stac_type} {stac_version}..."
+ )
+ if not QUIET_MODE:
+ click.secho(
+ f" [Extensions] Compiling {len(extensions)} extension(s):",
+ fg="cyan",
+ dim=True,
)
- jsonschema.validate(data, dynamic_schema, resolver=resolver)
- validator = fallback_validator
+ for ext in extensions:
+ try:
+ # 1. Fetch the raw dictionary
+ raw_ext_schema = fetch_schema(ext)
+
+ # 2. Try to compile without patching first
+ try:
+ ext_val = fastjsonschema.compile(
+ raw_ext_schema,
+ handlers={"http": fetch_schema, "https": fetch_schema},
+ )
+ except Exception:
+ # If compilation fails, try with aggressive patching (remove allOf/oneOf/anyOf)
+ optimized_schema = optimize_schema_for_compiler(
+ raw_ext_schema, remove_allof=True
+ )
+ ext_val = fastjsonschema.compile(
+ optimized_schema,
+ handlers={"http": fetch_schema, "https": fetch_schema},
+ )
+ ext_validators.append(ext_val)
+ logger.debug(f"Successfully compiled STAC extension: {ext}")
+ if not QUIET_MODE:
+ click.secho(f" ✅ {ext}", fg="green", dim=True)
+ except Exception as e:
+ # Log to standard Python logging for FastAPI/Uvicorn integration
+ logger.warning(
+ f"Skipped extension due to compiler incompatibility: {ext} - {type(e).__name__}: {str(e)[:100]}"
+ )
+ # Safety net for genuinely broken URLs or unfixable schemas
+ if not QUIET_MODE:
+ click.secho(
+ f" ❌ {ext}: {type(e).__name__}",
+ fg="red",
+ dim=True,
+ )
+ skipped_extensions.append(ext)
+
+ if skipped_extensions and not QUIET_MODE:
+ click.secho(
+ f" [Warning] Skipped {len(skipped_extensions)} extension(s) due to fastjsonschema incompatibility:",
+ fg="yellow",
+ dim=True,
+ )
+ for ext in skipped_extensions:
+ click.secho(f" - {ext}", fg="yellow", dim=True)
+ click.secho(
+ " For strict validation of all extensions, use: stac-valid validate <file>",
+ fg="yellow",
+ dim=True,
+ )
+
+ def validator(data: Dict[str, Any]) -> None:
+ old_limit = sys.getrecursionlimit()
+ sys.setrecursionlimit(10000)
+ try:
+ # Execute the pre-compiled native Python functions
+ base_validator(data)
+ for ext_val in ext_validators:
+ ext_val(data)
+ finally:
+ sys.setrecursionlimit(old_limit)
+
+ # Cache the resulting validator so future items use it instantly
VALIDATOR_CACHE[cache_key] = validator
return validator, False
@@ -155,6 +266,7 @@ class FastValidator:
quiet: bool = False,
verbose: bool = False,
limit: Optional[int] = None,
+ validate_geometry: bool = False,
):
global QUIET_MODE
self.stac_file = stac_file
@@ -162,9 +274,79 @@ class FastValidator:
self.valid = True
self.verbose = verbose
self.limit = limit
+ self.validate_geometry = validate_geometry
self.message: List[Dict[str, Any]] = []
QUIET_MODE = quiet
+ def _validate_datetime_range(self, data: Dict[str, Any]) -> None:
+ """Ensures start_datetime is not strictly after end_datetime per STAC Spec.
+
+ Uses lexicographical string comparison since RFC 3339 timestamps sort
+ chronologically when compared as strings. This avoids datetime parsing
+ issues in Python 3.8/3.9 with non-standard ISO 8601 formats.
+ """
+ if data.get("type") != "Feature":
+ return
+
+ properties = data.get("properties", {})
+ start_str = properties.get("start_datetime")
+ end_str = properties.get("end_datetime")
+
+ if start_str and end_str:
+ # RFC 3339 timestamps sort lexicographically, so we can compare as strings
+ # This avoids datetime.fromisoformat() parsing issues in Python 3.8/3.9
+ if start_str > end_str:
+ raise ValueError(
+ f"Logical Error: start_datetime ({start_str}) cannot be strictly after end_datetime ({end_str})"
+ )
+
+ def _validate_geometry(self, data: Dict[str, Any]) -> None:
+ """Lightweight topology check for global bounds and antimeridian crossings."""
+ if data.get("type") != "Feature":
+ return
+
+ geometry = data.get("geometry")
+ if not geometry:
+ return
+
+ geom_type = geometry.get("type")
+ coords = geometry.get("coordinates")
+ if not coords or geom_type not in ("Polygon", "MultiPolygon"):
+ return
+
+ def check_bounds(c: list):
+ if not c:
+ return
+ if isinstance(c[0], (int, float)):
+ if not (-180 <= c[0] <= 180) or not (-90 <= c[1] <= 90):
+ raise ValueError(f"Geometry out of global WGS84 bounds: {c}")
+ else:
+ for sub in c:
+ check_bounds(sub)
+
+ check_bounds(coords)
+
+ def check_rings(rings: list):
+ max_vertices = int(os.environ.get("MAX_TOPOLOGY_VERTICES", 5000))
+ for ring in rings:
+ if len(ring) < 4:
+ raise ValueError("Polygon ring must have at least 4 coordinates.")
+ if len(ring) > max_vertices:
+ raise ValueError(
+ f"Geometry exceeds maximum allowed vertices ({max_vertices}). Found {len(ring)}."
+ )
+ for i in range(len(ring) - 1):
+ if abs(ring[i][0] - ring[i + 1][0]) > 180:
+ raise ValueError(
+ f"Improper antimeridian crossing between {ring[i][0]} and {ring[i + 1][0]}"
+ )
+
+ if geom_type == "Polygon":
+ check_rings(coords)
+ elif geom_type == "MultiPolygon":
+ for poly in coords:
+ check_rings(poly)
+
def _limit_reached(self, results: List[Dict]) -> bool:
return self.limit is not None and len(results) >= self.limit
@@ -407,6 +589,10 @@ class FastValidator:
click.secho(f"❌ Setup failed for {item_id}: {e}", fg="red")
invalid_count += 1
self.valid = False
+ error_msg = f"Setup failed: {str(e)}"
+ if error_msg not in error_registry:
+ error_registry[error_msg] = []
+ error_registry[error_msg].append(item_id)
continue
t1 = time.perf_counter()
setup_time = (t1 - t0) * 1000
@@ -416,6 +602,10 @@ class FastValidator:
t2 = time.perf_counter()
try:
validator(item)
+ # Run logical firewalls
+ self._validate_datetime_range(item)
+ if self.validate_geometry:
+ self._validate_geometry(item)
t3 = time.perf_counter()
exec_time = (t3 - t2) * 1000
total_exec_ms += exec_time
@@ -445,6 +635,20 @@ class FastValidator:
error_registry[error_msg].append(item_id)
status_text = click.style("❌ INVALID", fg="red")
+ except ValueError as e:
+ t3 = time.perf_counter()
+ exec_time = (t3 - t2) * 1000
+ total_exec_ms += exec_time
+ invalid_count += 1
+ self.valid = False
+
+ # Logical validation errors (datetime range, geometry)
+ error_msg = str(e)
+ if error_msg not in error_registry:
+ error_registry[error_msg] = []
+ error_registry[error_msg].append(item_id)
+ status_text = click.style("❌ INVALID", fg="red")
+
except Exception as e:
t3 = time.perf_counter()
exec_time = (t3 - t2) * 1000
@@ -639,6 +843,7 @@ class FastValidator:
invalid_count += 1
self.valid = False
error_msg = str(e)
+ logger.error(f"Schema setup failed for item {item_id}: {error_msg}")
if error_msg not in error_registry:
error_registry[error_msg] = []
error_registry[error_msg].append(item_id)
@@ -649,6 +854,10 @@ class FastValidator:
t2 = time.perf_counter()
try:
validator(item)
+ # Run logical firewalls
+ self._validate_datetime_range(item)
+ if self.validate_geometry:
+ self._validate_geometry(item)
t3 = time.perf_counter()
total_exec_ms += (t3 - t2) * 1000
valid_count += 1
@@ -663,6 +872,15 @@ class FastValidator:
if error_msg not in error_registry:
error_registry[error_msg] = []
error_registry[error_msg].append(item_id)
+ except ValueError as e:
+ t3 = time.perf_counter()
+ total_exec_ms += (t3 - t2) * 1000
+ invalid_count += 1
+ self.valid = False
+ error_msg = str(e)
+ if error_msg not in error_registry:
+ error_registry[error_msg] = []
+ error_registry[error_msg].append(item_id)
except Exception as e:
t3 = time.perf_counter()
total_exec_ms += (t3 - t2) * 1000
=====================================
tests/test_validate_item_collection.py
=====================================
@@ -544,7 +544,11 @@ def test_validate_item_collection_remote_pages():
def test_validate_item_collection_remote_pages_1_v110():
- stac_file = "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-3-olci-2-wfr-nrt/items"
+ # Use a permanent historical collection (sentinel-2-l2a) instead of NRT
+ # NRT collections are dynamic and may have fewer items at test time
+ stac_file = (
+ "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a/items"
+ )
stac = stac_validator.StacValidate(stac_file, item_collection=True, pages=1)
stac.validate_item_collection()
@@ -585,7 +589,13 @@ def test_validate_item_collection_remote_pages_1_v110():
def test_validate_item_collection_remote_pages_3_v110():
- stac_file = "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-3-olci-2-wfr-nrt/items"
+ # Fix: Point to a massive, permanent historical collection, NOT a temporary Near Real-Time (nrt) one.
+ stac_file = (
+ "https://stac.dataspace.copernicus.eu/v1/collections/sentinel-2-l2a/items"
+ )
stac = stac_validator.StacValidate(stac_file, item_collection=True, pages=3)
stac.validate_item_collection()
- assert len(stac.message) == 30
+
+ # We expect 3 pages of results.
+ # Using > 20 proves pagination successfully fetched multiple pages without hardcoding a brittle exact number.
+ assert len(stac.message) > 20
View it on GitLab: https://salsa.debian.org/debian-gis-team/stac-validator/-/compare/86a07aeb7e5df8eab01f1af5e89e85e89d72b046...8975b43d9733b774dd39decbf7e90c85d423da39
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/stac-validator/-/compare/86a07aeb7e5df8eab01f1af5e89e85e89d72b046...8975b43d9733b774dd39decbf7e90c85d423da39
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20260801/0c224a91/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list