[Git][debian-gis-team/asf-search][upstream] New upstream version 7.0.4

Antonio Valentino (@antonio.valentino) gitlab at salsa.debian.org
Sat Feb 3 09:15:23 GMT 2024



Antonio Valentino pushed to branch upstream at Debian GIS Project / asf-search


Commits:
08c80550 by Antonio Valentino at 2024-02-03T09:11:22+00:00
New upstream version 7.0.4
- - - - -


11 changed files:

- CHANGELOG.md
- asf_search/ASFSession.py
- asf_search/CMR/datasets.py
- asf_search/CMR/subquery.py
- asf_search/constants/DATASET.py
- asf_search/constants/INTERNAL.py
- asf_search/constants/PRODUCT_TYPE.py
- asf_search/search/search_generator.py
- tests/ASFSession/test_ASFSession.py
- tests/Search/test_search.py
- tests/yml_tests/test_search.yml


Changes:

=====================================
CHANGELOG.md
=====================================
@@ -25,6 +25,29 @@ and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 -
 
 -->
+------
+## [v7.0.4](https://github.com/asfadmin/Discovery-asf_search/compare/v7.0.3...v7.0.4)
+### Changed
+- `OPERA-S1-CALIBRATION` dataset is now the `OPERA-S1-CALVAL` dataset, uses the `OPERA_S1_CALVAL` constant
+
+------
+## [v7.0.3](https://github.com/asfadmin/Discovery-asf_search/compare/v7.0.2...v7.0.3)
+### Fixed
+- Fixes typo for constant variable name `constants.PRODUCT_TYPE.CSLC_STATIC`
+- Normalizes concept-id lists for `OPERA-S1` dataset product types
+
+### Changed
+- Completely removes `CSLC-STATIC` Calval and `RTC-STATIC` Calval collections from concept-id lists
+
+------
+## [v7.0.2](https://github.com/asfadmin/Discovery-asf_search/compare/v7.0.1...v7.0.2)
+### Added
+- Adds `AUTH_COOKIES` to `constants.INTERNAL` and `auth_cookie_names` variable for `ASFSession`, used by `auth_with_creds()` and `auth_with_cookiejar()` to confirm login.
+
+### Fixed
+- Attempting to authorize `ASFSession` against CMR UAT using `auth_with_creds()` and `auth_with_cookiejar()` no longer raises an exception on valid login
+- Fixes custom host in `ASFSearchOptions` raising type error while searching.
+
 ------
 ## [v7.0.1](https://github.com/asfadmin/Discovery-asf_search/compare/v7.0.0...v7.0.1)
 ### Fixed


=====================================
asf_search/ASFSession.py
=====================================
@@ -1,4 +1,5 @@
 import platform
+from typing import Dict, List, Union
 import requests
 from requests.utils import get_netrc_auth
 import http.cookiejar
@@ -12,7 +13,8 @@ class ASFSession(requests.Session):
                 asf_auth_host: str = None,
                 cmr_host: str = None,
                 cmr_collections: str = None,
-                auth_domains: str = None 
+                auth_domains: List[str] = None,
+                auth_cookie_names: List[str] = None
                 ):
         """
         ASFSession is a subclass of `requests.Session`, and is meant to ease downloading ASF hosted data by simplifying logging in to Earthdata Login.
@@ -29,7 +31,7 @@ class ASFSession(requests.Session):
         `cmr_host`: the base CMR endpoint to test EDL login tokens against. Defaults to `asf_search.constants.INTERNAL.CMR_HOST`
         `cmr_collections`: the CMR endpoint path login tokens will be tested against. Defaults to `asf_search.constants.INTERNAL.CMR_COLLECTIONS`
         `auth_domains`: the list of authorized endpoints that are allowed to pass auth credentials. Defaults to `asf_search.constants.INTERNAL.AUTH_DOMAINS`. Authorization headers WILL NOT be stripped from the session object when redirected through these domains.
-        
+        `auth_cookie_names`: the list of cookie names to use when verifying with `auth_with_creds()` & `auth_with_cookiejar()`
         More information on Earthdata Login can be found here:
         https://urs.earthdata.nasa.gov/documentation/faq
         """
@@ -50,6 +52,7 @@ class ASFSession(requests.Session):
         self.cmr_host = INTERNAL.CMR_HOST if cmr_host is None else cmr_host
         self.cmr_collections = INTERNAL.CMR_COLLECTIONS if cmr_collections is None else cmr_collections
         self.auth_domains = INTERNAL.AUTH_DOMAINS if auth_domains is None else auth_domains
+        self.auth_cookie_names = INTERNAL.AUTH_COOKIES if auth_cookie_names is None else auth_cookie_names
 
     def __eq__(self, other):
         return self.auth == other.auth \
@@ -71,7 +74,7 @@ class ASFSession(requests.Session):
         self.auth = (username, password)
         self.get(login_url)
 
-        if "urs_user_already_logged" not in self.cookies.get_dict():
+        if not self._check_auth_cookies(self.cookies.get_dict()):
             raise ASFAuthenticationError("Username or password is incorrect")
 
         return self
@@ -87,14 +90,14 @@ class ASFSession(requests.Session):
         self.headers.update({'Authorization': 'Bearer {0}'.format(token)})
 
         url = f"https://{self.cmr_host}{self.cmr_collections}"
-        response = self.get(url)        
+        response = self.get(url)
 
         if not 200 <= response.status_code <= 299:
             raise ASFAuthenticationError("Invalid/Expired token passed")
 
         return self
 
-    def auth_with_cookiejar(self, cookies: http.cookiejar):
+    def auth_with_cookiejar(self, cookies: http.cookiejar.CookieJar):
         """
         Authenticates the session using a pre-existing cookiejar
 
@@ -103,7 +106,7 @@ class ASFSession(requests.Session):
         :return ASFSession: returns self for convenience
         """
         
-        if "urs_user_already_logged" not in cookies:
+        if not self._check_auth_cookies(cookies):
             raise ASFAuthenticationError("Cookiejar does not contain login cookies")
 
         for cookie in cookies:
@@ -114,6 +117,9 @@ class ASFSession(requests.Session):
 
         return self
 
+    def _check_auth_cookies(self, cookies: Union[http.cookiejar.CookieJar, Dict]) -> bool:
+        return any(cookie in self.auth_cookie_names for cookie in cookies)
+
     def rebuild_auth(self, prepared_request: requests.Request, response: requests.Response):
         """
         Overrides requests.Session.rebuild_auth() default behavior of stripping the Authorization header
@@ -150,6 +156,7 @@ class ASFSession(requests.Session):
             'asf_auth_host': self.asf_auth_host,
             'cmr_host': self.cmr_host,
             'cmr_collections': self.cmr_collections,
-            'auth_domains': self.auth_domains
+            'auth_domains': self.auth_domains,
+            'auth_cookie_names': self.auth_cookie_names
         }
         return state


=====================================
asf_search/CMR/datasets.py
=====================================
@@ -166,7 +166,7 @@ dataset_collections = {
         "OPERA_L2_RTC-S1-STATIC_V1": ["C1259981910-ASF", "C2795135174-ASF"],
         "OPERA_L2_RTC-S1_PROVISIONAL_V0": ["C1257995186-ASF"],
     },
-    "OPERA-S1-CALIBRATION": {
+    "OPERA-S1-CALVAL": {
         "OPERA_L2_CSLC-S1_CALVAL_V1": ["C1260721945-ASF", "C2803501758-ASF"],
         "OPERA_L2_RTC-S1_CALVAL_V1": ["C1260721853-ASF", "C2803501097-ASF"],
     },
@@ -331,6 +331,8 @@ dataset_collections = {
 
 collections_per_platform = {
     "SENTINEL-1A": [
+        "C2803501758-ASF",
+        "C2803501097-ASF",
         "C1214470488-ASF",
         "C1214470533-ASF",
         "C1214470576-ASF",
@@ -376,12 +378,13 @@ collections_per_platform = {
         "C1260726384-ASF",
         "C1258354200-ASF",
         "C1259982010-ASF",
+        "C2795135668-ASF",
         "C1260721945-ASF",
         "C1257995185-ASF",
         "C1259976861-ASF",
-        "C1260726378-ASF",
         "C1258354201-ASF",
         "C1259981910-ASF",
+        "C2795135174-ASF",
         "C1260721853-ASF",
         "C1257995186-ASF",
         "C1259974840-ASF",
@@ -417,6 +420,8 @@ collections_per_platform = {
         "C1240784657-ASFDEV",
     ],
     "SENTINEL-1B": [
+        "C2803501758-ASF",
+        "C2803501097-ASF",
         "C1327985661-ASF",
         "C1327985645-ASF",
         "C1595422627-ASF",
@@ -462,12 +467,13 @@ collections_per_platform = {
         "C1260726384-ASF",
         "C1258354200-ASF",
         "C1259982010-ASF",
+        "C2795135668-ASF",
         "C1260721945-ASF",
         "C1257995185-ASF",
         "C1259976861-ASF",
-        "C1260726378-ASF",
         "C1258354201-ASF",
         "C1259981910-ASF",
+        "C2795135174-ASF",
         "C1260721853-ASF",
         "C1257995186-ASF",
         "C1259974840-ASF",
@@ -850,17 +856,13 @@ collections_by_processing_level = {
     "CSLC": [
         "C2777443834-ASF",
         "C1260721945-ASF",
-        "C1259976861-ASF",
-        "C2777443834-ASF",
-        "C1260721945-ASF",
+        "C2803501758-ASF",
         "C1259976861-ASF",
     ],
     "RTC": [
         "C2777436413-ASF",
         "C1260721853-ASF",
-        "C1259974840-ASF",
-        "C2777436413-ASF",
-        "C1260721853-ASF",
+        "C2803501097-ASF",
         "C1259974840-ASF",
     ],
     "GRD_FD": ["C1214471197-ASF", "C1212200781-ASF"],
@@ -869,9 +871,6 @@ collections_by_processing_level = {
         "C2709161906-ASF",
         "C1257024016-ASF",
         "C1257175154-ASFDEV",
-        "C2709161906-ASF",
-        "C1257024016-ASF",
-        "C1257175154-ASFDEV",
     ],
     "GUNW_AMP": [
         "C1596065640-ASF",
@@ -897,12 +896,13 @@ collections_by_processing_level = {
         "C1595765183-ASF",
         "C1225776659-ASF",
     ],
-    "CSLC-STATIC": ["C1259982010-ASF", "C1259982010-ASF"],
+    "CSLC-STATIC": [
+        "C1259982010-ASF", 
+        "C2795135668-ASF"
+    ],
     "RTC-STATIC": [
-        "C1260726378-ASF",
-        "C1259981910-ASF",
-        "C1260726378-ASF",
         "C1259981910-ASF",
+        "C2795135174-ASF"
     ],
     "GRD": [
         "C1661710583-ASF",


=====================================
asf_search/CMR/subquery.py
=====================================
@@ -52,12 +52,14 @@ def _build_subquery(query: List[Tuple[dict]], opts: ASFSearchOptions, list_param
     q = dict()
     for p in query:
         q.update(p)
+    
+    q['provider'] = opts.provider
+    q['host'] = opts.host
+    q['session'] = copy(opts.session)
+
     return ASFSearchOptions(
         **q,
-         provider= opts.provider,
-         host= opts.host,
-         session= copy(opts.session),
-         **list_params
+        **list_params
     )
 
 def get_keyword_concept_ids(params: dict, use_collection_alias: bool=True) -> dict:


=====================================
asf_search/constants/DATASET.py
=====================================
@@ -1,6 +1,6 @@
 SENTINEL1 = 'SENTINEL-1'
 OPERA_S1 = 'OPERA-S1'
-OPERA_S1_CALIBRATION = 'OPERA-S1-CALIBRATION'
+OPERA_S1_CALVAL = 'OPERA-S1-CALVAL'
 SLC_BURST = 'SLC-BURST'
 ALOS_PALSAR = 'ALOS PALSAR'
 ALOS_AVNIR_2 = 'ALOS AVNIR-2'


=====================================
asf_search/constants/INTERNAL.py
=====================================
@@ -14,5 +14,6 @@ EDL_CLIENT_ID = 'BO_n7nTIlMljdvU6kRRB3g'
 DEFAULT_PROVIDER = 'ASF'
 
 AUTH_DOMAINS = ['asf.alaska.edu', 'earthdata.nasa.gov']
+AUTH_COOKIES = ['urs_user_already_logged', 'uat_urs_user_already_logged']
 
 ERROR_REPORTING_ENDPOINT = 'search-error-report.asf.alaska.edu'


=====================================
asf_search/constants/PRODUCT_TYPE.py
=====================================
@@ -99,4 +99,4 @@ GEOTIFF = 'GEOTIFF'
 RTC = 'RTC'
 CSLC = 'CSLC'
 RTC_STATIC = 'RTC-STATIC'
-CSLS_STATIC = 'CSLC-STATIC'
\ No newline at end of file
+CSLC_STATIC = 'CSLC-STATIC'
\ No newline at end of file


=====================================
asf_search/search/search_generator.py
=====================================
@@ -287,7 +287,7 @@ def _get_product_type_key(item: Dict) -> str:
 dataset_to_product_types = {
     'SENTINEL-1': ASFProductType.S1Product,
     'OPERA-S1': ASFProductType.OPERAS1Product,
-    'OPERA-S1-CALIBRATION': ASFProductType.OPERAS1Product,
+    'OPERA-S1-CALVAL': ASFProductType.OPERAS1Product,
     'SLC-BURST': ASFProductType.S1BurstProduct,
 
     'ALOS': ASFProductType.ALOSProduct,


=====================================
tests/ASFSession/test_ASFSession.py
=====================================
@@ -1,9 +1,11 @@
 import numbers
 from typing import List
+import asf_search
 from asf_search.ASFSession import ASFSession
 from requests.cookies import create_cookie
 import http.cookiejar
 import requests
+from multiprocessing import Pool
 
 from unittest.mock import patch
 
@@ -60,4 +62,117 @@ def run_test_asf_session_rebuild_auth(
       
             session.rebuild_auth(req, response)
 
-            assert req.headers.get("Authorization") == final_token
\ No newline at end of file
+            assert req.headers.get("Authorization") == final_token
+
+def test_ASFSession_INTERNAL_mangling():
+    session = asf_search.ASFSession()
+    session.cmr_host = asf_search.constants.INTERNAL.EDL_HOST
+    session.edl_host = asf_search.constants.INTERNAL.EDL_CLIENT_ID
+    session.auth_cookie_names = asf_search.constants.INTERNAL.ASF_AUTH_HOST
+    session.auth_domains = asf_search.constants.INTERNAL.CMR_HOST
+    session.asf_auth_host = asf_search.constants.INTERNAL.CMR_COLLECTIONS
+    session.cmr_collections = asf_search.constants.INTERNAL.AUTH_DOMAINS
+    session.edl_client_id = asf_search.constants.INTERNAL.AUTH_COOKIES
+    
+    # get the current defaults since we're going to mangle them
+    DEFAULT_EDL_HOST = asf_search.constants.INTERNAL.EDL_HOST
+    DEFAULT_EDL_CLIENT_ID = asf_search.constants.INTERNAL.EDL_CLIENT_ID
+    DEFAULT_ASF_AUTH_HOST = asf_search.constants.INTERNAL.ASF_AUTH_HOST
+    DEFAULT_CMR_HOST = asf_search.constants.INTERNAL.CMR_HOST
+    DEFAULT_CMR_COLLECTIONS = asf_search.constants.INTERNAL.CMR_COLLECTIONS
+    DEFAULT_AUTH_DOMAINS = asf_search.constants.INTERNAL.AUTH_DOMAINS
+    DEFAULT_AUTH_COOKIES = asf_search.constants.INTERNAL.AUTH_COOKIES
+
+    uat_domain = 'cmr.uat.earthdata.nasa.gov'
+    edl_client_id = 'custom_client_id'
+    auth_host = 'custom_auth_host'
+    cmr_collection = '/search/granules'
+    auth_domains = ['custom_auth_domain']
+    uat_login_cookie = ['uat_urs_user_already_logged']
+    uat_login_domain = 'uat.urs.earthdata.nasa.gov'
+    
+    asf_search.constants.INTERNAL.CMR_HOST = uat_domain
+    asf_search.constants.INTERNAL.EDL_HOST = uat_login_domain
+    asf_search.constants.INTERNAL.AUTH_COOKIES = uat_login_cookie
+    asf_search.constants.INTERNAL.EDL_CLIENT_ID = edl_client_id
+    asf_search.constants.INTERNAL.AUTH_DOMAINS = auth_domains
+    asf_search.constants.INTERNAL.ASF_AUTH_HOST = auth_host
+    asf_search.constants.INTERNAL.CMR_COLLECTIONS = cmr_collection
+    
+    mangeled_session = asf_search.ASFSession()
+
+    # set them back
+    asf_search.constants.INTERNAL.EDL_HOST = DEFAULT_EDL_HOST
+    asf_search.constants.INTERNAL.EDL_CLIENT_ID = DEFAULT_EDL_CLIENT_ID
+    asf_search.constants.INTERNAL.ASF_AUTH_HOST = DEFAULT_ASF_AUTH_HOST
+    asf_search.constants.INTERNAL.CMR_HOST = DEFAULT_CMR_HOST
+    asf_search.constants.INTERNAL.CMR_COLLECTIONS = DEFAULT_CMR_COLLECTIONS
+    asf_search.constants.INTERNAL.AUTH_DOMAINS = DEFAULT_AUTH_DOMAINS
+    asf_search.constants.INTERNAL.AUTH_COOKIES = DEFAULT_AUTH_COOKIES
+
+    assert mangeled_session.cmr_host == uat_domain
+    assert mangeled_session.edl_host == uat_login_domain
+    assert mangeled_session.auth_cookie_names == uat_login_cookie
+    assert mangeled_session.auth_domains == auth_domains
+    assert mangeled_session.asf_auth_host == auth_host
+    assert mangeled_session.cmr_collections == cmr_collection
+    assert mangeled_session.edl_client_id == edl_client_id
+
+    custom_session = asf_search.ASFSession(
+        cmr_host = uat_domain,
+        edl_host = uat_login_domain,
+        auth_cookie_names = uat_login_cookie,
+        auth_domains = auth_domains,
+        asf_auth_host = auth_host,
+        cmr_collections = cmr_collection,
+        edl_client_id = edl_client_id
+    )
+
+    assert custom_session.cmr_host == uat_domain
+    assert custom_session.edl_host == uat_login_domain
+    assert custom_session.auth_cookie_names == uat_login_cookie
+    assert custom_session.auth_domains == auth_domains
+    assert custom_session.asf_auth_host == auth_host
+    assert custom_session.cmr_collections == cmr_collection
+    assert custom_session.edl_client_id == edl_client_id
+
+def test_ASFSession_pooling():
+    uat_domain = 'cmr.uat.earthdata.nasa.gov'
+    edl_client_id = 'custom_client_id'
+    auth_host = 'custom_auth_host'
+    cmr_collection = '/search/granules'
+    auth_domains = ['custom_auth_domain']
+    uat_login_cookie = ['uat_urs_user_already_logged']
+    uat_login_domain = 'uat.urs.earthdata.nasa.gov'
+
+    custom_session = asf_search.ASFSession(
+        cmr_host = uat_domain,
+        edl_host = uat_login_domain,
+        auth_cookie_names = uat_login_cookie,
+        auth_domains = auth_domains,
+        asf_auth_host = auth_host,
+        cmr_collections = cmr_collection,
+        edl_client_id = edl_client_id
+    )
+    Pool()
+    pool = Pool(processes=2)
+    pool.map(_assert_pooled_instance_variables, [custom_session, custom_session])
+    pool.close()
+    pool.join()
+
+
+def _assert_pooled_instance_variables(session):
+        uat_domain = 'cmr.uat.earthdata.nasa.gov'
+        edl_client_id = 'custom_client_id'
+        auth_host = 'custom_auth_host'
+        cmr_collection = '/search/granules'
+        auth_domains = ['custom_auth_domain']
+        uat_login_cookie = ['uat_urs_user_already_logged']
+        uat_login_domain = 'uat.urs.earthdata.nasa.gov'
+        assert session.cmr_host == uat_domain
+        assert session.edl_host == uat_login_domain
+        assert session.auth_cookie_names == uat_login_cookie
+        assert session.auth_domains == auth_domains
+        assert session.asf_auth_host == auth_host
+        assert session.cmr_collections == cmr_collection
+        assert session.edl_client_id == edl_client_id
\ No newline at end of file


=====================================
tests/Search/test_search.py
=====================================
@@ -16,7 +16,7 @@ from typing import List
 import requests
 import requests_mock
 
-from asf_search.search.search_generator import as_ASFProduct
+from asf_search.search.search_generator import as_ASFProduct, preprocess_opts
 
 SEARCHAPI_URL = 'https://api.daac.asf.alaska.edu'
 SEARCHAPI_ENDPOINT = '/services/search/param?'
@@ -107,11 +107,21 @@ def run_test_dataset_search(datasets: List):
                 assert shortName in valid_shortnames
 
 def run_test_build_subqueries(params: ASFSearchOptions, expected: List):
+    # mainly for getting platform aliases
+    preprocess_opts(params)
     actual = build_subqueries(params)
     for a, b in zip(actual, expected):
-        for a_param, b_param in zip(a, b):
-            if isinstance(b_param, list):
-                assert len(set(b_param).difference(set(a_param))) == 0
+        for key, actual_val in a:
+            expected_val = getattr(b, key)
+            if isinstance(actual_val, list):
+                if len(actual_val) > 0: # ASFSearchOptions leaves empty lists as None
+                    expected_set = set(expected_val)
+                    actual_set = set(actual_val)
+
+                    difference = expected_set.symmetric_difference(actual_set)
+                    assert len(difference) == 0, f"Found {len(difference)} missing entries for subquery generated keyword: \"{key}\"\n{list(difference)}"
+            else:
+                assert actual_val == expected_val
 
 def run_test_keyword_aliasing_results(params: ASFSearchOptions):
     module_response = search(opts=params)


=====================================
tests/yml_tests/test_search.yml
=====================================
@@ -1,7 +1,178 @@
-empty_parameters: {}
-alos_parameters:
-  platform: ALOS
-  maxResults: 3
+sentinel-1_collections: &sentinel-1_collections
+  - "C2803501758-ASF"
+  - "C2803501097-ASF"
+  - "C1214470488-ASF" # S1A
+  - "C1214470533-ASF"
+  - "C1214470576-ASF"
+  - "C1595422627-ASF"
+  - "C1214470496-ASF"
+  - "C1214470532-ASF"
+  - "C1214472977-ASF"
+  - "C1214472336-ASF"
+  - "C1266376001-ASF"
+  - "C1214472994-ASF"
+  - "C1214470732-ASF"
+  - "C1214473170-ASF"
+  - "C1214470561-ASF"
+  - "C1214471521-ASF"
+  - "C1214470682-ASF"
+  - "C2777443834-ASF"
+  - "C2777436413-ASF"
+  - "C1214471197-ASF"
+  - "C1214471960-ASF"
+  - "C1214472978-ASF"
+  - "C1214473165-ASF"
+  - "C2709161906-ASF"
+  - "C1596065640-ASF"
+  - "C1596065639-ASF"
+  - "C1596065641-ASF"
+  - "C1595765183-ASF"
+  - "C2450786986-ASF"
+  - "C1205428742-ASF"
+  - "C1212201032-ASF"
+  - "C1212212560-ASF"
+  - "C1205264459-ASF"
+  - "C1208117434-ASF"
+  - "C1212209035-ASF"
+  - "C1212209226-ASF"
+  - "C1208115009-ASF"
+  - "C1212158327-ASF"
+  - "C1215704763-ASF"
+  - "C1225776654-ASF"
+  - "C1212158318-ASF"
+  - "C1212212493-ASF"
+  - "C1212158326-ASF"
+  - "C1212233976-ASF"
+  - "C1260726384-ASF"
+  - "C1258354200-ASF"
+  - "C1259982010-ASF"
+  - "C2795135668-ASF"
+  - "C1260721945-ASF"
+  - "C1257995185-ASF"
+  - "C1259976861-ASF"
+  - "C1258354201-ASF"
+  - "C1259981910-ASF"
+  - "C2795135174-ASF"
+  - "C1260721853-ASF"
+  - "C1257995186-ASF"
+  - "C1259974840-ASF"
+  - "C1212200781-ASF"
+  - "C1212209075-ASF"
+  - "C1257024016-ASF"
+  - "C1225776655-ASF"
+  - "C1225776657-ASF"
+  - "C1225776658-ASF"
+  - "C1225776659-ASF"
+  - "C1245953394-ASF"
+  - "C1234413245-ASFDEV"
+  - "C1234413229-ASFDEV"
+  - "C1234413237-ASFDEV"
+  - "C1234413238-ASFDEV"
+  - "C1234413236-ASFDEV"
+  - "C1234413230-ASFDEV"
+  - "C1234413232-ASFDEV"
+  - "C1234413235-ASFDEV"
+  - "C1234413240-ASFDEV"
+  - "C1234413234-ASFDEV"
+  - "C1234413241-ASFDEV"
+  - "C1234413233-ASFDEV"
+  - "C1234413243-ASFDEV"
+  - "C1234413244-ASFDEV"
+  - "C1244552887-ASFDEV"
+  - "C1234413228-ASFDEV"
+  - "C1234413231-ASFDEV"
+  - "C1234413239-ASFDEV"
+  - "C1234413242-ASFDEV"
+  - "C1257175154-ASFDEV"
+  - "C1244598379-ASFDEV"
+  - "C1240784657-ASFDEV"
+  - "C1327985661-ASF" # S1B
+  - "C1327985645-ASF"
+  - "C1595422627-ASF"
+  - "C1327985617-ASF"
+  - "C1327985660-ASF"
+  - "C1327985741-ASF"
+  - "C1327985578-ASF"
+  - "C1327985646-ASF"
+  - "C1327985650-ASF"
+  - "C1327985579-ASF"
+  - "C1327985740-ASF"
+  - "C1327985619-ASF"
+  - "C1327985739-ASF"
+  - "C1327985647-ASF"
+  - "C2777443834-ASF"
+  - "C2777436413-ASF"
+  - "C1327985697-ASF"
+  - "C1327985651-ASF"
+  - "C1327985644-ASF"
+  - "C1327985571-ASF"
+  - "C1327985674-ASF"
+  - "C2709161906-ASF"
+  - "C1596065640-ASF"
+  - "C1596065639-ASF"
+  - "C1596065641-ASF"
+  - "C1595765183-ASF"
+  - "C2450786986-ASF"
+  - "C1216244348-ASF"
+  - "C1216244589-ASF"
+  - "C1216244594-ASF"
+  - "C1216244593-ASF"
+  - "C1216244585-ASF"
+  - "C1216244592-ASF"
+  - "C1216244595-ASF"
+  - "C1225776654-ASF"
+  - "C1216244590-ASF"
+  - "C1216244601-ASF"
+  - "C1216244600-ASF"
+  - "C1216244591-ASF"
+  - "C1216244587-ASF"
+  - "C1216244598-ASF"
+  - "C1216244586-ASF"
+  - "C1260726384-ASF"
+  - "C1258354200-ASF"
+  - "C1259982010-ASF"
+  - "C2795135668-ASF"
+  - "C1260721945-ASF"
+  - "C1257995185-ASF"
+  - "C1259976861-ASF"
+  - "C1258354201-ASF"
+  - "C1259981910-ASF"
+  - "C2795135174-ASF"
+  - "C1260721853-ASF"
+  - "C1257995186-ASF"
+  - "C1259974840-ASF"
+  - "C1216244597-ASF"
+  - "C1216244596-ASF"
+  - "C1216244588-ASF"
+  - "C1216244599-ASF"
+  - "C1257024016-ASF"
+  - "C1225776655-ASF"
+  - "C1225776657-ASF"
+  - "C1225776658-ASF"
+  - "C1225776659-ASF"
+  - "C1245953394-ASF"
+  - "C1234413263-ASFDEV"
+  - "C1234413247-ASFDEV"
+  - "C1234413248-ASFDEV"
+  - "C1234413255-ASFDEV"
+  - "C1234413254-ASFDEV"
+  - "C1234413256-ASFDEV"
+  - "C1234413253-ASFDEV"
+  - "C1234413252-ASFDEV"
+  - "C1234413250-ASFDEV"
+  - "C1234413259-ASFDEV"
+  - "C1234413251-ASFDEV"
+  - "C1234413261-ASFDEV"
+  - "C1234413262-ASFDEV"
+  - "C1234413258-ASFDEV"
+  - "C1244552887-ASFDEV"
+  - "C1234413246-ASFDEV"
+  - "C1234413249-ASFDEV"
+  - "C1234413257-ASFDEV"
+  - "C1234413260-ASFDEV"
+  - "C1257175154-ASFDEV"
+  - "C1244598379-ASFDEV"
+
 tests:
   - test-ASFSearch ALOS Resp 1:
       response: Alos_response_maxResults3.yml
@@ -55,176 +226,8 @@ tests:
           - SENTINEL-1A
           - SENTINEL-1B
       expected:
-        - collections:
-            - C1214470488-ASF
-            - C1214470533-ASF
-            - C1214470576-ASF
-            - C1595422627-ASF
-            - C1214470496-ASF
-            - C1214470532-ASF
-            - C1214472977-ASF
-            - C1214472336-ASF
-            - C1266376001-ASF
-            - C1214472994-ASF
-            - C1214470732-ASF
-            - C1214473170-ASF
-            - C1214470561-ASF
-            - C1214471521-ASF
-            - C1214470682-ASF
-            - C2777443834-ASF
-            - C2777436413-ASF
-            - C1214471197-ASF
-            - C1214471960-ASF
-            - C1214472978-ASF
-            - C1214473165-ASF
-            - C2709161906-ASF
-            - C1596065640-ASF
-            - C1596065639-ASF
-            - C1596065641-ASF
-            - C1595765183-ASF
-            - C2450786986-ASF
-            - C1205428742-ASF
-            - C1212201032-ASF
-            - C1212212560-ASF
-            - C1205264459-ASF
-            - C1208117434-ASF
-            - C1212209035-ASF
-            - C1212209226-ASF
-            - C1208115009-ASF
-            - C1212158327-ASF
-            - C1215704763-ASF
-            - C1225776654-ASF
-            - C1212158318-ASF
-            - C1212212493-ASF
-            - C1212158326-ASF
-            - C1212233976-ASF
-            - C1260726384-ASF
-            - C1258354200-ASF
-            - C1259982010-ASF
-            - C1260721945-ASF
-            - C1257995185-ASF
-            - C1259976861-ASF
-            - C1260726378-ASF
-            - C1258354201-ASF
-            - C1259981910-ASF
-            - C1260721853-ASF
-            - C1257995186-ASF
-            - C1259974840-ASF
-            - C1212200781-ASF
-            - C1212209075-ASF
-            - C1257024016-ASF
-            - C1225776655-ASF
-            - C1225776657-ASF
-            - C1225776658-ASF
-            - C1225776659-ASF
-            - C1245953394-ASF
-            - C1234413245-ASFDEV
-            - C1234413229-ASFDEV
-            - C1234413237-ASFDEV
-            - C1234413238-ASFDEV
-            - C1234413236-ASFDEV
-            - C1234413230-ASFDEV
-            - C1234413232-ASFDEV
-            - C1234413235-ASFDEV
-            - C1234413240-ASFDEV
-            - C1234413234-ASFDEV
-            - C1234413241-ASFDEV
-            - C1234413233-ASFDEV
-            - C1234413243-ASFDEV
-            - C1234413244-ASFDEV
-            - C1244552887-ASFDEV
-            - C1234413228-ASFDEV
-            - C1234413231-ASFDEV
-            - C1234413239-ASFDEV
-            - C1234413242-ASFDEV
-            - C1257175154-ASFDEV
-            - C1244598379-ASFDEV
-            - C1240784657-ASFDEV
-            - C1327985661-ASF
-            - C1327985645-ASF
-            - C1595422627-ASF
-            - C1327985617-ASF
-            - C1327985660-ASF
-            - C1327985741-ASF
-            - C1327985578-ASF
-            - C1327985646-ASF
-            - C1327985650-ASF
-            - C1327985579-ASF
-            - C1327985740-ASF
-            - C1327985619-ASF
-            - C1327985739-ASF
-            - C1327985647-ASF
-            - C2777443834-ASF
-            - C2777436413-ASF
-            - C1327985697-ASF
-            - C1327985651-ASF
-            - C1327985644-ASF
-            - C1327985571-ASF
-            - C1327985674-ASF
-            - C2709161906-ASF
-            - C1596065640-ASF
-            - C1596065639-ASF
-            - C1596065641-ASF
-            - C1595765183-ASF
-            - C2450786986-ASF
-            - C1216244348-ASF
-            - C1216244589-ASF
-            - C1216244594-ASF
-            - C1216244593-ASF
-            - C1216244585-ASF
-            - C1216244592-ASF
-            - C1216244595-ASF
-            - C1225776654-ASF
-            - C1216244590-ASF
-            - C1216244601-ASF
-            - C1216244600-ASF
-            - C1216244591-ASF
-            - C1216244587-ASF
-            - C1216244598-ASF
-            - C1216244586-ASF
-            - C1260726384-ASF
-            - C1258354200-ASF
-            - C1259982010-ASF
-            - C1260721945-ASF
-            - C1257995185-ASF
-            - C1259976861-ASF
-            - C1260726378-ASF
-            - C1258354201-ASF
-            - C1259981910-ASF
-            - C1260721853-ASF
-            - C1257995186-ASF
-            - C1259974840-ASF
-            - C1216244597-ASF
-            - C1216244596-ASF
-            - C1216244588-ASF
-            - C1216244599-ASF
-            - C1257024016-ASF
-            - C1225776655-ASF
-            - C1225776657-ASF
-            - C1225776658-ASF
-            - C1225776659-ASF
-            - C1245953394-ASF
-            - C1234413263-ASFDEV
-            - C1234413247-ASFDEV
-            - C1234413248-ASFDEV
-            - C1234413255-ASFDEV
-            - C1234413254-ASFDEV
-            - C1234413256-ASFDEV
-            - C1234413253-ASFDEV
-            - C1234413252-ASFDEV
-            - C1234413250-ASFDEV
-            - C1234413259-ASFDEV
-            - C1234413251-ASFDEV
-            - C1234413261-ASFDEV
-            - C1234413262-ASFDEV
-            - C1234413258-ASFDEV
-            - C1244552887-ASFDEV
-            - C1234413246-ASFDEV
-            - C1234413249-ASFDEV
-            - C1234413257-ASFDEV
-            - C1234413260-ASFDEV
-            - C1257175154-ASFDEV
-            - C1244598379-ASFDEV
+        - collections: *sentinel-1_collections
+
   - test-search-build_subquery S1-Platform SLC:
       params:
         platform:
@@ -234,12 +237,12 @@ tests:
           - SLC
       expected:
         - collections:
-            - C1214470488-ASF
-            - C1205428742-ASF
-            - C1234413245-ASFDEV
-            - C1327985661-ASF
-            - C1216244348-ASF
-            - C1234413263-ASFDEV
+          - 'C1205428742-ASF'
+          - 'C1214470488-ASF'
+          - 'C1216244348-ASF'
+          - 'C1234413245-ASFDEV'
+          - 'C1234413263-ASFDEV'
+          - 'C1327985661-ASF'
   - test-search-build_subquery S1-Dataset SLC:
       params:
         dataset:
@@ -309,8 +312,9 @@ tests:
       params:
         processingLevel:
           - UNKNOWN_PROCESSING_TYPE
-      expected:
-        - {}
+      expected: [
+        {processingLevel: [UNKNOWN_PROCESSING_TYPE]}
+      ]
   - test-search-build_subquery UNKNOWN_PROCESSING_TYPE with collections:
       params:
         processingLevel:
@@ -323,16 +327,19 @@ tests:
           - C1216244348-ASF
           - C1234413263-ASFDEV
           - C1212001698-ASF
-      expected:
-        - collections:
-            - C1214470488-ASF
-            - C1205428742-ASF
-            - C1234413245-ASFDEV
-            - C1327985661-ASF
-            - C1216244348-ASF
-            - C1234413263-ASFDEV
-            - C1212001698-ASF
-  - test-search-build_subquery UNKNOWN_PROCESSING_TYPE with platform, SLC, & collection:
+      expected: [
+        {
+        processingLevel: [UNKNOWN_PROCESSING_TYPE],
+        collections: [C1214470488-ASF,
+            C1205428742-ASF,
+            C1234413245-ASFDEV,
+            C1327985661-ASF,
+            C1216244348-ASF,
+            C1234413263-ASFDEV,
+            C1212001698-ASF]
+        }
+      ]
+  - test-search-build_subquery UNKNOWN_PROCESSING_TYPE with platform, SLC, & collection (skip set difference with unknown processing level):
       params:
         platform:
           - SENTINEL-1A
@@ -343,16 +350,239 @@ tests:
           - SLC
         collections:
           - C1212001698-ASF
-      expected:
-        - collections:
-            - C1214470488-ASF
-            - C1205428742-ASF
-            - C1234413245-ASFDEV
-            - C1327985661-ASF
-            - C1216244348-ASF
-            - C1234413263-ASFDEV
-            - C1212001698-ASF
-    
+      expected: [{
+        processingLevel: UNKNOWN_PROCESSING_TYPE,
+        collections: [
+            "C1214353986-ASF",
+            "C1214336045-ASF",
+            "C1214336717-ASF",
+            "C1214335430-ASF",
+            "C1214335471-ASF",
+            "C1214335903-ASF",
+            "C1214336154-ASF",
+            "C1214336554-ASF",
+            "C1214353593-ASF",
+            "C1214353754-ASF",
+            "C1214353859-ASF",
+            "C1214337770-ASF",
+            "C1214354144-ASF",
+            "C1214354235-ASF",
+            "C1214343609-ASF",
+            "C1214354031-ASF",
+            "C1214408428-ASF",
+            "C1214419355-ASF",
+            "C1210487703-ASF",
+            "C1212030772-ASF",
+            "C1206116665-ASF",
+            "C1206132445-ASF",
+            "C1211962154-ASF",
+            "C1212001698-ASF",
+            "C1212005594-ASF",
+            "C1212019993-ASF",
+            "C1207638502-ASF",
+            "C1210025872-ASF",
+            "C1210485039-ASF",
+            "C1207188317-ASF",
+            "C1210546638-ASF",
+            "C1206122195-ASF",
+            "C1209970710-ASF",
+            "C1207038647-ASF",
+            "C1210599503-ASF",
+            "C1210599673-ASF",
+            "C1214470488-ASF",
+            "C1205428742-ASF",
+            "C1234413245-ASFDEV",
+            "C1327985661-ASF",
+            "C1216244348-ASF",
+            "C1234413263-ASFDEV",
+            "C2803501758-ASF", # OPERA
+            "C2803501097-ASF", # OPERA
+            "C1214470488-ASF", # S1A
+            "C1214470533-ASF",
+            "C1214470576-ASF",
+            "C1595422627-ASF",
+            "C1214470496-ASF",
+            "C1214470532-ASF",
+            "C1214472977-ASF",
+            "C1214472336-ASF",
+            "C1266376001-ASF",
+            "C1214472994-ASF",
+            "C1214470732-ASF",
+            "C1214473170-ASF",
+            "C1214470561-ASF",
+            "C1214471521-ASF",
+            "C1214470682-ASF",
+            "C2777443834-ASF",
+            "C2777436413-ASF",
+            "C1214471197-ASF",
+            "C1214471960-ASF",
+            "C1214472978-ASF",
+            "C1214473165-ASF",
+            "C2709161906-ASF",
+            "C1596065640-ASF",
+            "C1596065639-ASF",
+            "C1596065641-ASF",
+            "C1595765183-ASF",
+            "C2450786986-ASF",
+            "C1205428742-ASF",
+            "C1212201032-ASF",
+            "C1212212560-ASF",
+            "C1205264459-ASF",
+            "C1208117434-ASF",
+            "C1212209035-ASF",
+            "C1212209226-ASF",
+            "C1208115009-ASF",
+            "C1212158327-ASF",
+            "C1215704763-ASF",
+            "C1225776654-ASF",
+            "C1212158318-ASF",
+            "C1212212493-ASF",
+            "C1212158326-ASF",
+            "C1212233976-ASF",
+            "C1260726384-ASF",
+            "C1258354200-ASF",
+            "C1259982010-ASF",
+            "C2795135668-ASF",
+            "C1260721945-ASF",
+            "C1257995185-ASF",
+            "C1259976861-ASF",
+            "C1258354201-ASF",
+            "C1259981910-ASF",
+            "C2795135174-ASF",
+            "C1260721853-ASF",
+            "C1257995186-ASF",
+            "C1259974840-ASF",
+            "C1212200781-ASF",
+            "C1212209075-ASF",
+            "C1257024016-ASF",
+            "C1225776655-ASF",
+            "C1225776657-ASF",
+            "C1225776658-ASF",
+            "C1225776659-ASF",
+            "C1245953394-ASF",
+            "C1234413245-ASFDEV",
+            "C1234413229-ASFDEV",
+            "C1234413237-ASFDEV",
+            "C1234413238-ASFDEV",
+            "C1234413236-ASFDEV",
+            "C1234413230-ASFDEV",
+            "C1234413232-ASFDEV",
+            "C1234413235-ASFDEV",
+            "C1234413240-ASFDEV",
+            "C1234413234-ASFDEV",
+            "C1234413241-ASFDEV",
+            "C1234413233-ASFDEV",
+            "C1234413243-ASFDEV",
+            "C1234413244-ASFDEV",
+            "C1244552887-ASFDEV",
+            "C1234413228-ASFDEV",
+            "C1234413231-ASFDEV",
+            "C1234413239-ASFDEV",
+            "C1234413242-ASFDEV",
+            "C1257175154-ASFDEV",
+            "C1244598379-ASFDEV",
+            "C1240784657-ASFDEV",
+            "C1327985661-ASF", # S1B
+            "C1327985645-ASF",
+            "C1595422627-ASF",
+            "C1327985617-ASF",
+            "C1327985660-ASF",
+            "C1327985741-ASF",
+            "C1327985578-ASF",
+            "C1327985646-ASF",
+            "C1327985650-ASF",
+            "C1327985579-ASF",
+            "C1327985740-ASF",
+            "C1327985619-ASF",
+            "C1327985739-ASF",
+            "C1327985647-ASF",
+            "C2777443834-ASF",
+            "C2777436413-ASF",
+            "C1327985697-ASF",
+            "C1327985651-ASF",
+            "C1327985644-ASF",
+            "C1327985571-ASF",
+            "C1327985674-ASF",
+            "C2709161906-ASF",
+            "C1596065640-ASF",
+            "C1596065639-ASF",
+            "C1596065641-ASF",
+            "C1595765183-ASF",
+            "C2450786986-ASF",
+            "C1216244348-ASF",
+            "C1216244589-ASF",
+            "C1216244594-ASF",
+            "C1216244593-ASF",
+            "C1216244585-ASF",
+            "C1216244592-ASF",
+            "C1216244595-ASF",
+            "C1225776654-ASF",
+            "C1216244590-ASF",
+            "C1216244601-ASF",
+            "C1216244600-ASF",
+            "C1216244591-ASF",
+            "C1216244587-ASF",
+            "C1216244598-ASF",
+            "C1216244586-ASF",
+            "C1260726384-ASF",
+            "C1258354200-ASF",
+            "C1259982010-ASF",
+            "C2795135668-ASF",
+            "C1260721945-ASF",
+            "C1257995185-ASF",
+            "C1259976861-ASF",
+            "C1258354201-ASF",
+            "C1259981910-ASF",
+            "C2795135174-ASF",
+            "C1260721853-ASF",
+            "C1257995186-ASF",
+            "C1259974840-ASF",
+            "C1216244597-ASF",
+            "C1216244596-ASF",
+            "C1216244588-ASF",
+            "C1216244599-ASF",
+            "C1257024016-ASF",
+            "C1225776655-ASF",
+            "C1225776657-ASF",
+            "C1225776658-ASF",
+            "C1225776659-ASF",
+            "C1245953394-ASF",
+            "C1234413263-ASFDEV",
+            "C1234413247-ASFDEV",
+            "C1234413248-ASFDEV",
+            "C1234413255-ASFDEV",
+            "C1234413254-ASFDEV",
+            "C1234413256-ASFDEV",
+            "C1234413253-ASFDEV",
+            "C1234413252-ASFDEV",
+            "C1234413250-ASFDEV",
+            "C1234413259-ASFDEV",
+            "C1234413251-ASFDEV",
+            "C1234413261-ASFDEV",
+            "C1234413262-ASFDEV",
+            "C1234413258-ASFDEV",
+            "C1244552887-ASFDEV",
+            "C1234413246-ASFDEV",
+            "C1234413249-ASFDEV",
+            "C1234413257-ASFDEV",
+            "C1234413260-ASFDEV",
+            "C1257175154-ASFDEV",
+            "C1244598379-ASFDEV",
+          ]
+        }]
+  - test-search-build_subquery configuration params:
+      params:
+        host: 'cmr.uat.earthdata.nasa.gov'
+        provider: 'ASFDEV'
+        product_list: ['S1_069649_IW1_20151201T232154_VV_73D6-BURST']
+      expected: [
+        {
+          host: 'cmr.uat.earthdata.nasa.gov',
+          provider: 'ASFDEV',
+          product_list: ['S1_069649_IW1_20151201T232154_VV_73D6-BURST']
+        }
+      ]
+
   - test-aliasing-search-against-api SLC:
       params:
         processingLevel: SLC
@@ -362,6 +592,29 @@ tests:
         platform: SENTINEL-1A
         processingLevel: SLC
 
+  - test-aliasing-search-against-api S1A RTC:
+      params:
+        platform: SENTINEL-1A
+        processingLevel: RTC
+
+  - test-aliasing-search-against-api S1A CSLC:
+      params:
+        platform: SENTINEL-1A
+        processingLevel: CSLC
+
+  - test-aliasing-search-against-api RTC-STATIC:
+      params:
+        processingLevel: RTC-STATIC
+
+  - test-aliasing-search-against-api CSLC-STATIC:
+      params:
+        processingLevel: CSLC-STATIC
+
+  - test-aliasing-search-against-api OERA-S1 RTC:
+      params:
+        dataset: OPERA-S1
+        processingLevel: RTC
+
   - test-ASFSearch OPERA Resp 1:
       response: OPERA_Products.yml
 
@@ -371,10 +624,6 @@ tests:
   - test-ASFSearch RADARSAT Resp 1:
       response: RADARSAT_stack.yml
 
-  - test-ASFSearch-search-valid Alos:
-      parameters: *alos_parameters
-      answer: Alos_response_maxResults3.yml
-
   - test-aliasing-search-against-api S1B SLC:
       params:
         platform: SENTINEL-1B
@@ -467,4 +716,4 @@ tests:
   - test-aliasing-search-against-api UAVSAR AMPLITUDE:
       params:
         platform: UAVSAR
-        processingLevel: AMPLITUDE
\ No newline at end of file
+        processingLevel: AMPLITUDE



View it on GitLab: https://salsa.debian.org/debian-gis-team/asf-search/-/commit/08c80550d289aecfa00b866d67ec58d5be4bf048

-- 
View it on GitLab: https://salsa.debian.org/debian-gis-team/asf-search/-/commit/08c80550d289aecfa00b866d67ec58d5be4bf048
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20240203/8678541d/attachment-0001.htm>


More information about the Pkg-grass-devel mailing list