Provide a function to convert Earthdata Searh URLs to earthaccess options

[jhkennedy]

It's common in many contexts to jump between GUI search and discovery to programmatic search and discovery.

Earthdata Search encodes all the necessary query parameters in the search URL to derive the associated CMR query and, therefore, the associated earthaccess options.

For example, this Earthdata Search:

which I've separated into its parameters here:

https://search.earthdata.nasa.gov/search/granules
    ?p=C2617126679-POCLOUD
    &pg[0][v]=f
    &pg[0][qt]=2024-04-30T00:00:00.000Z,2024-05-05T23:59:59.999Z
    &pg[0][gsk]=-start_date
    &q=OPERA_L3_DSWX-HLS_V1
    &sb[0]=-95.19,30.59,-94.99,30.79
    &tl=1727374247.177!3!!
    &lat=30.689999999999998
    &long=-95.09&zoom=7

Can be directly translated into the associated CMR Query:
https://cmr.earthdata.nasa.gov/search/granules.umm_json?short_name=OPERA_L3_DSWX-HLS_V1&bounding_box=-95.19,30.59,-94.99,30.79&temporal[]=2024-04-30T00:00:00.000Z,2024-05-05T23:59:59.999Z

https://cmr.earthdata.nasa.gov/search/granules.umm_json
    ?short_name=OPERA_L3_DSWX-HLS_V1
    &bounding_box=-95.19,30.59,-94.99,30.79
    &temporal[]=2024-04-30T00:00:00.000Z,2024-05-05T23:59:59.999Z

And the same search in earthaccess would be:

import earthaccess

auth = earthaccess.login()

results = earthaccess.search_data(
    short_name='OPERA_L3_DSWX-HLS_V1',
    bounding_box=[-95.19, 30.59, -94.99, 30.79],
    temporal=('2024-04-30T00:00:00.000Z', '2024-05-05T23:59:59.999Z'),
)

print(len(results))

The "work" here would just be defining a mapping from the abbreviated parameter names used by Earthdata Search into the CMR query parameter names.

I'd love something like this (function names TBD):

import earthacess

url = 'https://search.earthdata.nasa.gov/search/granules?p=C2617126679-POCLOUD&pg[0][v]=f&pg[0][qt]=2024-04-30T00:00:00.000Z,2024-05-05T23:59:59.999Z&pg[0][gsk]=-start_date&q=OPERA_L3_DSWX-HLS_V1_1.0&sb[0]=-95.19,30.59,-94.99,30.79&tl=1727374247.177!3!!&lat=30.689999999999998&long=-95.09&zoom=7'

search_args = earthaccess.translate_earthdata_search_url(url)

results = earthaccess.search_data(**search_args)

And maybe:

import earthacess

search_args = earthaccess.translate_earthdata_search_url(prompt=True)
# paste URL into prompt

results = earthaccess.search_data(**search_args)

Likewise, once we have this mapping, we could also go the other way around and translate search options into CMR or Earthata Search URLs.

[chuckwondo]

I like the idea. I hadn't thought about Earthdata Search URLs, but I was recently thinking about translation from CMR search URLs to python_cmr query instances, so I have a few comments:

• Translation from CMR search URLs to query objects might better be pushed into python_cmr (particularly because of the next point)
• Translation from query objects to CMR search URLs is already available via python_cmr's Query._build_url method, but that method (or some variant, perhaps) should probably be made part of the public API
• Regarding translation from Earthdata Search URLs to CMR search URLs, is there existing Earthdata Search code that we can leverage (if not directly, at least for porting the translation logic)? (I agree that perhaps these URL-to-URL translations belong better in earthaccess rather than python_cmr)

[jhkennedy]

I agree CMR <-> parameter mappings is likely something best kept in python_cmr, although we probably want to expose it some way in earthaccess.

I expect there's code somewhere that does it here:
https://github.com/nasa/earthdata-search

But I didn't find it with a quick poke.

[jhkennedy]

Something to note:

I knew the short name of the dataset (OPERA_L3_DSWX-HLS_V1), so that's the first thing I searched by, by pasting it into the search box. That showed up as the q=OPERA_L3_DSWX-HLS_V1 parameter when I hit enter and it brought up the one matching collection. When I clicked on that collection, the p=C2617126679-POCLOUD parameter gets added to the URL, which is the collection_concept_id for that dataset and what is likely used for the CMR search.

So, my example would likely be translated by Earthdata search to:

CMR URL

https://cmr.earthdata.nasa.gov/search/granules.umm_json
   ?collection_concept_id=C2617126679-POCLOUD
   &bounding_box=-95.19,30.59,-94.99,30.79
   &temporal[]=2024-04-30T00:00:00.000Z,2024-05-05T23:59:59.999Z

earthaccess search:

import earthaccess

auth = earthaccess.login()

results = earthaccess.search_data(
    concept_id='C2617126679-POCLOUD',
    bounding_box=[-95.19, 30.59, -94.99, 30.79],
    temporal=('2024-04-30T00:00:00.000Z', '2024-05-05T23:59:59.999Z'),
)

print(len(results))

---

🤔 As an aside, Python CMR seems to use concept_id for collection_concept_id, which are distinct things on the CMR side.

[chuckwondo]

🤔 As an aside, Python CMR seems to use concept_id for collection_concept_id, which are distinct things on the CMR side.

In the context of a granule query, concept_id equates to collection_concept_id when the value of the ID starts with C. In python_cmr the GranuleQuery class has both a concept_id method (inherited) and a collection_concept_id method. The former can be used to specify either a granule concept ID or a collection concept ID, but given that there is a collection_concept_id method, there's no requirement to use the concept_id method to specify a collection concept ID for a granule query.