Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add standardized/normalized data to catalog #127

Merged
merged 4 commits into from
Jul 27, 2023

Conversation

tdrwenski
Copy link
Member

@tdrwenski tdrwenski commented Jul 26, 2023

This PR would add standardized and normalized versions of one type of satellite data (Cloud top temperature) and GFS Quarter Degree Analysis to the thredds catalog. This is already deployed on thredds-test if you want to see what it looks like. One difference between the preprocessed GFS data and the raw data is that for the preprocessed data we are using a datasetScan because NcML does not (currently) work with featureCollections.

Note that we only have one catalog for both thredds and thredds-test so adding this catalog means we either need to make a distinction between the catalogs of the two servers or else deploy on the thredds server as well.

@dopplershift
Copy link
Member

Are we using threddsDev anywhere? If so, then adding thredds-test back that inherits idd and the main thredds, then adds this catalog would be pretty easy.

@tdrwenski
Copy link
Member Author

tdrwenski commented Jul 26, 2023

Are we using threddsDev anywhere? If so, then adding thredds-test back that inherits idd and the main thredds, then adds this catalog would be pretty easy.

I guess threddsDev is used for the thredds-dev server. Can definitely add thredds-test as a separate folder/top catalog if you prefer that over offering the preprocessed data on both servers upon our next release.

@dopplershift
Copy link
Member

I forgot that thredds-dev still existed.

I don't necessarily have a preference; not sure whether this is something we want to offer in production or not.

@haileyajohnson
Copy link

I don't know if we want to offer this in production (at least yet), but if we add a separate test directory we want to make sure it gets everything that in thredds. It can be a superset of it but it needs to have everything we offer there

@tdrwenski
Copy link
Member Author

tdrwenski commented Jul 26, 2023

As discussed, we will separate the configs for thredds and thredds-test, with the latter having the preprocessed data.

I am not so familiar with this repo, can you check that I have included the correct files in threddsTest? There are some files I see that I am not sure what they are used for. Should I also verify by deploying this that thredds-test still works correctly (if so, can someone help me with this)?

Copy link
Member

@dopplershift dopplershift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few changes to make this better be just a small departure from the production thredds.ucar.edu config. Deployment on thredds-test should be as simple as downloading the newly created zip from the archive server and unzipping to the appropriate spot in /opt/tds-test.

threddsTest/build.info Show resolved Hide resolved
threddsTest/threddsConfig.xml Outdated Show resolved Hide resolved
Copy link
Member

@dopplershift dopplershift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Let's see how it goes on the server...

@dopplershift dopplershift merged commit be06c12 into Unidata:main Jul 27, 2023
2 checks passed
@tdrwenski tdrwenski deleted the add-preprocessed-dataset branch July 27, 2023 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants