Schema 2.0 Question

[ssesselmann]

Matthias, I have some issues with schema v2.0 as follows.

1. Saving all your spectra in one file like this increases the risk of file corruption which could result in the user loosing more than one file.

2. Larger files will take longer to open and close, especially when running Dash Plotly to refresh the screen

3. How to handle 3D spectra using this scheme, as spectrum names need to be unique ?

Steven

[NuclearPhoenixx]

Hi Steven,

thanks for your questions!

1. Yes, but it should be the user's decision if they want to store multiple in one file or just generate unique files for every spectrum. Ideally, software can allow user's this choice. However, you can also obviously opt in to hide such a function from the user and generate single spectrum files only. The schema is not requiring you to do that, it's just an option if you want.
2. Honestly, I'm a bit confused about this point. You can only show one spectrum at a time and if a user opens a file with multiple spectra in it, you can show a spectrum picker dialogue and let them choose one of the stored spectra to show. This doesn't change anything for Plotly. The added time it takes for the program to analyse the file in terms of how many spectra are stored is completely negligible, except maybe if you have ridiculously huge files.
3. No, they don't have to be unique, not sure where you read that. The only thing that is required for every entry in the data array is the resultData array with an energySpectrum or backgroundEnergySpectrum, just like with the 1.0 version of the schema. There is no unique identifier. If you want to create a unique identifier for each spectrum (should you really need it for whatever reason), you can use the deviceData or sampleInfo entries and/or create additional properties in there. That's all according to the schema :)
   https://opengammaproject.github.io/NPES-JSON/#data_items_sampleInfo_additionalProperties and https://opengammaproject.github.io/NPES-JSON/#data_items_deviceData_additionalProperties

Hope that clears up some of your concerns. Please let me know if I should update the documentation on some things if they were unclear.

[ssesselmann]

Thanks for reply.

If the schema is backwards compatible, I don't see any real need or benefit to include it into my program. With Impulse you can choose any previously saved spectrum file as a comparison spectrum.

The only use case I can see is if the user wants to open a file recorded with Becqmoni in an 2.0 Schema.

My main concern is to have a protocol for 3D files. I am currently doing this as a list of lists in spectrum.

"spectrum":[[0,1,2,3,4,5],[0,1,2,3,4,5],[0,1,2,3,4,5]]

It works quite well, but does not explicitly state the interval time.

Steven

[NuclearPhoenixx]

If the schema is backwards compatible, I don't see any real need or benefit to include it into my program. With Impulse you can choose any previously saved spectrum file as a comparison spectrum.

It's not fully backwards compatible unfortunately. The fact is that it now requires an array around the data packages to accomodate an arbitrary number of entries and this does not work with the 1.0 schema. The 1.0 schema was 1 spectrum only.

The only use case I can see is if the user wants to open a file recorded with Becqmoni in an 2.0 Schema.

Well, yes, interoperability is one of the main points of the schema 😄

The new version is designed in a way that it requires little change to existing code if you only want to save single spectra to a file with the added option of allowing more.

My main concern is to have a protocol for 3D files. I am currently doing this as a list of lists in spectrum.

"spectrum":[[0,1,2,3,4,5],[0,1,2,3,4,5],[0,1,2,3,4,5]]

It works quite well, but does not explicitly state the interval time.

I see. Well, can the user access these files? If they're only some kind of internal data format, it really doesn't matter what you're using and you can keep doing whatever. But if the files are exposed to the users it would be better to avoid confusion and stick to the schema anyways. Because these files won't import anywhere else if you export them like this.

I can't give you any suggestions on how to implement this correctly within impulse, because I don't know what kind of data you (need to) save to the files and what other requirements the program has. What's keeping you from generating a new entry in the data array for each sub-spectrum of your 3D spectrum? Each of these only requires the spectrum itself and the number of bins, nothing else (no interval time or something).

There really aren't many required properties to the schema, please have a look: https://opengammaproject.github.io/NPES-JSON/#data_items_resultData_energySpectrum
Per data package/entry you only really must include the resultData entry which itself requires a energySpectrum and/or backgroundEnergySpectrum. These two only require the numberOfChannels and spectrum inside the single energySpectrum entry.

I.e. the minimal structure for an entry in the data array looks like this:
resultData -> energySpectrum AND/OR backgroundEnergySpectrum -> energySpectrum -> numberOfChannels AND spectrum

If the docs are too confusing, I might add a new kind of flow diagram to make it easier to read. I admit that it can be a bit confusing, but it's the best autogenerator for JSON schema docs I could find.