What is the versioning policy?

[timj]

Is there a document somewhere explaining the versioning policy for astroquery.

The 0.4.8 version completely broke compatibility with the Simbad queries as implemented in 0.4.7. At first glance this seems like a patch release that shouldn't break anything so I think at Rubin we may have understood the meaning of versions. Would it be possible in the release notes to indicate more strongly that breaking changes were present for a particular query component? Should we be using exact pins for astroquery in the future?

We did get some deprecation warnings that turned out to be immediately fatal (using flux(U) as a column name) but the resulting table had columns that were completely incompatible with the old version.

cc/ @gpdf

[keflavich]

@timj could you provide more details about the failing queries?

There is a brief policy explanation here:
https://astroquery.readthedocs.io/en/latest/#installation

In short - we do not, and cannot, guarantee backward compatibility at any point because we're subject to the whims of the upstream services: if they change a keyword or do anything different, everything in astroquery necessarily breaks, and so we don't attempt to hold to normal versioning standards.

That said, SIMBAD tends to be very stable and I would not have expected this regression. We did do some cleanup/upgrades/fixing stuff that wasn't working as expected in a few PRs recently, but I wouldn't have expected it to affect your usage.

[bsipocz]

I agree that the version numbers are somewhat unfortunate, practically any tagged version is to be treated as full-fledged feature release as we do not do patch releases (we should have deleted the leading 0 a really long time ago).

That said, Simbad has been fully refactored. I don't think we anticipated breaking changes, and added deprecations at a lot of places, so if you could open an issue with the details it would be useful.

cc @ManonMarchand

[keflavich]

I'll add another informal part of the policy that we haven't discussed, but I'd like to say out loud for the record: if upstream didn't break it, we shouldn't break it, if at all possible.

[bsipocz]

And for the future: We plan to switch to date based releasing, and having much shorter release cycle. As Adam said above, we will never be able to guarantee any past releases to keep working as they may rely on a upstream services that have changed, but we do change the API with care and with deprecations whenever possible.

[timj]

Regarding the breaking changes for the Simbad queries: At a guess the old simbad query system returned RA, DEC etc in all upper case but the TAP service returns everything in all lower case. This does break all the code that is looking for RA etc in the table. RA and DEC were hms/dms strings but in the new version they are decimal degrees.

From the query side flux(U) is no longer supported and requires to be U. It looks like an attempt was made to have a deprecation warning period since I get:

  /Users/timj/work/lsstsw/build/spectractor/lib/python/spectractor/extractor/targets.py:262: DeprecationWarning: The notation 'flux(U)' is deprecated since 0.4.8 in favor of 'U'. See section on filters in https://astroquery.readthedocs.io/en/latest/simbad/simbad_evolution.html to see the new ways to interact with SIMBAD's fluxes.
    simbadQuerier.add_votable_fields('flux(U)', 'flux(B)', 'flux(V)', 'flux(R)', 'flux(I)', 'flux(J)', 'sptype',

but then it fails anyhow:

ValueError: 'flux(j)' is not one of the accepted options which can be listed with 'list_votable_fields'. Did you mean 'flux'?

This feels like an inconsistency in the deprecation handling.

The sptype and z_value fields do get migrated to the new sp_type and rzv_redshift names for the query side and issue a warning saying what the new names should be in the results.

And pm doesn't get a migration warning even though that's meant to be propermotions now.

[timj]

Regarding versions, date-based versions would be a huge improvement since it would then be obvious that semantic versioning is not involved.

[bsipocz]

semantic versioning is not involved.

😄 that's kind of true all over in python land. (I mean the major version is usually indicated by semver minor :) ).

Anyway, the timeline for the switch is soon, the packaging overhaul will be my first task once the paperwork is sorted out for the roses grant.