2023.11

[javierggt]

ska3-matlab 2023.11

The following changes should cover all changes that were included in ska3-flight 2023.10 and some from ska3-flight 2023.9.

This PR includes:

• chandra_aca:
  • Add new functions to get Earth boresight angles and Earth block intervals
  • Improve accuracy and speed of planet functions
  • New fid offset code to use aimpoint drift model to estimate fid positions on CCD
• kadi
  • Improve Command and Command not run events
    • now correctly supports cut-and-paste from Backstop text with no text edits required
  • Add support of "HRC not run" event type to handle HRC being disabled
  • Change limits in SPM and off-nom-roll validators to fix violations
• proseco.
  • Apply temperature-dependent drift model offset to fid positions
• ska_sun
  • change default sun position method to new accurate method.
• starcheck:
  • Update High IR Zone check to -25 +27 around perigee.
• performance improvements in chandra_aca, chandra_maneuver, kadi and quaternion

This version of ska3-matlab will use the latest ska3-core 2023.10.

Interface Impacts:

• 12 packages removed. These packages are handled under our non-FSDS package process.
• proseco:
  • Change default behavior to apply offsets in y and z to expected fid positions via chandra_aca.drift.get_fid_offset. Respects new environment variable PROSECO_ENABLE_FID_OFFSET .
• kadi:
  • Format for Command and Command not run event Params changed to match the existing documentation (see link in the Sheet).
  • Supports new HRC not run entry on Events Sheet.
• chandra_cmd_states package discontinued. Importing it raises an exception unless SKA_ALLOW_DISCONTINUED_PACKAGES=1
• new functions
  • chandra_aca.get_earth_boresight_angle
  • chandra_aca.get_earth_blocks
  • paths functions / classes in parse_cm top level
  • add Quaternion.QuatDescr and Quaternion.QuatLike
  • Add cxotime.CxoTimeDescr, cxotime.CxoTimeLike and cxotime.CxoTime.NOW
• ska_sun.position_at_jd was renamed to ska_sun.position_fast_at_jd to clarify that this function does not respect the sun_position_method_default configuration value.

Testing:

• GRETA.

The latest release candidates will be installed in /proj/sot/ska3/matlab/test on GRETA,
and all release candidates will be available for testing from the usual channels:

conda create -n ska3-matlab-2023.11rc1 --override-channels \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/flight \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/test \
  ska3-matlab==2023.11rc1

Review

All operations critical or impacting PR's are independently and carefully reviewed. For other PR's the level of detail for review is calibrated to operations criticality. Some PR's that are confined to aspect-team-specific processing may have little to no independent review.

Deployment

ska3-matlab 2023.11 will be promoted to flight conda channel and installed on GRETA Linux after approval from FOT team.

Code changes

ska3-matlab changes (2023.8 -> 2023.11rc1)

Removed Packages

• skare3_tools

Updated Packages

• aca_view: 0.13.0 -> 0.14.0 (0.13.0 -> 0.14.0)
  • PR 172 (Javier Gonzalez): add kadi_commands_scenario setting to data service
• acis_thermal_check: 4.7.0 -> 4.8.0 (4.7.0 -> 4.8.0)
  • PR 68 (John ZuHone): Change namespace package imports and update answers after updated sun position calculation
  • PR 66 (John ZuHone): Fixing ruff and black linting
• agasc: 4.17.0 -> 4.18.0 (4.17.0 -> 4.17.1 -> 4.18.0)
  • PR 160 (Tom Aldcroft): Apply ruff check and format redux
  • PR 158 (Tom Aldcroft): Update with black formatting and ruff linting
  • PR 162 (Javier Gonzalez): fix typo in filename in test_agasc_2.py::test_write
  • PR 161 (Javier Gonzalez): move some functions from create_derived_agasc_h5.py
• chandra_aca: 4.42.0 -> 4.45.0 (4.42.0 -> 4.43.0 -> 4.44.0 -> 4.45.0)
  • PR 138 (Tom Aldcroft): Performance improvements for planets
  • PR 160 (Jean Connelly): Support fid drift times before 2012:001
  • PR 157 (Jean Connelly): Add fid offset support to drift
  • PR 158 (Tom Aldcroft): Update docstring in drift.py
  • PR 161 (Tom Aldcroft): Add tolerance args to clip_and_warn
  • PR 164 (Tom Aldcroft): Add new functions to get Earth boresight angles and Earth block intervals
  • PR 159 (Javier Gonzalez): Make maude_decom.get_aca_images robust agains missing/repeated frames
  • PR 163 (Tom Aldcroft): Allow None input to convert_time_format_spk
  • PR 162 (Tom Aldcroft): Improve accuracy and speed of planet functions
• chandra_cmd_states: 4.0.0 -> 4.1.0 (4.0.0 -> 4.1.0)
  • PR 68 (Tom Aldcroft): Discontinue support for chandra_cmd_states
• chandra_maneuver: 4.0.0 -> 4.2.0 (4.0.0 -> 4.1.0 -> 4.2.0)
  • PR 26 (Tom Aldcroft): Enforce quaternion sign convention
  • PR 24 (Tom Aldcroft): Performance improvements
  • PR 27 (Tom Aldcroft): Use efficient ska_sun pitch and off_nom_roll methods
• cheta: 4.60.0 -> 4.61.0 (4.60.0 -> 4.61.0)
  • PR 254 (Jean Connelly): Update computed pitch expected vals for a regression test
  • PR 253 (Tom Aldcroft): Add a negative lookahead assertion for log check
  • PR 252 (Tom Aldcroft): Ruff format and lint and tidy top-level
• cxotime: 3.6.1 -> 3.8.0 (3.6.1 -> 3.7.0 -> 3.8.0)
  • PR 42 (Tom Aldcroft): Fix a precision issue in convert_time_format
  • PR 40 (Tom Aldcroft): Short circuit convert_time_format for a CxoTime object
  • PR 43 (Tom Aldcroft): Add CxoTimeDescr descriptor and CxoTime.NOW sentinel
  • PR 41 (Tom Aldcroft): Ruff format and linting
• fot-matlab: 2.3.0 -> 2.4.0 (2.3.0 -> 2.4.0)
  • PR 26 (James Kristoff): Support parsing YAML comments from file or as raw text
  • PR 25 (James Kristoff): Add support for parsing YAML text from OR List Comments
• kadi: 7.6.0 -> 7.8.1 (7.6.0 -> 7.7.0 -> 7.7.1 -> 7.8.0 -> 7.8.1)
  • PR 266 (Tom Aldcroft): Performance improvement kadi states
  • PR 268 (Tom Aldcroft): Use ruff for linting and formatting
  • PR 296 (Jean Connelly): Use ska_helpers.utils.convert_to_int_float_str to stop Deprecation warns
  • PR 294 (Jean Connelly): Update log statement with a time that prints
  • PR 303 (Jean Connelly): Broaden setup.py package data glob to install pkl.gz and ecsv.gz test data
  • PR 300 (Tom Aldcroft): Add unit and regression tests for validation
  • PR 299 (Tom Aldcroft): Revert the change to decorate Validate.update_tlm method
  • PR 298 (Tom Aldcroft): Convert docstrings to numpydoc style and refactor API docs
  • PR 308 (Tom Aldcroft): Improve Command and Command not run events
  • PR 309 (Tom Aldcroft): Add "HRC not run" event to handle HRC being disabled
  • PR 307 (Tom Aldcroft): Fix, document, and test get_rltt and get_schedule_stop_time
  • PR 302 (Tom Aldcroft): Add email notifications for validation violations
  • PR 305 (Tom Aldcroft): Change limits in SPM and off-nom-roll validators to fix violations
  • PR 304 (Tom Aldcroft): Update code and tests for use with ska_sun accurate position
  • PR 301 (Tom Aldcroft): flatten namespace package names
  • PR 295 (Tom Aldcroft): Use cached property in commands validation
  • PR 311 (Javier Gonzalez): Skip tests if internet is not available
• mica: 4.33.2 -> 4.34.1 (4.33.2 -> 4.34.0 -> 4.34.1)
  • PR 286 (Jean Connelly): Handle no-starcat observations better in reports
  • PR 287 (Jean Connelly): Update test colnames for DS 10.12.3 outputs
• parse_cm: 3.13.0 -> 3.14.1 (3.13.0 -> 3.14.0 -> 3.14.1)
  • PR 46 (Tom Aldcroft): Sort file paths in load_file_paths function
  • PR 45 (Tom Aldcroft): Add functions related to load file names and paths
  • PR 47 (Javier Gonzalez): skip path tests if load directory is not there
• proseco: 5.11.0 -> 5.12.0 (5.11.0 -> 5.12.0)
  • PR 390 (Tom Aldcroft): Add Row type aliases
  • PR 389 (Jean Connelly): Rename a test so error isn't in the name
  • PR 388 (Jean Connelly): Apply temperature-dependent drift model offset to fid positions
• quaternion: 4.1.1 -> 4.3.0 (4.1.1 -> 4.2.0 -> 4.3.0)
  • PR 42 (Tom Aldcroft): Apply ruff format (black) and linting
  • PR 41 (Tom Aldcroft): Add functions for improved performance for scalar operations
  • PR 43 (Tom Aldcroft): Use astropy.utils.shapes instead of copied version
  • PR 45 (Tom Aldcroft): Update QuatDescriptor for typed-descriptor-lazy-default
  • PR 44 (Tom Aldcroft): Add QuatDescriptor descriptor and QuatLike type alias
• ska3-core: 2023.3 -> 2023.10
• ska_helpers: 0.12.0 -> 0.14.0 (0.12.0 -> 0.13.0 -> 0.14.0)
  • PR 50 (Jean Connelly): Rename test so error isn't in the name
  • PR 54 (Tom Aldcroft): Coerce TypedDescriptor default at instance creation
  • PR 53 (Tom Aldcroft): Add TypedDescriptor class for typed descriptors
  • PR 52 (Tom Aldcroft): Suppress the setuptools_scm warning
• ska_sun: 3.11.0 -> 3.13.0 (3.11.0 -> 3.12.0 -> 3.13.0)
  • PR 35 (Tom Aldcroft): Add position_at_jd to all
  • PR 36 (Tom Aldcroft): Change default sun position method to accurate and other minor improvements
• sparkles: 4.24.0 -> 4.25.0 (4.24.0 -> 4.24.1 -> 4.25.0)
  • PR 200 (Tom Aldcroft): Ruff changes
  • PR 203 (Tom Aldcroft): Factor checks out of ACAReviewTable class
  • PR 202 (Jean Connelly): Update test data with proseco fid offsets
  • PR 201 (Jean Connelly): Update tests for ska_sun change to position_accurate
• starcheck: 14.5.0 -> 14.7.0 (14.5.0 -> 14.6.0 -> 14.7.0)
  • PR 433 (Jean Connelly): Update High IR Zone check to -25 +27 around perigee
  • PR 432 (Jean Connelly): Add up NMM time before NPNT states to calculate maneuver equiv angle
  • PR 431 (Jean Connelly): Add fid offset support
• xija: 4.31.0 -> 4.31.1 (4.31.0 -> 4.31.1)
  • PR 135 (Tom Aldcroft): Use cheta not Ska.engarchive

ska3-core (2023.3 -> 2023.10rc4)

New Packages

• astropy-iers-data: 0.2024.1.8.0.30.55

Related Issues

Fixes #1201
Fixes #1247