array-shape mismatch error if a referenced signal axis is shorter than the signal itself

[rjwolke]

Python version

'os=Windows-10-10.0.19045-SP0'
'numpy=1.26.4'
'asammdf=7.4.2'

Code

MDF version

4.10

Code snippet

m = asammdf.MDF("Short_Referenced_Axis.mf4")
group, index = m.whereis("TVRSPDIVFBDLVIHKIQMVB")[0]
m.get(group=group, index=index)

Traceback

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Work\40_Dev\asammdf\.venv\Lib\site-packages\asammdf\blocks\mdf_v4.py", line 6670, in get
    vals, timestamps, invalidation_bits, encoding = self._get_array(
                                                    ^^^^^^^^^^^^^^^^
  File "C:\Work\40_Dev\asammdf\.venv\Lib\site-packages\asammdf\blocks\mdf_v4.py", line 7226, in _get_array
    vals = fromarrays(arrays, dtype(types))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Work\40_Dev\asammdf\.venv\Lib\site-packages\numpy\core\records.py", line 676, in fromarrays
    raise ValueError(f'array-shape mismatch in array {k} ("{name}")')

Description

Short_Referenced_Axis.zip

The channel "TVRSPDIVFBDLVIHKIQMVB" is a curve signal that is referencing the channel "BOBNKYCWZQJVUAVHLDJHC" as its axis. When the value of TVRSPDIVFBDLVIHKIQMVB is changed during measurement, ETAS INCA will create an entry in the measurement for that channel, but not for the referenced channel.
This causes the channel lengths to differ (the axis has one less entry), ultimately causing the above exception when accessing the signal in most ways (e.g. get, to_dataframe, merge).

The reason for not updating the axis is that mutliple signals may in theory reference that same axis channel, which would then require updating all channels referencing that axis whenever any one of them gets an entry.

Workaround

The problem can be worked around by replacing the axis reference with a reference to an interpolated copy of the axis. But since that will modify the original measurement file, it would be preferable not to do this.

for dep in m.groups[group].channel_dependencies:
    if dep is None:
        continue
    
    for cab in dep:
        if not cab.axis_channels:
            continue
        
        # Get timestamps for the signal group
        timestamps = m.get_master(group)
        
        # Iterate over axis channels
        for i, (depgrp, depidx) in enumerate(cab.axis_channels):
            axis= m.get(group=depgrp, index=depidx)
            # Check if we need to fix the length
            if len(axis.timestamps) < len(timestamps):
                m.append(
                    axis.interp(timestamps, 0, 0), 
                    comment = "", 
                    acq_name = "", 
                    acq_source = axis.source
                )

                # Replace reference with newly added signal group/index
                cab.axis_channels[i] = m.whereis(axis.name)[-1]