LRU embedding net requires very new torch version

[michaeldeistler]

🐛 Bug Description

Our test suite fail for torch v2.4.0, because the LRU requires at least v2.5.0. I am not sure how to best handle this, as I would prefer to not pin torch to such a new version. Together with #1380, this would make torch=2.5.0 the only compatible version.

To reproduce:

pip install torch==2.4.0
pytest tests/test_embedding_nets.py

The error is

FAILED tests/embedding_net_test.py::test_lru_isolated[scan-one-directional] - TypeError: associative_scan() got an unexpected keyword argument 'combine_mode'

(and many other LRU tests with the same error message)

Tagging @famura @Matthijspals FYI.

[famura]

One could add a pytest.mark.skipif which is true if the torch version is not >= 2.5 and a user warning or error when using the scan mode for the LRUEmbedding. the other mode should work as expected. I can't promise to be able to do this until next week.

Please excuse the brevity, I'm on the phone.

[Matthijspals]

Some context: the LRU recurrency can be run in two modes, either using a for loop (which is the default), or using associative / parallel scans (which are potentially much faster for long time series as it scales log(sequence length)). However even with the for loop mode, the LRUs should be very useful to embed time-series (and probably still a lot faster than many other RNNs).

The associative scan function is still in development in PyTorch - in fact gradients are not supported (but will be very soon, potentially merged in the coming days).

I think @famura's suggestion makes sense for now!

[janfb]

If we know the type of Error that will be raised it makes sense to use a

marks=pytest.mark.xfail(reason="LRU not supported ...", condition=torch.__version__ >= "2.5.0", strict=True

so that we will get an XFAIL once this is implemented.