Dependency Pinning Missing, Compatibility Issues with Newer Python Versions, and Unhelpful Results

[faberto]

I've encountered several critical issues while attempting to use this project, which have significantly hindered my ability to effectively leverage its capabilities. These issues pertain to the lack of dependency pinning, compatibility problems with newer versions of Python, and the overall usefulness of the results produced. Below is a detailed account of each problem and its impact on my experience.

1. Missing Dependency Pinning

It took a while to finde the working dependencies. env.yml.txt

2. Incompatibility with Newer Python Versions

New versions just break it enirely.

3. Missing sample data

The project is missing sample data to verify that it works as expected.

4. Unhelpful Results

Testing muliple datasets the results were always 0 or a few minutes of sleep. As I cannot upload our samples the same happens with sample data from another issue:
#33 (comment)
Output:

Saving files to dir: outputs/sample
Raw data file already exists. Skipping raw data parsing.
   Unnamed: 0                           time         x         y         z  non_wear
0           0  2023-08-15 15:00:00.017000000 -0.461000  0.894996 -0.116996     False
1           1  2023-08-15 15:00:00.050333333 -0.443447  0.781798 -0.121022     False
2           2  2023-08-15 15:00:00.083666666 -0.527073  0.767216 -0.148742     False
3           3  2023-08-15 15:00:00.117000000 -0.605458  0.903918 -0.139534     False
4           4  2023-08-15 15:00:00.150333333 -0.689671  1.284508 -0.111031     False
Data2model file already exists. Skip data transformation.
data2model shape: (8640, 3, 900)
times shape: (8640,)
Non_wear flag shape: (8640,)
prediction set sample count: 8640
Using local /home/tobi/asleep/src/asleep/torch_hub_cache/OxWearables_ssl-wearables_v1.0.0
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [01:38<00:00, 19.62s/it]
(8640,)
(array([0., 1., 2.]), array([5807,  451, 2382]))
                     time  label  is_wear
0 2023-08-15 15:00:00.017      0     True
1 2023-08-15 15:00:30.017      0     True
2 2023-08-15 15:01:00.017      0     True
3 2023-08-15 15:01:30.017      0     True
4 2023-08-15 15:02:00.017      0     True
No sleep windows >30 mins detected. Exiting...
Non-wear time has been written to outputs/sample/non_wear.npy
Current sleep classification has been written to outputs/sample/ssl_sleep.npy

Any thoughts @angerhang

[angerhang]

I've encountered several critical issues while attempting to use this project, which have significantly hindered my ability to effectively leverage its capabilities. These issues pertain to the lack of dependency pinning, compatibility problems with newer versions of Python, and the overall usefulness of the results produced. Below is a detailed account of each problem and its impact on my experience.

1. Missing Dependency Pinning

It took a while to finde the working dependencies. env.yml.txt

2. Incompatibility with Newer Python Versions

New versions just break it enirely.

3. Missing sample data

The project is missing sample data to verify that it works as expected.

4. Unhelpful Results

Testing muliple datasets the results were always 0 or a few minutes of sleep. As I cannot upload our samples the same happens with sample data from another issue: #33 (comment) Output:

Saving files to dir: outputs/sample
Raw data file already exists. Skipping raw data parsing.
   Unnamed: 0                           time         x         y         z  non_wear
0           0  2023-08-15 15:00:00.017000000 -0.461000  0.894996 -0.116996     False
1           1  2023-08-15 15:00:00.050333333 -0.443447  0.781798 -0.121022     False
2           2  2023-08-15 15:00:00.083666666 -0.527073  0.767216 -0.148742     False
3           3  2023-08-15 15:00:00.117000000 -0.605458  0.903918 -0.139534     False
4           4  2023-08-15 15:00:00.150333333 -0.689671  1.284508 -0.111031     False
Data2model file already exists. Skip data transformation.
data2model shape: (8640, 3, 900)
times shape: (8640,)
Non_wear flag shape: (8640,)
prediction set sample count: 8640
Using local /home/tobi/asleep/src/asleep/torch_hub_cache/OxWearables_ssl-wearables_v1.0.0
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [01:38<00:00, 19.62s/it]
(8640,)
(array([0., 1., 2.]), array([5807,  451, 2382]))
                     time  label  is_wear
0 2023-08-15 15:00:00.017      0     True
1 2023-08-15 15:00:30.017      0     True
2 2023-08-15 15:01:00.017      0     True
3 2023-08-15 15:01:30.017      0     True
4 2023-08-15 15:02:00.017      0     True
No sleep windows >30 mins detected. Exiting...
Non-wear time has been written to outputs/sample/non_wear.npy
Current sleep classification has been written to outputs/sample/ssl_sleep.npy

Any thoughts @angerhang

@faberto hey thanks a lot for reporting these issues.

We are very tided up at the moment so won't be able to have a fix on the dependencies, python compatibility soon.

However, we are aware of the need for better error handling, which I will try to look at tomorrow with a bunch of fixes from our side. No further promises made.

If you can make some of the fixes, really happy to review those PRs :D

[stuartgo]

Any updates here? I am running into a similar issue where the library predicts close to no sleep.

[angerhang]

One work around is to fix the Python package to 3.8.

There is no plan to support news Python package.

Sample test files can be downloaded from any public repos with supported data format: such as CAPTURE24 and the Newcastle data https://zenodo.org/records/1160410