[DO NOT MERGE] canary PR

[williamFalcon]

Non changes shouldn't fail CI... (noticed this when merging #1291)

This implies that:

• CI has bugs
• or it's flaky
• also it's super slow to do certain installs...

cc @rasbt @carmocca @Andrei-Aksionov @lantiga

[rasbt]

This is perhaps related to this that happened last Fri:

#1272 (comment)

I might be wrong, but it looks like there is a missing pre-build wheel for Python 3.9.
So whenever you try to install this package with this Python, it tries to build a wheel. Since it's written in Rust, it expects a corresponding toolset.
Don't know why before there were no issues, maybe somewhat from HF team accidentally deleted the wheel? 🤷

It looks like hf-transfer==0.1.4 (Nov 6, 2023) can be installed in Python 3.9.19 without problem

[Andrei-Aksionov]

There are two issues:

1. Starting last week, UV (a replacement for pip install written in Rust) decided that it no longer wants to cooperate with Python 3.9. It cannot find a suitable version of botocore/boto3 (litdata dependency), it checks all the combination starting from the newest package (1.34.x) down to the oldest one (1.10.x), but that doesn't always happen due to timeout. Plus it says that cannot find an crt extension for boto3. While pip install doesn't have any of these issues. This should be solved by Replace UV with regular pip #1292
2. There is an encoding problem on Windows platform and the newest addition to supported models - CodeGemma. It should be solved by Specify UTF-8 encoding for every open function #1283

[williamFalcon]

@Andrei-Aksionov merged #1292 into this PR. However, the tests are still failing.

[Andrei-Aksionov]

Yep, as I stated above: there are 2 issues 😆.
Now it's time to merge #1283 into main and then merge main into this branch.

[williamFalcon]

nice. love it. nice job @Andrei-Aksionov @carmocca @rasbt @lantiga