chore: Allow numpy >= 2.0.0

[booxter]

A large e2e run with forced numpy 2.0+ suggests that the library is compatible with numpy 2.x series.

Also, removed numpy dependency from requirements-dev.txt because it's already listed in requirements.txt.

See: https://github.com/instructlab/training/actions/runs/14539292904

Signed-off-by: Ihar Hrachyshka ihar.hrachyshka@gmail.com

[github-actions]

E2E (NVIDIA L40S x4) workflow launched on this PR: View run

[github-actions]

e2e workflow succeeded on this PR: View run, congrats!

[RobotSail]

@booxter We use numpy in a few critical points like the collator function in the dataloader as well as the data processing logic when converting from messages into raw input IDs for the model to consume. This is a PR where we'd need to run training and validate that the loss curve from your changes produces the same loss curve as what's in the main branch.

[booxter]

Fair enough. We'll need to automate validation for patches like this one (also refactors etc.) I don't feel we should continue with human assessments (or at the very least artifacts - charts etc. - needed for such assessment should be generated and posted in comments.) I will mark this as a draft and report an issue to implement the necessary automation.

[RobotSail]

@booxter Yes, we have a script to do this already but I think it broke and I haven't had time to look into it. If you can fix it then that would be a huge contribution which would automate most of the testing we do manually today. You can find it here: https://github.com/instructlab/training/blob/main/scripts/create-loss-graph.py

And then the actual CI code to call it can be found here:

training/.github/workflows/e2e-nvidia-l40s-x4.yml

Line 387 in 65056fa

python training/scripts/create-loss-graph.py \

[booxter]

@Mergifyio rebase

[mergify]

rebase

✅ Branch has been successfully rebased