Release v4.6.2

Announcing release v4.6.2

Here are the main changes:

Changes

1. Increasing code proportion in the data mix
Activation block: 4_453_709

We are replacing the-stack-dedup with the-stack-v2-dedup in the 14B-star competition and increasing the code proportion in the validation dataset from ~5% to ~15%.

Datasets used during evaluation for the 14B-star competition is as follows:

• HuggingFaceFW/fineweb-edu-score-2 (85%)
• bigcode/the-stack-v2-dedup (15%)

Datasets for the 3B and 14B competitions are left unchanged.

2. New epsilon lower bounds and decay intervals
Activation block 4_453_709

The epsilon decay interval and bounds will be updated for all competitions as follows:

• 3B competition:
  Current: decays from 0.005 to 0.0005 over 7 days
  Updated: decays from 0.005 to 0.0002 over 4 days

• 14B and 14Bstar competitions:
  Current: decays from 0.005 to 0.0005 over 7 days
  Updated: decays from 0.005 to 0.0002 over 5 days

3. Updated emission distribution for competitions
Activation block 4_453_709

• 3B → 20%
• 14B → 40%
• 14B-star → 40%

4. Fixed all package version in requirements.txt
To avoid any installation issues and package compatibilities, we have fixed all dependency package versiosn in the requirements.txt file. The installation experience should be smoother now.

NOTES TO VALIDATORS

-IMPORTANT: The newly added dataset for code the-stack-v2-dedup requires a Hugging Face access token and S3 secret and access keys. You can learn how to obtain and configure those tokens in our validator documentation here.

-Please also make sure to rerun pip install to ensure updated dependencies.
python -m pip install -e