Code

Dmoe integration#1210

Code

Open

DayOfThePenguin wants to merge 18 commits intomainEleutherAI/gpt-neox:mainfrom dmoe_integrationEleutherAI/gpt-neox:dmoe_integrationCopy head branch name to clipboard

+1,317-703

Commits

Commits on May 6, 2024

• feat: ensure only the right architectures get built vs all of them for 11.x

  DayOfThePenguin

  committed
  52a2001

  Copy full SHA for 52a2001

• feat: clean up megablocks-based DMoE implementation

  DayOfThePenguin

  committed
  9c66895

  Copy full SHA for 9c66895

• feat: update args, configs, and requirements

  DayOfThePenguin

  committed
  3388c51

  Copy full SHA for 3388c51

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  e987126

  Copy full SHA for e987126

Commits on May 21, 2024

• Merge branch 'main' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  7d6f265

  Copy full SHA for 7d6f265

• Merge branch 'dmoe_integration' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  4aebc2c

  Copy full SHA for 4aebc2c

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  d9f8d55

  Copy full SHA for d9f8d55

• feat: Update readme and example config

  DayOfThePenguin

  committed
  f8c3776

  Copy full SHA for f8c3776

• Merge branch 'dmoe_integration' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  3ef5b66

  Copy full SHA for 3ef5b66

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  33e41d7

  Copy full SHA for 33e41d7

Commits on May 22, 2024

• Merge branch 'main' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  613aeb9

  Copy full SHA for 613aeb9

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  35c7225

  Copy full SHA for 35c7225

Commits on Dec 3, 2024

• Merge branch 'main' into dmoe_integration

  Quentin-Anthony

  committed
  fb68c07

  Copy full SHA for fb68c07

Commits on Dec 10, 2024

• - Updated sinkhorn initialization and add max_iter argument.

  Show description for 542103f
  aurelion-source

  committed
  542103f

  Copy full SHA for 542103f

Commits on Jan 29, 2025

• Merge branch 'main' into dmoe_integration

  Quentin-Anthony

  committed
  7b9679a

  Copy full SHA for 7b9679a

Commits on Jan 31, 2025

• - Updated dmoe config

  Show description for aa7c4bc
  aurelion-source

  committed
  aa7c4bc

  Copy full SHA for aa7c4bc

Commits on Feb 1, 2025

• - Updated transformer_engine requirments

  Show description for 2118a14
  aurelion-source

  committed
  2118a14

  Copy full SHA for 2118a14

• - Disable torch.compile explicitly when using TE

  aurelion-source

  committed
  b51938d

  Copy full SHA for b51938d