Code

Rebased ds#38

Code

Closed

dsikka wants to merge 35 commits intomainneuralmagic/vllm:mainfrom rebased-dsneuralmagic/vllm:rebased-dsCopy head branch name to clipboard

+6,268-44

Commits

Commits on Dec 7, 2024

• Add cutlass 2:4 infrastructure

  

  Faraz9877

  authored and

  dsikka

  committed
  ffeb193

  Copy full SHA for ffeb193

• Update with test code

  

  Faraz9877

  authored and

  dsikka

  committed
  3cbd85a

  Copy full SHA for 3cbd85a

• Clean up a bit; both fp8 and int8 working

  

  Faraz9877

  authored and

  dsikka

  committed
  977dbb4

  Copy full SHA for 977dbb4

• Add fp16 and bf16 support to sparse cutlass mm

  

  Faraz9877

  authored and

  dsikka

  committed
  8ac4d5a

  Copy full SHA for 8ac4d5a

• Add multiprocessing for kernel sweep benchmarking

  

  Faraz9877

  authored and

  dsikka

  committed
  d720dd0

  Copy full SHA for d720dd0

• Add multi-GPU

  

  Faraz9877

  authored and

  dsikka

  committed
  969be36

  Copy full SHA for 969be36

• Add cutlass_scaled_sparse_mm op

  

  Faraz9877

  authored and

  dsikka

  committed
  a442cd2

  Copy full SHA for a442cd2

• Clean up

  

  Faraz9877

  authored and

  dsikka

  committed
  e7e6753

  Copy full SHA for e7e6753

• Update code

  

  Faraz9877

  authored and

  dsikka

  committed
  767b454

  Copy full SHA for 767b454

• Update code

  

  Faraz9877

  authored and

  dsikka

  committed
  cc30afe

  Copy full SHA for cc30afe

• Clean up the benchmarking

  

  Faraz9877

  authored and

  dsikka

  committed
  61f0989

  Copy full SHA for 61f0989

• Clean up the cutlass benchmarking

  

  Faraz9877

  authored and

  dsikka

  committed
  de1aa13

  Copy full SHA for de1aa13

• Fix cmake errors

  

  Faraz9877

  authored and

  dsikka

  committed
  69ce94d

  Copy full SHA for 69ce94d

• Fix the cmake TAG

  

  Faraz9877

  authored and

  dsikka

  committed
  b5f1915

  Copy full SHA for b5f1915

• update

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  89229ff

  Copy full SHA for 89229ff

• updated

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  bfec174

  Copy full SHA for bfec174

• updated, calling things properly

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  394eac8

  Copy full SHA for 394eac8

• running end to end but not passing

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  0c125fc

  Copy full SHA for 0c125fc

• updated

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  3824dd8

  Copy full SHA for 3824dd8

• stash

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  6badb0b

  Copy full SHA for 6badb0b

• update

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  cba55ef

  Copy full SHA for cba55ef

• Fix batch size and zeros issue

  

  Faraz9877

  authored and

  dsikka

  committed
  4f6be15

  Copy full SHA for 4f6be15

• working e2e

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  bf0db2c

  Copy full SHA for bf0db2c

• Enable other datatypes

  

  Faraz9877

  authored and

  dsikka

  committed
  c58b5e5

  Copy full SHA for c58b5e5

• Add the heuristics for fp8

  

  Faraz9877

  authored and

  dsikka

  committed
  6f0fb81

  Copy full SHA for 6f0fb81

• updated

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  527f61c

  Copy full SHA for 527f61c

• Cleanup

  

  rahul-tuli

  authored and

  dsikka

  committed
  0d0e2c5

  Copy full SHA for 0d0e2c5

• Move model compressor to scheme

  

  rahul-tuli

  authored and

  dsikka

  committed
  05105da

  Copy full SHA for 05105da

• Cleanup + updates for compressed_tensor changes

  

  rahul-tuli

  authored and

  dsikka

  committed
  19d8a27

  Copy full SHA for 19d8a27

• Add cherry-picked heuristic for Llama3 8B model

  

  Faraz9877

  authored and

  dsikka

  committed
  dd948f8

  Copy full SHA for dd948f8

• updated with latest kernel

  

  robertgshaw2-redhat

  authored and

  dsikka

  committed
  8a91b55

  Copy full SHA for 8a91b55

• remove compressed support; validate against ct models for tp=1,24

  dsikka

  committed
  0129598

  Copy full SHA for 0129598

• add testing cases

  dsikka

  committed
  8cc0205

  Copy full SHA for 8cc0205

• add support for all cases; update tests

  dsikka

  committed
  7355870

  Copy full SHA for 7355870

• fix rebase

  dsikka

  committed
  a794d5a

  Copy full SHA for a794d5a