Code

[DRAFT] use cutlass for 24#33

Code

Draft

rahul-tuli wants to merge 53 commits intomainneuralmagic/vllm:mainfrom rahul-cleanupneuralmagic/vllm:rahul-cleanupCopy head branch name to clipboard

+6,347-615

Commits

Commits on Oct 22, 2024

• Add cutlass 2:4 infrastructure

  Faraz9877

  committed
  5d51361

  Copy full SHA for 5d51361

Commits on Oct 28, 2024

• Update with test code

  Faraz9877

  committed
  17f5b96

  Copy full SHA for 17f5b96

Commits on Oct 30, 2024

• Clean up a bit; both fp8 and int8 working

  Faraz9877

  committed
  471a03c

  Copy full SHA for 471a03c

• Add fp16 and bf16 support to sparse cutlass mm

  Faraz9877

  committed
  0b332fb

  Copy full SHA for 0b332fb

Commits on Nov 5, 2024

• semi_structured for fp16 and bf16 and int8

  ilmarkov

  committed
  da31648

  Copy full SHA for da31648

• Fix A100 int8 tests

  ilmarkov

  committed
  e655f94

  Copy full SHA for e655f94

• Add fp8 cusparseLt

  ilmarkov

  committed
  5fc3c1c

  Copy full SHA for 5fc3c1c

• wip

  ilmarkov

  committed
  9cf36d6

  Copy full SHA for 9cf36d6

• Fix signatures

  ilmarkov

  committed
  ad09e79

  Copy full SHA for ad09e79

• Fix compilation and tests

  ilmarkov

  committed
  e75eabc

  Copy full SHA for e75eabc

• Update for older platforms

  ilmarkov

  committed
  0306390

  Copy full SHA for 0306390

• Add benchmarks

  ilmarkov

  committed
  1021acb

  Copy full SHA for 1021acb

• Fix typo

  ilmarkov

  committed
  19ce358

  Copy full SHA for 19ce358

• Added scaled_mm for fp8.

  Show description for 959408c
  ilmarkov

  committed
  959408c

  Copy full SHA for 959408c

• Add docstrings

  ilmarkov

  committed
  117b87b

  Copy full SHA for 117b87b

• Update for torch 2.5

  ilmarkov

  committed
  2c7e68e

  Copy full SHA for 2c7e68e

• Add handling contiguous dense input for int8 and fp8

  ilmarkov

  committed
  922f4f8

  Copy full SHA for 922f4f8

• Add fp8 cusparseLt

  ilmarkov

  committed
  beca038

  Copy full SHA for beca038

• Fix compilation and tests

  ilmarkov

  committed
  5d9cd25

  Copy full SHA for 5d9cd25

• Add caching of cusparseLT meta

  ilmarkov

  committed
  39ad9d4

  Copy full SHA for 39ad9d4

• Cached cusparseLt

  ilmarkov

  committed
  520eb62

  Copy full SHA for 520eb62

• Fix destroy function

  ilmarkov

  committed
  20956e6

  Copy full SHA for 20956e6

• Prepare for reproduce

  ilmarkov

  committed
  87c8088

  Copy full SHA for 87c8088

• Fix cusparseLt caching

  ilmarkov

  committed
  4ea58b1

  Copy full SHA for 4ea58b1

• Make cached version default function

  ilmarkov

  committed
  f0551ef

  Copy full SHA for f0551ef

Commits on Nov 6, 2024

• Fixes and polishing after rebase

  ilmarkov

  committed
  d7476e8

  Copy full SHA for d7476e8

• add sparse 2:4 weight loading suport

  

  dsikka

  authored and

  rahul-tuli

  committed
  681ea5e

  Copy full SHA for 681ea5e

• Some more changes!

  rahul-tuli

  committed
  ecf878f

  Copy full SHA for ecf878f

• Cleanup

  rahul-tuli

  committed
  80952dc

  Copy full SHA for 80952dc

• get uncompressed to work; update gemm to use contiguous; use alex's utils instead of our decompressor

  

  dsikka

  authored and

  rahul-tuli

  committed
  8462c9d

  Copy full SHA for 8462c9d

• patch

  

  dsikka

  authored and

  rahul-tuli

  committed
  0a3e506

  Copy full SHA for 0a3e506

• use our decompressor

  

  dsikka

  authored and

  rahul-tuli

  committed
  2e28972

  Copy full SHA for 2e28972

• Some more work

  rahul-tuli

  committed
  28f0abb

  Copy full SHA for 28f0abb

Commits on Nov 7, 2024

• Use new scaled_T function

  rahul-tuli

  committed
  c7a97a8

  Copy full SHA for c7a97a8

Commits on Nov 8, 2024

• Add multiprocessing for kernel sweep benchmarking

  Faraz9877

  committed
  ccadad0

  Copy full SHA for ccadad0

• Add multi-GPU

  Faraz9877

  committed
  807737c

  Copy full SHA for 807737c

Commits on Nov 14, 2024

• Add cutlass_scaled_sparse_mm op

  Faraz9877

  committed
  04c19a5

  Copy full SHA for 04c19a5

• Clean up

  Faraz9877

  committed
  2a85c5a

  Copy full SHA for 2a85c5a

• Update code

  Faraz9877

  committed
  1b381c9

  Copy full SHA for 1b381c9

• Update code

  Faraz9877

  committed
  4e31076

  Copy full SHA for 4e31076

• Clean up the benchmarking

  Faraz9877

  committed
  13fccf4

  Copy full SHA for 13fccf4

• Clean up the cutlass benchmarking

  Faraz9877

  committed
  b345cc8

  Copy full SHA for b345cc8

• Fix cmake errors

  Faraz9877

  committed
  2d03e1d

  Copy full SHA for 2d03e1d

• Fix the cmake TAG

  Faraz9877

  committed
  e9439cc

  Copy full SHA for e9439cc

Commits on Nov 15, 2024

• Merge branch 'buildable' into rahul-quant-merged-rs

  robertgshaw2-redhat

  committed
  4ba7c0f

  Copy full SHA for 4ba7c0f

• update

  robertgshaw2-redhat

  committed
  f74ef37

  Copy full SHA for f74ef37

• fixed

  robertgshaw2-redhat

  committed
  f5bc9eb

  Copy full SHA for f5bc9eb

• updated

  robertgshaw2-redhat

  committed
  1316076

  Copy full SHA for 1316076

• updated

  robertgshaw2-redhat

  committed
  4d2b12c

  Copy full SHA for 4d2b12c

• updated, calling things properly

  robertgshaw2-redhat

  committed
  fe30b53

  Copy full SHA for fe30b53

• running end to end but not passing

  robertgshaw2-redhat

  committed
  4c61b19

  Copy full SHA for 4c61b19

• updated

  robertgshaw2-redhat

  committed
  86716f8

  Copy full SHA for 86716f8

• Some cleanup

  rahul-tuli

  committed
  c796ac8

  Copy full SHA for c796ac8