Code

Tpu v1 mgoin#53

Code

Draft

mgoin wants to merge 35 commits intomainneuralmagic/vllm:mainfrom tpu-v1-mgoinneuralmagic/vllm:tpu-v1-mgoinCopy head branch name to clipboard

+77,573-29,866

Commits

Commits on Nov 12, 2024

• prototype tpu on v1

  robertgshaw2-redhat

  committed
  f69bdea

  Copy full SHA for f69bdea

• profile run complete

  robertgshaw2-redhat

  committed
  1142c89

  Copy full SHA for 1142c89

• actually dummy run

  robertgshaw2-redhat

  committed
  9cc4fbe

  Copy full SHA for 9cc4fbe

• stash

  robertgshaw2-redhat

  committed
  61f7792

  Copy full SHA for 61f7792

• update workflow

  robertgshaw2-redhat

  committed
  1887d81

  Copy full SHA for 1887d81

• updated

  Show description for b8c6444
  robertgshaw2-redhat

  committed
  b8c6444

  Copy full SHA for b8c6444

• updated

  robertgshaw2-redhat

  committed
  75e2e53

  Copy full SHA for 75e2e53

• more cleaning

  robertgshaw2-redhat

  committed
  bebabfc

  Copy full SHA for bebabfc

• cleanup llmengine

  robertgshaw2-redhat

  committed
  338e11c

  Copy full SHA for 338e11c

• Revert "cleanup llmengine"

  Show description for db49d3b
  robertgshaw2-redhat

  committed
  db49d3b

  Copy full SHA for db49d3b

• fixt

  robertgshaw2-redhat

  committed
  4ade5b0

  Copy full SHA for 4ade5b0

• warmup is working!

  robertgshaw2-redhat

  committed
  dc78451

  Copy full SHA for dc78451

• stash

  robertgshaw2-redhat

  committed
  7f8fdee

  Copy full SHA for 7f8fdee

Commits on Nov 15, 2024

• stash

  robertgshaw2-redhat

  committed
  f7de1b4

  Copy full SHA for f7de1b4

Commits on Nov 16, 2024

• workin for prefill, except when I compile decode cudagraphs?

  robertgshaw2-redhat

  committed
  5de1d9f

  Copy full SHA for 5de1d9f

• working! It was the type of the position ids!

  robertgshaw2-redhat

  committed
  15a2f74

  Copy full SHA for 15a2f74

• forward pass

  robertgshaw2-redhat

  committed
  14b9500

  Copy full SHA for 14b9500

• correct output for single prompt with --enforce-eager

  robertgshaw2-redhat

  committed
  6eeecb7

  Copy full SHA for 6eeecb7

• end to end passing working for single request with CUDAGraphs!

  robertgshaw2-redhat

  committed
  0b256c2

  Copy full SHA for 0b256c2

• yay! working with multiple requests! the issue was copy_() does not seem to work for xla

  robertgshaw2-redhat

  committed
  b44227d

  Copy full SHA for b44227d

• yay! working end to end via lm eval harness!

  robertgshaw2-redhat

  committed
  451dfbf

  Copy full SHA for 451dfbf

• we have end to end correctness

  robertgshaw2-redhat

  committed
  d2ae4a5

  Copy full SHA for d2ae4a5

• nits

  robertgshaw2-redhat

  committed
  7dd18e0

  Copy full SHA for 7dd18e0

• updated

  robertgshaw2-redhat

  committed
  d89200d

  Copy full SHA for d89200d

Commits on Nov 17, 2024

• update to call .cpu() before slicing to avoid recompilation

  robertgshaw2-redhat

  committed
  75c44b4

  Copy full SHA for 75c44b4

• a bit faster

  robertgshaw2-redhat

  committed
  58e85eb

  Copy full SHA for 58e85eb

• better performance due to better input processing

  robertgshaw2-redhat

  committed
  fcf4681

  Copy full SHA for fcf4681

• cleanup PR

  robertgshaw2-redhat

  committed
  d9dc36a

  Copy full SHA for d9dc36a

• cleanup

  robertgshaw2-redhat

  committed
  85bc154

  Copy full SHA for 85bc154

• cleanup pr

  robertgshaw2-redhat

  committed
  25fff99

  Copy full SHA for 25fff99

• formatting

  robertgshaw2-redhat

  committed
  5a87b99

  Copy full SHA for 5a87b99

• updated

  robertgshaw2-redhat

  committed
  63b301a

  Copy full SHA for 63b301a

• updated

  robertgshaw2-redhat

  committed
  1af03e0

  Copy full SHA for 1af03e0

• fixed accuracy bug

  robertgshaw2-redhat

  committed
  02ee304

  Copy full SHA for 02ee304

Commits on Jan 9, 2025

• Attempt to merge with main, fails on compilation

  Show description for 14fc6bc
  mgoin

  committed
  14fc6bc

  Copy full SHA for 14fc6bc