Code

[pull] main from vllm-project:main#8

Code

Merged

z103cb merged 25 commits intoopendatahub-io:mainopendatahub-io/vllm:mainfrom vllm-project:mainvllm-project/vllm:mainCopy head branch name to clipboardApr 30, 2024

Commits

Commits on Apr 26, 2024

• [Misc][Refactor] Generalize linear_method to be quant_method (#4373)

  comaniac

  authored
  a62aaf1

  Copy full SHA for a62aaf1

• [Misc] add RFC issue template (#4401)

  Show description for aba47be
  

  youkaichao

  and

  simon-mo

  authored
  aba47be

  Copy full SHA for aba47be

Commits on Apr 27, 2024

• [Core] Introduce DistributedGPUExecutor abstract class (#4348)

  njhill

  authored
  258a2c5

  Copy full SHA for 258a2c5

• [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343)

  Show description for 12628d3
  

  pcmoritz

  and

  WoosukKwon

  authored
  12628d3

  Copy full SHA for 12628d3

• [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355)

  DarkLight1337

  authored
  8947bc3

  Copy full SHA for 8947bc3

• [Misc] Fix logger format typo (#4396)

  esmeetu

  authored
  87f545b

  Copy full SHA for 87f545b

• [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406)

  hongxiayang

  authored
  18d23f6

  Copy full SHA for 18d23f6

• [Kernel] Full Tensor Parallelism for LoRA Layers (#3524)

  Show description for eefeb16
  

  FurtherAI

  and

  Yard1

  authored
  eefeb16

  Copy full SHA for eefeb16

• [Model] Phi-3 4k sliding window temp. fix (#4380)

  caiom

  authored
  3da24c2

  Copy full SHA for 3da24c2

• [Bugfix][Core] Fix get decoding config from ray (#4335)

  esmeetu

  authored
  7134303

  Copy full SHA for 7134303

• [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363)

  chestnut-Q

  authored
  dfea173

  Copy full SHA for dfea173

• [BugFix] Fix min_tokens when eos_token_id is None (#4389)

  Show description for 81661da
  

  njhill

  and

  DefTruth

  authored
  81661da

  Copy full SHA for 81661da

• [Core] Support offline use of local cache for models (#4374)

  Show description for d6e520e
  

  prashantgupta24

  and

  tjohnson31415

  authored
  d6e520e

  Copy full SHA for d6e520e

• [BugFix] Fix return type of executor execute_model methods (#4402)

  njhill

  authored
  ba4be44

  Copy full SHA for ba4be44

• [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418)

  robertgshaw2-redhat

  authored
  4ea1f96

  Copy full SHA for 4ea1f96

Commits on Apr 28, 2024

• [Misc] fix typo in llm_engine init logging (#4428)

  DefTruth

  authored
  9c7306a

  Copy full SHA for 9c7306a

• Add more Prometheus metrics (#2764)

  Show description for bf480c5
  

  3 peopleauthored
  bf480c5

  Copy full SHA for bf480c5

• [CI] clean docker cache for neuron (#4441)

  simon-mo

  authored
  03dd7d5

  Copy full SHA for 03dd7d5

Commits on Apr 29, 2024

• [mypy][5/N] Support all typing on model executor (#4427)

  rkooo567

  authored
  df29793

  Copy full SHA for df29793

• [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922)

  Show description for 73c8d67
  

  3 peopleauthored
  73c8d67

  Copy full SHA for 73c8d67

• [CI] hotfix: soft fail neuron test (#4458)

  simon-mo

  authored
  ac5ccf0

  Copy full SHA for ac5ccf0

• [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444)

  youkaichao

  authored
  f4f921b

  Copy full SHA for f4f921b

Commits on Apr 30, 2024

• [Misc] Upgrade to torch==2.3.0 (#4454)

  mgoin

  authored
  d627a3d

  Copy full SHA for d627a3d

• [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463)

  WoosukKwon

  authored
  fa32207

  Copy full SHA for fa32207

• [Core]Refactor gptq_marlin ops (#4466)

  jikunshang

  authored
  26f2fb5

  Copy full SHA for 26f2fb5