Sync with upstream@v0.4.3-60-gbaa15a9e#47

Merged

dtrifiro merged 30 commits intoopendatahub-io:mainfrom vllm-project:main

Jun 10, 2024

+4,362-1,520

Commits on Jun 7, 2024

[Core] Avoid copying prompt/output tokens if no penalties are used (#5289 )
Yard1
authored
[Core] Change LoRA embedding sharding to support loading methods (#5038 )
Yard1
authored
[Misc] Missing error message for custom ops import (#5282 )
DamonFool
authored
[Feature][Frontend]: Add support for stream_options in ChatCompletionRequest (#5135 )
Etelis
authored
[Misc][Utils] allow get_open_port to be called for multiple times (#5333 )
youkaichao
authored
[Kernel] Switch fp8 layers to use the CUTLASS kernels (#5183 )
tlrmchlsmth
authored
Remove Ray health check (#4693 )
Yard1
authored
Addition of lacked ignored_seq_groups in _schedule_chunked_prefill (#5296 )
JamesLim-sy
authored
[Kernel] Dynamic Per-Token Activation Quantization (#5037 )

authored
[Frontend] Add OpenAI Vision API Support (#5237 )

ywang96
and
DarkLight1337
authored
[Misc] Remove unused cuda_utils.h in CPU backend (#5345 )
DamonFool
authored
fix DbrxFusedNormAttention missing cache_config (#5340 )

Calvinnncy97
and
team
authored