Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do-not-merge] Ibm 20241217 #264

Closed
wants to merge 393 commits into from
Closed

[do-not-merge] Ibm 20241217 #264

wants to merge 393 commits into from

Conversation

fialhocoelho
Copy link

build image.

2015aroras and others added 30 commits November 25, 2024 17:26
@2015aroras
[Model] Add OLMo November 2024 model (vllm-project#10503)
9db713a
@youkaichao
[misc] do not read HOST_IP (vllm-project#10644)
6e9ff05 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[bugfix] fix aria model and add torch.compile (vllm-project#10645)
45ac4ff 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@sanketkaleoss @mgoin
[Feature] vLLM ARM Enablement for AARCH64 CPUs (vllm-project#9228)
a6760f6 
Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
@rickyyx
[v1] EngineArgs for better config handling for v1 (vllm-project#10382)
519e8e4 
Signed-off-by: rickyx <rickyx@anyscale.com>
@SageMoore @youkaichao
custom allreduce + torch.compile (vllm-project#10121)
9a88f89 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
@DarkLight1337
[Misc] Remove outdated init protocols (vllm-project#10655)
9406353 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[ci] add vllm_test_utils (vllm-project#10659)
334d64d 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@jikunshang
[V1] Enable profile for LLMEngine (vllm-project#10665)
1f6584e
@andoorve @sroy745
[Bugfix] Fix for Spec model TP + Chunked Prefill (vllm-project#10232)
db66e01 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>
@conroy-cheers
[Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (vllm-project#9735)
f5792c7 
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
@mgoin
[Bugfix] Fix using -O[0,3] with LLM entrypoint (vllm-project#10677)
9a99273 
Signed-off-by: mgoin <michael@neuralmagic.com>
@mgoin
[Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (vllm-project#…
7576cd3 
…10642)
@ywang96
[V1] Refactor model executable interface for multimodal models (vllm-…
2f0a0a1 
…project#10570)

Signed-off-by: Roger Wang <ywang@roblox.com>
@xuechendi
Remove hard-dependencies of Speculative decode to CUDA workers (vllm-…
0a71900 
…project#10587)

Signed-off-by: Chendi Xue <chendi.xue@intel.com>
@ywang96
[V1] Update interface for idefics3 (vllm-project#10680)
0a4d968 
Signed-off-by: Roger Wang <ywang@roblox.com>
@jeongin601
[Bugfix][SpecDecode] apply sampling parameters to target probabilitie…
1bf905d 
…s for consistency in rejection sampling. (vllm-project#10198)

Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>
@yansh97
[bugfix] fix the default value of llm_int8_threshold in BitsAndBytesC…
cfb3bf2 
…onfig (vllm-project#10657)
@jikunshang
[Hardware][Gaudi]add get_name method for HPUAttentionBackend (vllm-pr…
e85250b 
…oject#10667)

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
@jeejeelee
[Misc]Further reduce BNB static variable (vllm-project#10597)
15cc2a9 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@tlrmchlsmth
[Kernel] Remove if-else with identical branches in marlin 2:4 (vllm-p…
e225110 
…roject#10687)

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
@shunxing12345 @Isotr0py
[Model] Support telechat2 (vllm-project#10311)
1209261 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: xiangw2 <xiangw2@chinatelecom.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
@bigPYJ1151
[Bugfix][Hardware][CPU] Fix intel-omp version to avoid segfault (vllm…
418cb3b 
…-project#10700)

Signed-off-by: jiang1.li <jiang1.li@intel.com>
@ywang96
[V1] Update interface for mistral-format Pixtral (vllm-project#10703)
9e0a147 
Signed-off-by: Roger Wang <ywang@roblox.com>
@youkaichao
[ci] fix slow tests (vllm-project#10698)
308cc5e 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[torch.compile] fix shape specialization (vllm-project#10722)
c411def 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@Isotr0py
[Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (vllm-pr…
b98c62b 
…oject#10675)

Signed-off-by: Isotr0py <2037008807@qq.com>
@mzusman
[Bugfix][Mamba] Fix Multistep on Mamba-like models (vllm-project#10705)
197b448 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
@DarkLight1337
[Bugfix] Ignore lm_head when loading embedding models (vllm-project…
9b4b150 
…#10719)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@tomeras91
[Frontend] don't block event loop in tokenization (preprocess) in Ope…
395b1c7 
…nAI compatible server (vllm-project#10635)

Signed-off-by: Tomer Asida <tomera@ai21.com>
DarkLight1337 and others added 22 commits December 15, 2024 06:30
@DarkLight1337
[Misc] Clean up multi-modal processor (vllm-project#11207)
b10609e 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Bugfix] Fix error handling of unsupported sliding window (vllm-proje…
96d673e 
…ct#11213)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@KuntaiDu
[Doc] add documentation for disaggregated prefilling (vllm-project#11197
38e599d 
)

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
@ShangmingCai
[Core] Support disaggregated prefill with Mooncake Transfer Engine (v…
d263bd9 
…llm-project#10884)

Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
@WoosukKwon
[V1][Minor] Cache np arange to reduce input preparation overhead (vll…
25ebed2 
…m-project#11214)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@AlexHe99
Update deploying_with_k8s.rst (vllm-project#10922)
da6f409
@chenqianfzh
[Bugfix] Fix block size validation (vllm-project#10938)
69ba344
@yansh97
[Bugfix] Fix the default value for temperature in ChatCompletionReque…
17138af 
…st (vllm-project#11219)
@cennn @drikster80
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (vllm-pro…
b3b1526 
…ject#11212)

Signed-off-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
@janimo
[Model] Support Cohere2ForCausalLM (Cohere R7B) (vllm-project#11203)
bddbbcb
@Isotr0py @DarkLight1337
[Model] Refactor Ultravox to use merged input processor (vllm-project…
d927dbc 
…#11198)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@Isotr0py
[Doc] Reorder vision language examples in alphabet order (vllm-projec…
2ca830d 
…t#11228)

Signed-off-by: Isotr0py <2037008807@qq.com>
@varun-sundar-rabindranath
[misc] Layerwise profile updates (vllm-project#10242)
efbce85 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
@youkaichao
[core] overhaul memory profiling and fix backward compatibility (vllm…
551603f 
…-project#10511)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@bk-TurbaAI @mgoin
[Docs] hint to enable use of GPU performance counters in profiling to…
35ffa68 
…ols for multi-node distributed serving (vllm-project#11235)

Co-authored-by: Michael Goin <michael@neuralmagic.com>
@youkaichao
[ci][tests] add gh200 tests (vllm-project#11244)
c301616 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao @tlrmchlsmth
[torch.compile] fast inductor (vllm-project#11108)
88a412e 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
@youkaichao
fix gh200 tests on main (vllm-project#11246)
35bae11 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@fialhocoelho
Merge branch 'main' of https://github.com/vllm-project/vllm into ibm-…
e08d963 
…20241217

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 6357
aefef9a 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
Squash 10235
6ef1de1 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho
install numactl to enable fastsafetensors
ac7676e 
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
@fialhocoelho fialhocoelho self-assigned this Dec 18, 2024
@fialhocoelho fialhocoelho requested a review from njhill as a code owner December 18, 2024 12:23
@openshift-ci openshift-ci bot requested review from rpancham and tarukumar December 18, 2024 12:23
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: fialhocoelho
Once this PR has been reviewed and has the lgtm label, please assign tarukumar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Dec 18, 2024

@fialhocoelho: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/smoke-test ac7676e link true /test smoke-test
ci/prow/images ac7676e link true /test images

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@rpancham rpancham Awaiting requested review from rpancham

@tarukumar tarukumar Awaiting requested review from tarukumar

Assignees

@fialhocoelho fialhocoelho

Labels
do-not-merge/hold
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.