Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP upstream merge (DO NOT MERGE) #70

Closed
wants to merge 72 commits into from
Closed

WIP upstream merge (DO NOT MERGE) #70

wants to merge 72 commits into from

Conversation

prashantgupta24
Copy link

@prashantgupta24 prashantgupta24 commented Jun 28, 2024 •
edited
Loading

Created only for the sole purpose of trying to merge squash the commits in a different way

mgoin and others added 30 commits June 28, 2024 10:14
@mgoin @prashantgupta24
[Doc] Documentation on supported hardware for quantization methods (v…
3e30234 
…llm-project#5745)
@zhyncs @prashantgupta24
[BugFix] exclude version 1.15.0 for modelscope (vllm-project#5668)
23a692c
@youkaichao @prashantgupta24
[ci][test] fix ca test in main (vllm-project#5746)
5c2718b
@rohithkrn @prashantgupta24
[LoRA] Add support for pinning lora adapters in the LRU cache (vllm-p…
7da245c 
…roject#5603)
@jikunshang @prashantgupta24
[CI][Hardware][Intel GPU] add Intel GPU(XPU) ci pipeline (vllm-projec…
542b6b0 
…t#5616)
@DamonFool @ywang96 @prashantgupta24
[Model] Support Qwen-VL and Qwen-VL-Chat models with text-only inputs (…
9740329 
…vllm-project#5710)

Co-authored-by: Roger Wang <ywang@roblox.com>
@zifeitong @prashantgupta24
[Misc] Remove vllm-project#4789 workaround left in vllm/entrypoints/o…
6c39f95 
…penai/run_batch.py (vllm-project#5756)
@WoosukKwon @prashantgupta24
[Bugfix] Fix pin_lora error in TPU executor (vllm-project#5760)
a8c66a1
@WoosukKwon @prashantgupta24
[Docs][TPU] Add installation tip for TPU (vllm-project#5761)
1c147d6
@youkaichao @prashantgupta24
[core][distributed] improve shared memory broadcast (vllm-project#5754)
39b10b5
@varun-sundar-rabindranath @prashantgupta24
[BugFix] [Kernel] Add Cutlass2x fallback kernels (vllm-project#5744)
2bf04dc 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
@andoorve @prashantgupta24
[Distributed] Add send and recv helpers (vllm-project#5719)
3f87930
@Isotr0py @prashantgupta24
[Bugfix] Add phi3v resize for dynamic shape and fix torchvision requi…
d26db13 
…rement (vllm-project#5772)
@youkaichao @prashantgupta24
[doc][faq] add warning to download models for every nodes (vllm-proje…
9a77164 
…ct#5783)
@mgoin @prashantgupta24
[Doc] Add "Suggest edit" button to doc pages (vllm-project#5789)
eb05098
@mgoin @prashantgupta24
[Doc] Add Phi-3-medium to list of supported models (vllm-project#5788)
b8897c0
@CatherineSue @prashantgupta24
[Bugfix] Fix FlexibleArgumentParser replaces _ with - for actual args (…
2d30ec8 
…vllm-project#5795)
@khluu @prashantgupta24
[ci] Remove aws template (vllm-project#5757)
b9e54df 
Signed-off-by: kevin <kevin@anyscale.com>
@DarkLight1337 @prashantgupta24
[Doc] Add notice about breaking changes to VLMs (vllm-project#5818)
69c42d0
@wooyeonlee0 @prashantgupta24
[Speculative Decoding] Support draft model on different tensor-paral…
256623a 
…lel size than target model (vllm-project#5414)
@DamonFool @prashantgupta24
[Misc] Remove useless code in cpu_worker (vllm-project#5824)
f071f1a
@Yard1 @prashantgupta24
[Core] Add fault tolerance for RayTokenizerGroupPool (vllm-project#…
85ca6e3 
…5748)
@youkaichao @prashantgupta24
[doc][distributed] add both gloo and nccl tests (vllm-project#5834)
85466db
@mgoin @prashantgupta24
[CI/Build] Add unit testing for FlexibleArgumentParser (vllm-project#…
7d2a8a5 
…5798)
@dsikka @prashantgupta24
[Misc] Update w4a16 compressed-tensors support to include w8a16 (…
00f38d6 
…vllm-project#5794)
@WoosukKwon @prashantgupta24
[Hardware][TPU] Refactor TPU backend (vllm-project#5831)
03eacd4
@mawong-amd @prashantgupta24
[Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improv…
cb40d2d 
…ements, test fixes (vllm-project#5422)
@WoosukKwon @prashantgupta24
[Hardware][TPU] Raise errors for unsupported sampling params (vllm-pr…
d8238d7 
…oject#5850)
@tdoublep @prashantgupta24
[CI/Build] Add E2E tests for MLPSpeculator (vllm-project#5791)
1c02c31 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@aws-patlange @prashantgupta24
[Bugfix] Fix assertion in NeuronExecutor (vllm-project#5841)
b18f8f4
ywang96 and others added 15 commits June 28, 2024 10:14
@ywang96 @prashantgupta24
[CI/Build] Fix Args for _get_logits_warper in Sampler Test (vllm-pr…
087025d 
…oject#5922)
@WoosukKwon @prashantgupta24
[Model] Add Gemma 2 (vllm-project#5908)
2213a43
@youkaichao @prashantgupta24
[core][misc] remove logical block (vllm-project#5882)
0315ed0
@divakar-amd @prashantgupta24
[Kernel][ROCm][AMD] fused_moe Triton configs v2 for mi300X (vllm-proj…
6b75966 
…ect#5932)
@WoosukKwon @prashantgupta24
[Hardware][TPU] Optimize KV cache swapping (vllm-project#5878)
9f7eed7
@xwjiang2010 @ywang96 @prashantgupta24
[VLM][BugFix] Make sure that multi_modal_kwargs can broadcast prope…
779e4e0 
…rly with ring buffer. (vllm-project#5905)

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
@Isotr0py @prashantgupta24
[Bugfix][Hardware][Intel CPU] Fix unpassed multi_modal_kwargs for CPU…
cd41d94 
… runner (vllm-project#5956)
@DarkLight1337 @ywang96 @prashantgupta24
[Core] Registry for processing model inputs (vllm-project#5214)
0d31364 
Co-authored-by: ywang96 <ywang@roblox.com>
@tlrmchlsmth @prashantgupta24
Unmark fused_moe config json file as executable (vllm-project#5960)
c1c98fd
@ilya-lavrenov @prashantgupta24
[Hardware][Intel] OpenVINO vLLM backend (vllm-project#5379)
ecaeb79
@tdoublep @prashantgupta24
[Bugfix] Better error message for MLPSpeculator when `num_speculative…
6092257 
…_tokens` is set too high (vllm-project#5894)

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@DarkLight1337 @prashantgupta24
[CI/Build] [2/3] Reorganize entrypoints tests (vllm-project#5904)
52247e4
@xwjiang2010 @prashantgupta24
[Distributed] Make it clear that % should not be in tensor dict keys. (…
d7707e4 
…vllm-project#5927)

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
@comaniac @prashantgupta24
[Spec Decode] Introduce DraftModelRunner (vllm-project#5799)
f2db67d
@tlrmchlsmth @prashantgupta24
[Bugfix] Fix compute datatype for cutlass 3.x epilogues (vllm-project…
d2c456a 
…#5931)
@openshift-ci openshift-ci bot requested review from heyselbi and rpancham June 28, 2024 17:17
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Jun 28, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: prashantgupta24
Once this PR has been reviewed and has the lgtm label, please assign xaenalt for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@prashantgupta24
Copy link
Author

/test all

prashantgupta24 and others added 3 commits June 28, 2024 13:59
@prashantgupta24
Squash 5930
fa97f74
@njhill @prashantgupta24
[Core] Make Ray an optional "extras" requirement
076dcc6 
Still included in built docker images

Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24
🚧 add ibm-adapter branch
dff144a 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@prashantgupta24 prashantgupta24 changed the title WIP upstream merge WIP upstream merge (DO NOT MERGE) Jun 28, 2024
@prashantgupta24 prashantgupta24 deleted the wip-upstream-merge branch July 1, 2024 19:17
prarit pushed a commit to prarit/vllm that referenced this pull request Oct 18, 2024
@mawong-amd @Alexei-V-Ivanov-AMD
[2/2] Using xfail instead of skip for ROCm 6.2 tests (opendatahub-io#70)
cce6281 
* Enabling some basic tests for ROCm 6.2

Use strict xfail for ROCm 6.2 test repairs

* Use lenient xfail instead

---------

Co-authored-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@heyselbi heyselbi Awaiting requested review from heyselbi

@rpancham rpancham Awaiting requested review from rpancham

Assignees
No one assigned
Labels
do-not-merge/work-in-progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.