Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Logprobs #45

Open
wants to merge 1,231 commits into
base: afeldman-nm/logprobs
Choose a base branch
from
Open

Logprobs #45

wants to merge 1,231 commits into from

Conversation

robertgshaw2-redhat
Copy link
Collaborator

FILL IN THE PR DESCRIPTION HERE

FIX #xxxx (link existing issues this PR will resolve)

**BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing/overview.html **

patrickvonplaten and others added 30 commits November 30, 2024 07:45
@patrickvonplaten @youkaichao
[Interleaved ATTN] Support for Mistral-8B (vllm-project#10591)
e7cfc4e 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
@wangxiyuan
[doc] format fix (vllm-project#10789)
7e4bbda 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
@DarkLight1337
[Model] Replace embedding models with pooling adapter (vllm-project#1…
1337071 
…0769)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Misc] Improve type annotations for support_torch_compile (vllm-pro…
f877a7d 
…ject#10763)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Misc] Rename embedding classes to pooling (vllm-project#10801)
d2f058e 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[doc] add warning about comparing hf and vllm outputs (vllm-project#1…
169a0ff 
…0805)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96 @heheda12345 @Isotr0py
[Misc] Adding MMMU-Pro vision dataset to serving benchmark (vllm-pr…
c11f172 
…oject#10804)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
f446a78
@afeldman-nm
removed fast tests from pipeline
fda0fcb
@KuntaiDu
[Core] Implement disagg prefill by StatelessProcessGroup (vllm-projec…
0590ec3 
…t#10502)

This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>
@Isotr0py
[Model] Add BNB support to Llava and Pixtral-HF (vllm-project#10795)
b18c9bb 
Signed-off-by: Isotr0py <2037008807@qq.com>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
0bcf5f4
@cduk
[core] Avoid metrics log noise when idle - include speculative decodi… (
b795477 
vllm-project#10809)
@WoosukKwon
[Kernel] Use out arg in flash_attn_varlen_func (vllm-project#10811)
073a4bd 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@maxdebayser
Fill TorchSDPAAttentionMetadata seq_lens_field for prefill (vllm-proj…
e25810a 
…ect#10799)

Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
@youkaichao
[misc] remove xverse modeling file (vllm-project#10814)
63a1641 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@wangxiyuan
[doc]Update config docstring (vllm-project#10732)
995a148 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
@xffxff @Isotr0py
[Model]: add some tests for aria model (vllm-project#10770)
ef31eab 
Signed-off-by: xffxff <1247714429@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
ee1b910
@afeldman-nm @WoosukKwon
Update vllm/outputs.py
1fa0b71 
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
@afeldman-nm
Merge branch 'afeldman-nm/v1_logprobs' of https://github.com/neuralma…
6ba743f 
…gic/vllm into v1_logprobs
@afeldman-nm
small fixes
bc1c004
@DarkLight1337
[CI/Build] Update mistral_common version for tests and docs (vllm-p…
e95f275 
…roject#10825)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@afeldman-nm
moved output processing commands into processor
bec886b 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
fef9f30
@youkaichao
[misc] use out argument for flash attention (vllm-project#10822)
a4c4daf 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@afeldman-nm
added explanatory comment to EngineCore.update_from_output()
554f431
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
5022307
@youkaichao @afeldman-nm
[misc] move functions to config.py (vllm-project#10624)
5dea1d5 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@DarkLight1337 @afeldman-nm
[Model] Support is_causal HF config field for Qwen2 model (vllm-pro…
930f2cc 
…ject#10621)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
DarkLight1337 and others added 29 commits December 7, 2024 07:22
@DarkLight1337
[Model] Composite weight loading for multimodal Qwen2 (vllm-project#1…
bf0e382 
…0944)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@afeldman-nm
Merge branch 'main' into v1_logprobs
cef5ddb
@DarkLight1337
[Doc] Explicitly state that InternVL 2.5 is supported (vllm-project#1…
1c768fe 
…0978)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Model] Update multi-modal processor to support Mantis(LLaVA) model (v…
39e227c 
…llm-project#10711)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Doc] Explicitly state that PP isn't compatible with speculative deco…
c889d58 
…ding yet (vllm-project#10975)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@xffxff
[BugFix][Kernel]: fix illegal memory access in causal_conv1d when con…
78029b3 
…v_states is None (vllm-project#10928)

Signed-off-by: xffxff <1247714429@qq.com>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
bed24db
@afeldman-nm
move
5ce8128
@youkaichao
[core][executor] simplify instance id (vllm-project#10976)
1b62745 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[core][misc] remove use_dummy driver for _run_workers (vllm-project#1…
7be15d9 
…0920)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[torch.compile] allow candidate compile sizes (vllm-project#10984)
fd57d2b 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96
[V1] Initial support of multimodal models for V1 re-arch (vllm-projec…
a11f326 
…t#10699)

Signed-off-by: Roger Wang <ywang@roblox.com>
@youkaichao
[torch.compile][misc] fix comments (vllm-project#10993)
43b05fa 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@youkaichao
[misc] clean up and unify logging (vllm-project#10999)
46004e8 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96
[Doc][V1] Add V1 support column for multimodal models (vllm-project#1…
af7c4a9 
…0998)

Signed-off-by: Roger Wang <ywang@roblox.com>
@youkaichao
[torch.compile] add dynamo time tracking (vllm-project#11005)
d1c2e15 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@ywang96
[V1] Fix Detokenizer loading in AsyncLLM (vllm-project#10997)
c690357 
Signed-off-by: Roger Wang <ywang@roblox.com>
@russellb
[Core] Require xgrammar >= 0.1.6 (vllm-project#11021)
e691b26 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@wangxiyuan
[Platform] Move async output check to platform (vllm-project#10768)
aea2fc3 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
@varun-sundar-rabindranath
[V1] Input Batch Relocation (vllm-project#10962)
25b79d9 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
@afeldman-nm
merge
14c7e56 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
removed VLLM_USE_V1 checks
bdd0abf 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
revert logprobs name changes
1fc981e 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
removing some unnecessary changes'
dc63ac1 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
removed fast checks
4f30408 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
wip test_completion
77488cb
@afeldman-nm
toward completion tests
f1a689c 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@afeldman-nm
serialization fix
e962aa7 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
@robertgshaw2-redhat
merged
0f6790d
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.