[pull] main from vllm-project:main #32
Conversation
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Signed-off-by: kerthcet <kerthcet@gmail.com>
|
Hi @pull[bot]. Thanks for your PR. I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Pass the CUDA stream into the CUTLASS GEMMs, to avoid future issues with CUDA graphs
The 2nd PR for #4532. This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com> Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Elisei Smirnov <el.smirnov@innopolis.university>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
…-Small model (#4799) Co-authored-by: beagleski <yunanzhang@microsoft.com> Co-authored-by: bapatra <bapatra@microsoft.com> Co-authored-by: Barun Patra <codedecde@users.noreply.github.com> Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Alexey Kondratiev <alexey.kondratiev@amd.com> Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com> Co-authored-by: Alexei V. Ivanov <alexei.ivanov@amd.com> Co-authored-by: omkarkakarparthi <okakarpa>
Co-authored-by: Breno Faria <breno.faria@intrafind.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
…red_metadata modifier (introduced with PTX 8.5) (#5136)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
…e ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)
Co-authored-by: xuhao <xuhao@cambricon.com>
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: pull[bot] The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Adds support for multi-lora adapters. Passing tests added over in this PR: https://github.ibm.com/ai-foundation/tgis-deploy-tests/pull/25/files --------- Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
* add gaudi installation readme * readme writeup * Create README_GAUDI.md * Update README.md * Update README_GAUDI.md * Update README.md * Update readmes
Update linear.py
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )