New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] Update PagedAttention creation logic #28802

Merged

vladimir-paramuzov merged 1 commit into openvinotoolkit:master from sshlyapn:update_paged_attention_creation_logic

Feb 4, 2025

Contributor

sshlyapn commented Feb 4, 2025

Details:

Currently GenAI library sets fully dynamic shapes for Key/Value cache buffers, which causes GPU Plugin to fail during compile_model() call. Therefore, update PagedAttention creation logic to use head_size and heads_num parameters from the rt_info if available
GenAI related PR: [CB] Add KVHeadConfig parameters to PagedAttention's rt_info openvino.genai#1666


          [GPU] Update PagedAttention creation logic: use head_size and heads_n…

9102e88

…um parameters from rt_info if available

sshlyapn added the category: GPU label

sshlyapn added this to the 2025.1 milestone

sshlyapn requested review from a team as code owners

February 4, 2025 07:00

vladimir-paramuzov approved these changes

View reviewed changes

vladimir-paramuzov enabled auto-merge

February 4, 2025 07:09

vladimir-paramuzov added this pull request to the merge queue

Merged via the queue into openvinotoolkit:master with commit b603d00

167 checks passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels