Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] support 64 alignment for 8bit weights layout #28864

Merged
merged 2 commits into from
Feb 10, 2025
Merged

[GPU] support 64 alignment for 8bit weights layout #28864

merged 2 commits into from
Feb 10, 2025
+11 −2

Conversation

ahnyoung-paul
Copy link
Contributor

@ahnyoung-paul ahnyoung-paul commented Feb 7, 2025 •
edited
Loading

Details:

  • In the int8 model, if the batch size of the FC layer is 1025 (which is aligned to 1040), the FC layer can't use SLM because 1040 is not aligned to 64. The reason for this misalignment is that 64 alignment is only applied for i4 weights in the fake alignment function.
  • Modified to allow 64 alignment for i8/u8 weights to use SLM for 1045 batch size.

Tickets:

  • 161596

@ahnyoung-paul ahnyoung-paul added the category: GPU OpenVINO GPU plugin label Feb 7, 2025
@ahnyoung-paul ahnyoung-paul requested review from a team as code owners February 7, 2025 02:47
@ahnyoung-paul ahnyoung-paul added pr: needs tests PR needs tests updating and removed pr: needs tests PR needs tests updating labels Feb 7, 2025
@ahnyoung-paul ahnyoung-paul force-pushed the support_64_alignment_for_8bit_weights_layout branch from 9085a7e to 1ecd878 Compare February 10, 2025 03:40
@p-durandin p-durandin enabled auto-merge February 10, 2025 07:07
@p-durandin p-durandin added this pull request to the merge queue Feb 10, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 10, 2025
@p-durandin p-durandin added this pull request to the merge queue Feb 10, 2025
Merged via the queue into openvinotoolkit:master with commit 70ec531 Feb 10, 2025
167 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sshlyapn sshlyapn sshlyapn approved these changes

@p-durandin p-durandin p-durandin approved these changes

Assignees
No one assigned
Labels
category: GPU OpenVINO GPU plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ahnyoung-paul @sshlyapn @p-durandin