#

flashmla

Here is 1 public repository matching this topic...

Bruce-Lee-LY / decoding_attention

Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.

gpu cuda inference nvidia mha mla multi-head-attention gqa mqa llm large-language-model flash-attention cuda-core decoding-attention flashinfer flashmla

Updated Mar 3, 2025
C++

Improve this page

Add a description, image, and links to the flashmla topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the flashmla topic, visit your repo's landing page and select "manage topics."