Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
-
Updated
Feb 25, 2025 - Python
#
sparse-attention
Here are 8 public repositories matching this topic...
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
-
Updated
Oct 30, 2024 - Python
The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
-
Updated
Dec 6, 2024 - Python
SpargeAttention: A training-free sparse attention that can accelerate any model inference.
attention quantization mlsys inference-acceleration ai-infra vision-transformer sparse-attention llm sageattention
-
Updated
Feb 25, 2025 - Cuda
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
attention-mechanism fast-inference sparse-neural-networks low-rank vision-transformer efficient-transformers sparse-attention efficient-vision-transformers
-
Updated
Jul 4, 2023 - Python
Pytorch implementation of "Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution"
-
Updated
Nov 19, 2024
Building Native Sparse Attention
-
Updated
Feb 20, 2025 - Python
Text Summarization Modeling with three different Attention Types
-
Updated
May 29, 2024 - Python
Improve this page
Add a description, image, and links to the sparse-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sparse-attention topic, visit your repo's landing page and select "manage topics."