# DeepSeek Sparse Attention (DSA)

> A lightweight indexer scores past tokens and each query attends only to its top-k picks.

Category: Sparse & compressed attention

DSA adds a small, fast indexer that estimates how relevant each earlier token is to the current query. A selector keeps the top-k tokens (2,048 in DeepSeek's code), and full attention is computed only over those.

Unlike a fixed sliding window, the selection is learned and can reach anywhere in the context. It was introduced with DeepSeek V3.2 on top of MLA and later adopted by GLM-5.

**Trade-off:** Long-context attention cost grows with k instead of the full sequence length.

## Models using DSA (10)

- [Mistral Large 4 (1T)](https://llm-atlas-7or.pages.dev/models/mistral-large-4.md) — Mistral AI, 2026-10-29
- [Naive-N0.5-Flash (309B)](https://llm-atlas-7or.pages.dev/models/naive-n0-5-flash-309b.md) — NaiveAI, 2026-09-27
- [Tencent Hy4-preview (770B-A49B)](https://llm-atlas-7or.pages.dev/models/tencent-hy4-preview-770b-a49b.md) — Tencent, 2026-08-28
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [GLM-5.2 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-2.md) — Z.ai, 2026-06-17
- [DeepSeek V4-Flash (284B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-flash.md) — DeepSeek, 2026-04-24
- [DeepSeek V4-Pro (1.6T)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-pro.md) — DeepSeek, 2026-04-24
- [GLM-5.1 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-1.md) — Z.ai, 2026-04-07
- [GLM-5 (744B)](https://llm-atlas-7or.pages.dev/models/glm-5-744b.md) — Z.ai, 2026-02-11
- [DeepSeek V3.2 (671B)](https://llm-atlas-7or.pages.dev/models/deepseek-v3-2.md) — DeepSeek, 2025-12-01

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/deepseek-sparse-attention/
Web page: https://llm-atlas-7or.pages.dev/concepts/deepseek-sparse-attention/