# Compressed Sparse & Heavily Compressed Attention (CSA / HCA)

> Pool groups of older tokens into fewer KV entries, compressing along the sequence axis.

Category: Sparse & compressed attention

MLA shrinks each token's cache entry; CSA and HCA reduce how many entries there are. Groups of past tokens are pooled into compressed entries.

CSA compresses moderately and reads a sparse selection of entries; HCA compresses aggressively and reads all of them. DeepSeek V4 combines both to reach million-token contexts.

**Trade-off:** Drastically smaller long-context caches, with some loss of token-level detail in the far past.

## Models using CSA / HCA (2)

- [DeepSeek V4-Flash (284B)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-flash.md) — DeepSeek, 2026-04-24
- [DeepSeek V4-Pro (1.6T)](https://llm-atlas-7or.pages.dev/models/deepseek-v4-pro.md) — DeepSeek, 2026-04-24

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/csa-hca/
Web page: https://llm-atlas-7or.pages.dev/concepts/csa-hca/