# Qwen Sparse Attention (QSA)

> Like DSA, but the indexer picks whole four-token blocks rather than single tokens.

Category: Sparse & compressed attention

The indexer averages keys within each complete four-token block, scores the pooled keys against the query, and expands the winning blocks back into their tokens. The current incomplete block is always visible.

Selecting contiguous blocks is friendlier to memory access patterns than scattering over individual token positions.

**Trade-off:** Hardware-friendlier sparse reads at a coarser selection granularity.

## Models using QSA (1)

- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/qwen-sparse-attention/
Web page: https://llm-atlas-7or.pages.dev/concepts/qwen-sparse-attention/