Skip to content

Sparse & compressed attention · IndexShare

IndexShare

Nearby layers reuse the token selection computed by one DSA indexer.

Computing a DSA indexer in every layer still costs time. IndexShare computes the top-k selection once and lets the following layers (three, in GLM-5.2) reuse it.

The goal is to keep most of the quality of per-layer selection while cutting the indexer overhead.

Read the full explainer Browse all 3 models
IndexShare computes top-k indices once and reuses them in following layers. Figure from Sebastian Raschka’s explainer.

Adoption over time

Models in the atlas using IndexShare, per half-year of release.

Loading

Used by

3

models in the atlas

Earliest: GLM-5.2 (744B) (Jun 17, 2026)

Often paired with

Models using IndexShare

  • Architecture diagram of Mistral Large 4 (1T)

    Mistral Large 4 (1T)

    Mistral AI, released 2026-10-29. Sparse MoE decoder, 1T total, 52B active, 1M context. Attention: MLA with modified DeepSeek Sparse Attention, index reuse, and KV sharing.

  • Architecture diagram of Tencent Hy4-preview (770B-A49B)

    Tencent Hy4-preview (770B-A49B)

    Tencent, released 2026-08-28. Sparse MoE decoder, 770B total, 49B active, 1M context. Attention: Gated MLA with DeepSeek Sparse Attention and IndexCache.

  • Architecture diagram of GLM-5.2 (744B)

    GLM-5.2 (744B)

    Z.ai, released 2026-06-17. Sparse MoE decoder, 744B total, 40B active, 1M context. Attention: MLA with DeepSeek Sparse Attention and IndexShare.

Search the atlas

Jump to a model, concept or page