Sparse hybridZ.ai 🇨🇳 · August 26, 2026

GLM-5.3-Flash (320B)

Natively multimodal GLM refresh, previously previewed as Ox Alpha, that replaces most MLA/DSA layers with Kimi Delta Attention and adds four-stream mHC.

Key detail

Mixes 34 Kimi Delta Attention layers with 11 MLA/DSA layers and routes residuals through four mHC streams.

Total params

320B

Active / token

18B

Context

Fact sheet

Scale: 320B total, 18B active (5.6% active)
Context window: 1,048,576 tokens
Decoder type: Sparse hybrid
Attention: 3:1 Kimi Delta Attention and MLA with DeepSeek Sparse Attention
Layer mix: 34 KDA + 11 MLA/DSA + 1 MTP layer
KV cache / token (bf16): 11 KiB · Very low
License: MIT License
Released: August 26, 2026

Parameters active per token

5.6%

18B of 320B weights run for each token.

KV cache per token (bf16)

11 KiB

Smaller than 90% of models in the atlas. How it’s calculated

AA Intelligence Index v5.0

Source

27.8

Knowledge Work: 43.7
Coding: 23.7
Science: 12.5
General: 30.9

Architecture concepts

Sources & resources

config.json License Tech report Original gallery card

Compare with… JSON

Use the ← and → keys to step through releases.

Similar architectures

Ling 3.1 Flash (560B)
InclusionAI, released 2026-10-13. Hybrid MoE decoder, 560B total, 25B active, 1M context. Attention: 7:1 Kimi Delta Attention and gated MLA.
Ling 3.0 Flash (124B)
InclusionAI, released 2026-08-04. Hybrid MoE decoder, 124B total, 5.1B active, 256K context. Attention: 5:1 Kimi Delta Attention and gated MLA.
Kimi K3 (2.8T)
Moonshot AI, released 2026-07-27. Hybrid MoE decoder, 2.8T total, 104B active, 1M context. Attention: 3:1 Kimi Delta Attention and Gated MLA with NoPE.
Ember-1 (2.8T)
Fireworks AI, released 2026-09-23. Hybrid MoE decoder, 2.8T total, 104B active, 1M context. Attention: 3:1 Kimi Delta Attention and Gated MLA with NoPE.