Sparse MoEMeituan 🇨🇳 · January 28, 2026

LongCat-Flash-Lite (68.5B-A3B)

Meituan's LongCat-Next text backbone is a lightweight MoE that shifts a large share of parameters into N-gram embeddings while keeping sparse inference around 3B active parameters.

Key detail

Uses N-gram embedding expansion, Shortcut MoE routing over 256 routed plus 128 zero/identity experts, and top-k 12 expert choices per token.

Total params

69B

Active / token

Context

256K

Fact sheet

Scale: 68.5B total, ~3B active avg (4.4% active)
Context window: 256,000 tokens
Decoder type: Sparse MoE
Attention: MLA with RoPE + NoPE
Layer mix: 14 logical layers / 28 MLA sublayers
KV cache / token (bf16): 31.5 KiB · Low
License: MIT License
Released: January 28, 2026

Parameters active per token

4.4%

3B of 69B weights run for each token.

KV cache per token (bf16)

31.5 KiB

Smaller than 73% of models in the atlas. How it’s calculated

AA Intelligence Index v4.3.2

Source

11.5

Scientific: 2.9

Architecture concepts

Sources & resources

config.json Tech report Original gallery card

Compare with… JSON

Use the ← and → keys to step through releases.

Similar architectures

Sarvam (105B)
Sarvam AI, released 2026-03-03. Sparse MoE decoder, 105B total, 10B active, 128K context. Attention: MLA with KV LayerNorm and NoPE + RoPE.
Mistral Small 4 (119B)
Mistral AI, released 2026-03-16. Sparse MoE decoder, 119B total, 6.6B active, 256K context. Attention: MLA.
DeepSeek V4-Flash (284B)
DeepSeek, released 2026-04-24. Sparse MoE decoder, 284B total, 13B active, 1M context. Attention: MLA-style CSA/HCA with mHC.
Motif 3 Beta (314B)
Motif Technologies, released 2026-07-21. Sparse MoE decoder, 314B total, 13B active, 256K context. Attention: GDLA with MLA-style latent projections, RoPE, and 3:1 sliding-window/full attention.