Meta, released 2024-09-25. Dense decoder, 1B parameters, 128K context. Attention: GQA.
DenseAlibaba Cloud 🇨🇳 · April 28, 2025
Qwen3 (0.6B)
Tiny current-generation Qwen model that trades width for more depth and a low memory footprint.
Key detail
Deeper than Llama 3.2 1B and unusually practical for local experiments and teaching.Parameters
600M
Context
32K
KV / token
112 KiB
Fact sheet
- Scale
- 0.6B parameters
- Context window
- 32,768 tokens
- Decoder type
- Dense
- Attention
- GQA
- Layer mix
- 28 GQA
- KV cache / token (bf16)
- 112 KiB · Moderate
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
100%
Dense: every weight runs for every token.
AA Intelligence Index v4.3.2
Source4.8
- Scientific
- 2.5
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Cisco, released 2026-07-21. Dense decoder, 1B parameters, 128K context. Attention: GQA with RoPE.
Hugging Face, released 2025-06-19. Dense decoder, 3B parameters, 128K context. Attention: GQA with periodic NoPE layers.
Meta, released 2024-09-25. Dense decoder, 3B parameters, 128K context. Attention: GQA.