# Ouro-Thinking (2.6B)

> ByteDance's dense LoopLM reuses one 48-layer MHA stack across four recurrent passes and trains an exit gate over the intermediate representations.

| Property | Value |
| --- | --- |
| Lab | ByteDance (China) |
| Released | 2025-10-29 |
| Decoder type | Dense |
| Family | Dense |
| Scale | 2.6B parameters |
| Context window | 65,536 tokens |
| Attention | MHA with RoPE |
| Layer mix | 48 shared MHA layers × 4 passes |
| KV cache / token (bf16) | 1.5 MiB (Very high) |
| License | Apache License 2.0 |

**Key detail:** Reuses the same 48-layer MHA stack across four passes, yielding 192 block applications, with sandwich RMSNorm and a learned exit gate.

## Architecture concepts

- [Looped Transformer (Looped depth)](https://llm-atlas-7or.pages.dev/concepts/looped-depth-sharing.md): The same blocks are applied several times, reusing weights across depth.
- [Multi-Head Attention (MHA)](https://llm-atlas-7or.pages.dev/concepts/mha.md): The original transformer attention: every head has its own queries, keys and values.

## Links

- Architecture diagram: https://sebastianraschka.com/llm-architecture-gallery/images/architectures/ouro-2-6b.webp
- config.json: https://huggingface.co/ByteDance/Ouro-2.6B-Thinking/blob/main/config.json
- License: https://huggingface.co/ByteDance/Ouro-2.6B-Thinking/blob/main/README.md#license
- Tech report: https://arxiv.org/abs/2510.25741
- Source card (source of truth): https://sebastianraschka.com/llm-architecture-gallery/#card-ouro-thinking-2-6b
- Web page: https://llm-atlas-7or.pages.dev/models/ouro-thinking-2-6b/
- JSON: https://llm-atlas-7or.pages.dev/api/models/ouro-thinking-2-6b.json

_Data from Sebastian Raschka's LLM Architecture Gallery (https://sebastianraschka.com/llm-architecture-gallery/). Synced 2026-11-04._