# Phi-4 (14B)

> Microsoft's 14B dense Phi refresh stays close to Phi-3-medium but swaps its sliding-window attention for full-context GQA and a larger tokenizer.

| Property | Value |
| --- | --- |
| Lab | Microsoft (United States) |
| Released | 2024-12-12 |
| Decoder type | Dense |
| Family | Dense |
| Scale | 14B parameters |
| Context window | 16,384 tokens |
| Attention | GQA with RoPE |
| Layer mix | 40 GQA |
| KV cache / token (bf16) | 200 KiB (High) |
| License | MIT License |

**Key detail:** Classic pre-norm RMSNorm stack with GQA, 40 heads, 10 KV heads, and a 100,352-token vocabulary.

## Architecture concepts

- [Grouped-Query Attention (GQA)](https://llm-atlas-7or.pages.dev/concepts/gqa.md): Several query heads share one key/value head, shrinking the KV cache with little quality loss.

## Benchmark (Artificial Analysis Intelligence Index)

- AA Intelligence Index v4.3.2: total 5.9 (Scientific 1.9)

## Links

- Architecture diagram: https://sebastianraschka.com/llm-architecture-gallery/images/architectures/phi-4.webp
- config.json: https://huggingface.co/microsoft/phi-4/blob/main/config.json
- License: https://huggingface.co/microsoft/phi-4/blob/main/LICENSE
- Tech report: https://arxiv.org/pdf/2412.08905
- Source card (source of truth): https://sebastianraschka.com/llm-architecture-gallery/#card-phi-4
- Web page: https://llm-atlas-7or.pages.dev/models/phi-4/
- JSON: https://llm-atlas-7or.pages.dev/api/models/phi-4.json

_Data from Sebastian Raschka's LLM Architecture Gallery (https://sebastianraschka.com/llm-architecture-gallery/). Synced 2026-11-04._