Model
Model explorer

DeepSeekMoE 16B

OPEN
DeepSeek · DeepSeekMoE family · released Jan 11, 2024

Introduced the DeepSeekMoE fine-grained + shared-expert architecture; matched LLaMA2 7B at roughly 40% of the compute.

ReasoningCodingVisionFunction callingTool useAgentic
-116.2
Elo · rank #429
Parameters
16.4B
Active params
2.8B (MoE)
Context
4K tokens
Architecture
MoE — 2 shared + 64 fine-grained routed experts (2 shared + 6 routed active/token), 4K context
License
DeepSeek Model License (custom, commercial use permitted)
Languages
2+
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning49.8%#126
best: Llama 3.1 405B · 96.9%
ARC-EasyReasoning68.1%#52
best: Phi-3-medium (14B) · 97.7%
C-EvalKnowledge40.6%#52
best: Qwen3.6-Plus · 93.3%
CMMLUKnowledge42.5%#30
best: Doubao-1.5-Pro · 90.9%
GSM8KMath18.8%#165
best: Llama 3.1 405B · 96.8%
HellaSwagReasoning77.1%#98
best: Claude 3 Opus · 95.4%
HumanEvalCoding26.8%#161
best: Claude Opus 4.5 · 99.4%
MATH-500Math4.3%#198
best: GPT-5 · 99.4%
MBPPCoding39.2%#87
best: Llama-3.3-Nemotron-Super-49B v1 (Reasoning On) · 91.3%
MMLUKnowledge45.0%#248
best: OpenAI o3 · 92.9%
PIQAReasoning80.2%#41
best: GPT-4o mini · 93.1%
RACE-HReasoning46.4%#18
best: Claude 3 Opus · 92.9%
TriviaQAKnowledge64.8%#29
best: Sarvam-1 (2B) · 90.6%
WinoGrandeReasoning70.2%#100
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
10 GB
VRAM @ FP16
33 GB
Fits on (Q4)
RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Fits comfortably within a 24GB RTX 4090 at Q4 (~10GB); no specific verified 4090 llama.cpp tokens/sec benchmark was found.
Quantizations
GGUF
Fine-tune it
Conditional / custom
QLoRA12.3 GB1× RTX 4070 Ti 16GB
LoRA36.1 GB1× A100 80GB
Full fine-tune264.4 GB2× H200 141GB
QLoRA SFT on ~10k samples ≈ $1.02 (1× RTX 4070 Ti 16GB)
Plan a fine-tune →
DeepSeekMoE family
Elo progression across releases
DeepSeekMoE 16BJan 2024-116.2
Resources
HFdeepseek-ai/deepseek-moe-16b-baseGHdeepseek-ai/DeepSeek-MoEAPIAvailable via DeepSeek API
API price weights · each benchmark row carries its own source badge (see methodology)