Benchmarks

SWE-bench Pro

Codingunit % · normalized over [0, 85]

Harder, held-out variant of SWE-bench Verified.

#ModelSourceScoreNormalized

Score distribution

43 tracked results across the normalization window

085

Score vs. parameters

Open-weights models, log-x params