Agents' Last Exam v1.1
At a glance
122
55
sub-industries covered
13
40+
academic institutions
Leaderboard
| Rank | Model | Effort | Harness | Pass Rate | Score | Runtime | Output Tokens |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | ALE-Claw |
45.9% |
77.4% | 156h 47m | 10.2M |
| 2 | Kimi 3.1 Pro | Max | Kimi Code |
43.4% |
74.4% | 61h 26m | 8.5M |
| 3 | Opus 5.5 | Max | Claude Code |
43.4% |
71.6% | 134h 17m | 21.5M |
| 4 | GPT-6 Astra | Codex |
42.6% |
73.1% | 100h 13m | 5.1M | |
| 5 | GPT-6.1 Terra | Max | Codex |
41.8% |
69.8% | 124h 36m | 5.5M |
| 6 | Gemini 4 Argon | High | ALE-Claw |
40.2% |
69% | 87h 4m | 6.5M |
| 7 | Muse Wonder 1.0 | High | Muse Code |
38.5% |
68.4% | 116h 27m | 4.7M |
| 8 | Kimi K3 | Max | Kimi Code |
35.2% |
63% | 107h 48m | 7.5M |
| 9 | GPT-5.6 Sol | Max | Codex |
30.3% |
48.4% | 78h 22m | 3.1M |
| 10 | MiniMax M3.1 Flash | Max | Codex |
29.5% |
58.9% | 117h 37m | 17.4M |
| 11 | MiniMax M3.1 Flash | Max | Claude Code |
29.5% |
57.9% | 122h 12m | 18.1M |
| 12 | GLM 5.3 | Max | ZCode |
26.2% |
54.4% | 111h 10m | 17.4M |
| 13 | GPT-5.6 Luna | Off | Codex |
5.7% |
26.5% | 40h 35m | 1.1M |
Key takeaways
Pass rate vs output tokens
Hover any dot for the harness, model, and exact metrics.
Best result by harness
Top pass rate each harness achieved on this evaluation split, regardless of which model it ran.
Highest Pass Rate observed per harness on this split. Bars are not normalized for sample size; some harnesses were tested across many model variants, others on only one or two.
Industry coverage
Breakdown of the 122 v1.1 tasks by industry. View all tasks on GitHub →
- Business & finance 19
- Computing & math 19
- Health & medicine 19
- Engineering 16
- Physical sciences 14
- Life sciences 12
- Visual media 11
- Psychology & neuroscience 3
- Education & information 2
- Legal 2
- Transport & safety 2
- Other 2
- Social sciences 1
Motion & VFX
3D modeling
Game development
Mold flow analysis
Architectural modeling
Brain imaging
Sample tasks
business_finance
sec_10k_financial_parsing
Parse a SEC 10-K filing into a structured financial schema. Multi-step extraction, table normalization, and cross-reference validation against the original document.
business finance
Health medicine
life sciences
computing math
visual media
Methodology
Metrics
Pass Rate — fraction of tasks the agent fully completed (strict success). Score — average graded outcome across all tasks, including partial credit. Both computed by deterministic graders against hidden references.
Every ~6 months, a new public subset releases with fresh instances. Private tasks rotate into the public pool, retired public tasks rotate out, and held-out private tasks score the official leaderboard, to limit benchmark leakage.
Reference Harnesses
Resources
Behind the benchmark
Agents’ Last Exam (ALE) measures agent performance on long-horizon, economically valuable tasks with verifiable outcomes. The benchmark covers non-physical industries defined with reference to O*NET / SOC 2018 (the U.S. federal occupational taxonomy), spanning 55 sub-industries.
ALE v1.1 ships 122 reference tasks across 55 sub-industries.
Acknowledgments
Agents’ Last Exam is co-led by UC Berkeley RDI and the RDI Foundation, with funding support and contributions from Snorkel AI via the Open Benchmarks Grants program.
