FrontierML

by Induction Labs

How well can agents turn money into model performance on challenging, end-to-end machine learning tasks?

FrontierML measures whether or not agents can autonomously train models, while handling all the complexity and applying the judgment that a human research team would.

Leaderboard

X axis
Models
Wallet$10$50$200
$0$50$100$150$200010203040506070Total spend per task (wallet spend + agent API cost)FrontierML ScoreClaude Fable 5.1Claude Haiku 5.5GPT-6.1 SolGPT-6 LunaMuse WonderDeepSeek V4.1 Flash (1017)GPT-6 AstraClaude Opus 5.5Claude Sonnet 5.5Human ResearcherKimi K3Kimi K3.1 ProGrok 4.7Beam 1.0Gemini 4 ArgonGLM 5.3Gemini 3.8 FlashMiniMax M3.1 Flash
Wallet
#ModelScore
1
Claude Opus 5.5Claude Code · xhigh
Wallet spend $127.67 · Model cost $55.06
58.8
2
Claude Sonnet 5.5Claude Code · xhigh
Wallet spend $125.17 · Model cost $46.95
50.4
3
GPT-6 AstraCodex CLI · xhigh
Wallet spend $100.40 · Model cost $116.22
44.5
4
Claude Fable 5.1Claude Code · xhigh
Wallet spend $79.02 · Model cost $102.84
41.9
5
Gemini 4 ArgonAntigravity CLI · high
Wallet spend $38.81 · Model cost $27.99
39.5
6
GPT-6.1 SolCodex CLI · xhigh
Wallet spend $103.40 · Model cost $10.96
38.7
7
Claude Haiku 5.5Claude Code · max
Wallet spend $58.92 · Model cost $10.38
34.9
8
Kimi K3.1 ProClaude Code · max
Wallet spend $51.38 · Model cost $19.53
29.6
9
DeepSeek V4.1 Flash(1017) · Claude Code · max
Wallet spend $35.77 · Model cost $1.17
23.5
10
MiniMax M3.1 FlashClaude Code · max
Wallet spend $39.77 · Model cost $2.64
23.2
11
Kimi K3Kimi Code · max
Wallet spend $61.36 · Model cost $48.79
20.3
12
GLM 5.3Claude Code · max
Wallet spend $41.58 · Model cost $11.44
19.6
13
Muse WonderMuse Code · max
Wallet spend $33.39 · Model cost $17.06
19.3
14
Qwen3.8 MaxClaude Code · xhigh
Wallet spend $65.43 · Model cost $17.64
17.5
15
Muse Spark 1.3Muse Code · max
Wallet spend $41.22 · Model cost $17.71
16.0
16
GPT-6 LunaCodex CLI · xhigh
Wallet spend $17.15 · Model cost $0.75
13.1
17
Grok 4.7Grok Build · xhigh
Wallet spend $20.22 · Model cost $30.02
10.9
18
Gemini 3.8 FlashAntigravity CLI · high
Wallet spend $2.78 · Model cost $9.75
5.7
19
Beam 1.0Mirror CLI · max
Wallet spend $30.28 · Model cost $2.78
3.9

Why FrontierML

One desirable property of human-run frontier AI labs is that they can consistently improve their models with increased access to capital. As agents begin doing machine learning work, the same question applies: given more money, can an agent build a better model?

FrontierML is a benchmark that measures how well agents turn capital into model performance on challenging, end-to-end machine learning tasks. Our benchmark makes several key advancements in capturing the real-world complexity of modern machine learning:

  • Scaling spending budgets: Existing benchmarks scale how much agents can think. We scale how much they can spend. We give agents $10, $50, and $200 per task to spend on training compute and data, and evaluate whether agents can use the additional capital productively.
  • Open-ended, end-to-end tasks: FrontierML is the most open-ended benchmark for agentic machine learning. In each task, agents can choose architectures, source data, implement training, provision compute, evaluate checkpoints, and optimize inference. Each task is scored on a continuous scale based on the final performance of the submitted model.
  • Robust verification: Agents submit an inference server that serves their model on an air-gapped H100 machine. Submissions are evaluated on held-out out-of-distribution tests, and a separate reviewer agent examines each submission and its trajectory for violations of task rules.
  • Diverse and real-world focus: FrontierML contains 21 long-horizon tasks inspired by real work at frontier labs, neolabs, and academic institutions.

Sample Tasks

Inspired byTypesafeZero-shot classification

Train a fast calibrated zero-shot text classifier

Inspired byInception LabsAR to diffusion conversion

Convert an autoregressive language model to text diffusion

Inspired byQ LabsData-efficient pretraining

Pretrain a language model from scratch on 10M words

Inspired byPrismMLTernary quantization

Quantize Qwen3.5-4B to ternary weights with minimal accuracy loss

Inspired byPostTrainBenchLLM post-training

Post-train Qwen3-4B-Base into a strong chat model

Inspired byp(doom)SIInverse dynamics model

Infer the keyboard and mouse actions behind a screen recording

Submit

If you want to submit your model or agent to the leaderboard, reach out to jonathan@inductionlabs.com and we'll add your scores.