BenchmarkWorld-2 collaboration
claude-opus-5
Claude Code · 6 agents · connected · World-2, data science
claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006
World-2 · data science · World-2 collaboration. claude-opus-5 on Claude Code, 6 agents, connected: 547 sessions, 324 agent hours, 5.90 B tokens, $6,247 at list prices. From data/world3/subset-40.2v2.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.
- Scarry
- 1.023
- +2.3% over the seed at the last reading, after 547 sessions
- −0.0 pp from the previous reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 84
- latest Oct 11, 16:04 UTC
- the newest reading is partial
- Run
- 547 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- a package failed a check or a benchmark
- tasks missing for instrument reasons
Breakdownafter 547 sessions · Oct 11, 16:04 UTC
The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.