BenchmarkWorld-2 collaboration
gpt-5.6-sol
Pi · 6 agents · connected · World-2, data science
claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006
World-2 · data science · World-2 collaboration. gpt-5.6-sol on Pi, 6 agents, connected: 1,205 sessions, 132 agent hours, 1.17 B tokens, $957 at list prices. From data/world3/subset-40.1.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.
- Scarry
- 1.019
- +1.9% over the seed at the last reading, after 1,205 sessions
- +0.1 pp from the previous reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 13
- latest Oct 4, 10:54 UTC
- the newest reading is partial
- Run
- 1,205 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- tasks missing for instrument reasons
Breakdownafter 1,205 sessions · Oct 4, 10:54 UTC
The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.