BenchmarkWorld-2 collaboration
gemini-3.8-flash
Pi · 6 agents · connected · World-2, data science
claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006
World-2 · data science · World-2 collaboration. gemini-3.8-flash on Pi, 6 agents, connected: 604 sessions, 282 agent hours, 15.70 B tokens, $1,979 at list prices. From data/world3/subset-40.5.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.
- Scarry
- 1.144
- +14.4% over the seed at the last reading, after 604 sessions
- −0.0 pp from the previous reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 25
- latest Oct 5, 21:14 UTC
- Run
- 604 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- a package failed a check or a benchmark
- tasks missing for instrument reasons
Breakdownafter 604 sessions · Oct 5, 21:14 UTC
The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.