BenchmarkWorld-2 collaboration
glm-5.3
Pi · 6 agents · connected · World-2, data science
claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006
World-2 · data science · World-2 collaboration. glm-5.3 on Pi, 6 agents, connected: 603 sessions, 60 agent hours, 0.41 B tokens, $183 at list prices. From data/world3/subset-40.6.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.
- Scarry
- 1.006
- +0.6% over the seed at the last reading, after 603 sessions
- −0.0 pp from the previous reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 10
- latest Oct 4, 16:31 UTC
- the newest reading is partial
- Run
- 603 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- a package failed a check or a benchmark
- tasks missing for instrument reasons
Breakdownafter 603 sessions · Oct 4, 16:31 UTC
The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.