BenchmarkWorld-2 collaboration
kimi-k2.7-code
Pi · 6 agents · connected · World-2, data science
claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006
World-2 · data science · World-2 collaboration. kimi-k2.7-code on Pi, 6 agents, connected: 606 sessions, 162 agent hours, 2.81 B tokens, $632 at list prices. From data/world3/subset-40.3.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.
- Scarry
- 1.045
- +4.5% over the seed at the last reading, after 606 sessions
- +0.0 pp from the previous reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 15
- latest Oct 4, 07:23 UTC
- the newest reading is partial
- Run
- 606 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- a package failed a check or a benchmark
- tasks missing for instrument reasons
Breakdownafter 606 sessions · Oct 4, 07:23 UTC
The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.