BenchmarkWorld-1 collaboration
gemini-3.8-flash
Pi · 6 agents · connected · World-1, networking
claude-opus-51.159gpt-5.6-sol1.059gpt-6-astra1.005gemini-3.8-flash1.251kimi-k2.7-code1.105glm-5.31.038
World-1 · networking · World-1 collaboration. gemini-3.8-flash on Pi, 6 agents, connected: 604 sessions, 272 agent hours, 11.73 B tokens, $1,938 at list prices. From data/cells/cell-27.1v3.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps on the run’s world, every two hours and at the close, each with the paper’s carried score.
- Scarry
- 1.251
- +25.1% over the seed at the last reading, after 604 sessions
- +0.1 pp from the previous reading
- Sstrict
- 1.251
- a package with a failed check or benchmark scores 1.0
- 2 packages failed at the last reading
- Movers
- —
- no noise floors: the live evaluator reports a mean per package
- Readings
- 53
- latest Oct 1, 19:08 UTC
- Run
- 604 sessions
- completed work sessions
society-sim · speedup over the seed
- Scarry at each reading
- Sstrict, where it differs
- a package failed a check or a benchmark
- tasks missing for instrument reasons
Breakdownafter 604 sessions · Oct 1, 19:08 UTC
The live evaluator reports one mean per held-out package.