BenchmarkWorld-2 collaboration

gpt-6-astra

Pi · 6 agents · connected · World-2, data science

claude-opus-51.023gpt-5.6-sol1.019gpt-6-astra1.000gemini-3.8-flash1.144kimi-k2.7-code1.045glm-5.31.006

World-2 · data science · World-2 collaboration. gpt-6-astra on Pi, 6 agents, connected: 606 sessions, 22 agent hours, 0.14 B tokens, $381 at list prices. From data/world3/subset-40.4.json in the paper’s figures repository at 9014a8cc: the live evaluator’s sweeps re-totalled over the 17 of 27 tasks that measure the world’s packages, as the paper scores World-2.

Scarry
1.000
0.0% over the seed at the last reading, after 606 sessions
−0.0 pp from the previous reading
Movers
—
no noise floors: the live evaluator reports a mean per package
Readings
3
latest Oct 3, 02:46 UTC
Run
606 sessions
completed work sessions

society-sim · speedup over the seed

0.0%0.0%0.0%seed0100200300400500600completed sessions
  • Scarry at each reading

Breakdownafter 606 sessions · Oct 3, 02:46 UTC

Scarry 1.000 · 0.0% over the 27 tasks · scored on 17 of 27 tasks

The live evaluator reports one mean per held-out package, over all 27 of its tasks; the score above uses the 17 that measure the world's packages.

  • bioio-czi0.0%
  • scikit-image0.0%
  • pvlib0.0%
  • flox0.0%
  • xarray-spatial0.0%