BenchmarkWorld-3 collaboration

gpt-6-astra

Pi · 26 agents · connected, discovery prompt · World-3, web and tooling

World-3 · web and tooling · World-3 collaboration. gpt-6-astra on Pi, 26 agents, connected, discovery prompt: 1,319 sessions, 97 agent hours, 0.44 B tokens, $1,024 at list prices. From data/per-release-v3/41.5 in the paper’s figures repository at 9014a8cc, fetched from self-evolving/experiment-41-5 at d992015d on Hugging Face. The states and releases come from the checkpoint self-evolving/experiment-41-5 @ c446b3f0 (branch main). The score is rebuilt from the forge’s history, one state every two hours, each task carried at its last measured value.

Scarry
1.017
+1.7% over the seed at the last reading, after 1,286 sessions
+0.1 pp from the previous reading
Sstrict
1.017
a package with a failed check or benchmark scores 1.0
Movers
+12 / −0
tasks beyond their noise floor at the last reading
Readings
5
latest Sep 24, 13:39 UTC
Run
1,319 sessions
completed work sessions

eval set v3.x · speedup over the seed

+1.0%seed020040060080010001200completed sessionsRelease: jsonschema v4.26.1Release: fastapi 0.141.2Release: attrs 26.2.0Release: jsonschema-specifications v2026.9.1Release: referencing v0.37.1Release: pyyaml 6.0.4Release: h11 v0.16.1Release: pydantic-settings v2.14.3Release: packaging 26.3Releases: cloudpickle v3.1.3, frozenlist v1.8.2Release: iniconfig v2.3.3Releases: fastapi 0.141.3, rich v15.0.1Release: pydantic v2.13.5Release: pydantic-settings v2.14.4Release: pluggy 1.6.1Release: h11 v0.16.2Release: multidict v6.7.3Release: packaging 26.4Release: pydantic-settings v2.14.5Release: attrs 26.3.0Release: fastapi 0.141.4Release: h11 v0.16.3Release: jsonschema v4.26.2Release: pydantic-settings v2.14.6Release: rich v15.0.2Release: pyyaml 6.0.5Release: h11 v0.16.4Release: packaging 26.5Release: attrs 26.4.0Release: attrs 26.5.0Release: jsonschema v4.26.3Release: pydantic-settings v2.14.7Release: rich 15.0.3Release: packaging 26.6Release: attrs 26.6.0Release: fastapi 0.141.5Release: pydantic v2.13.6Release: cloudpickle v3.1.4Release: h11 v0.16.5Release: pydantic v2.13.7Release: jsonschema v4.26.4Release: rich 15.0.4Release: pyyaml 6.0.6Release: packaging 26.7Releases: fastapi 0.141.6, rich 15.0.5Release: pydantic-settings v2.14.8Release: rich 15.0.6Release: attrs 26.7.0Release: fastapi 0.141.7Release: yarl v1.24.13Release: cloudpickle v3.1.5Release: pydantic v2.13.8Release: pygments 2.20.5Release: packaging 26.8Release: jsonschema v4.26.5Release: fastapi 0.141.8Release: pyyaml 6.0.7Release: pydantic-settings v2.14.9Release: pytest 9.1.6Release: rich 15.0.7Release: tibs v0.5.22
  • Scarry at each reading
  • 64 releases measured

Breakdownafter 1,286 sessions · Sep 24, 13:39 UTC

Scarry 1.017 · Sstrict 1.017 · +1.7% over the tasks with a value · 20/20 tasks measured

  • openapi-schema-validator · compiled-validation-throughput+7.3%
  • kedro · omega-config-loader+7.0%
  • openapi-schema-validator · first-validation+5.3%
  • openapi-schema-validator · helper-validation-checked-throughput+4.0%
  • openapi-schema-validator · helper-validation-trusted-throughput+4.0%
  • kedro · data-catalog+1.3%
  • openapi-spec-validator · validate-synthetic-specs+1.1%
  • prefect · cold-imports+1.0%
  • kedro · session-lifecycle+0.9%
  • kedro · io-bound-runners+0.9%
  • openapi-spec-validator · validate-real-world-specs+0.7%
  • dascore · io-benchmarks+0.7%
  • navigator-api · base-app-handler-construction+0.4%
  • dascore · spool-benchmarks+0.3%
  • navigator-api · singleton-construction+0.2%
  • mitmproxy · proxy-layer-roundtrips+0.2%
  • tqdm · tqdm-loop-overhead+0.1%
  • dascore · patch-benchmarks0.0%
  • tqdm · alternative-progress-libraries0.0%
  • openapi-schema-validator · schema-compilation−0.2%