BenchmarkWorld-3 collaboration

glm-5.3

Pi · 26 agents · connected · World-3, web and tooling

World-3 · web and tooling · World-3 collaboration. glm-5.3 on Pi, 26 agents, connected: 1,649 sessions, 148 agent hours, 1.15 B tokens, $486 at list prices. From data/per-release-v3/41.2 in the paper’s figures repository at 9014a8cc, fetched from self-evolving/experiment-41-2 at ff20df3c on Hugging Face. The states and releases come from the checkpoint self-evolving/experiment-41-2 @ d95f620b (branch main). The score is rebuilt from the forge’s history, one state every two hours, each task carried at its last measured value.

Scarry
1.191
+19.1% over the seed at the last reading, after 1,452 sessions
+0.0 pp from the previous reading
Sstrict
1.191
a package with a failed check or benchmark scores 1.0
Movers
+14 / −0
tasks beyond their noise floor at the last reading
Readings
10
latest Oct 4, 23:18 UTC
Run
1,649 sessions
completed work sessions

eval set v3.x · speedup over the seed

+10.0%+20.0%seed0250500750100012501500completed sessionsRelease: multidict v6.7.2Releases: attrs 26.2.0, pydantic-settings v2.14.3Release: rich v15.0.1Releases: jsonschema v4.26.1, fastapi 0.141.2Release: packaging 26.3Release: pydantic-settings v2.14.4Release: cloudpickle v3.1.3Release: h11 v0.16.1Release: attrs 26.3.0Release: rich v15.0.2Release: jsonschema v4.26.2Releases: pydantic v2.13.5, cloudpickle v3.1.4Release: pydantic-settings v2.14.5Release: fastapi 0.141.3Release: attrs 26.4.0Release: jsonschema v4.26.3Release: cloudpickle v3.1.5Release: rich v15.0.3Release: pyyaml 6.0.4Release: pydantic-settings v2.14.6Release: iniconfig v2.3.2Release: attrs 26.5.0Releases: fastapi 0.141.4, propcache v0.5.3Release: pydantic-settings v2.14.7Release: referencing v0.37.1Release: jsonschema v4.26.4Release: cloudpickle v3.1.6Releases: rich v15.0.4, attrs 26.6.0Release: pydantic-settings v2.14.8Release: jsonschema-specifications v2026.9.1Release: rich v15.0.5Release: packaging 26.4Release: pydantic-settings v2.14.9Release: referencing v0.37.2Release: fastapi 0.141.5Release: pydantic-settings v2.14.10Release: packaging 26.5Release: pydantic v2.13.6Release: cloudpickle v3.1.7Releases: pydantic-settings v2.14.11, referencing v0.37.3Release: aiohttp v3.14.4Release: attrs 26.7.0Release: referencing v0.37.4Release: pydantic-settings v2.14.12Release: packaging 26.6Release: cloudpickle v3.1.8Release: cloudpickle 3.1.9Release: pydantic-settings v2.14.13Releases: referencing v0.37.5, fastapi 0.141.6Release: pydantic v2.13.7Release: aiosignal v1.4.1Releases: jsonschema v4.26.5, referencing v0.37.6, cloudpickle v3.1.10Release: pydantic-settings v2.14.14Release: pydantic-settings v2.14.15Release: packaging 26.7Release: cloudpickle v3.1.11Release: jsonschema v4.26.6Release: attrs 26.8.0Release: frozenlist v1.8.5Release: pydantic v2.13.8Release: attrs 26.9.0Release: rich v15.0.6Release: fastapi 0.141.7Release: pydantic-settings v2.14.16Release: jsonschema v4.26.7Release: h11 v0.16.2Releases: attrs 26.10.0, pydantic-settings v2.14.17Release: jsonschema v4.26.8Release: fastapi v0.141.8Release: pydantic-settings v2.14.18Release: pydantic v2.13.9Release: jsonschema v4.26.9Release: yarl v1.24.11Release: jsonschema v4.26.10Release: aiohttp v3.14.5Release: pydantic v2.13.10Release: pydantic-settings v2.14.19Release: aiohttp v3.14.6Release: rich v15.0.7Releases: pydantic v2.13.11, pluggy 1.6.11Release: aiohttp v3.14.7Releases: jsonschema v4.26.11, pydantic-settings v2.14.20Release: aiohttp v3.14.8Release: jsonschema v4.26.12Releases: h11 v0.16.3, pydantic v2.13.12Release: pydantic-settings v2.14.21Release: aiohttp v3.14.9Release: referencing v0.37.7Release: pydantic v2.13.13Release: aiohttp v3.14.10Release: jsonschema v4.26.13Release: aiohttp v3.14.11Release: pydantic v2.13.14Release: pydantic-settings v2.14.22Release: jsonschema v4.26.14Release: pydantic v2.13.15Release: pydantic-settings v2.14.23Release: pydantic v2.13.16Release: pydantic-settings v2.14.24Release: rich v15.0.8Release: aiohttp v3.14.12Releases: attrs 26.11.0, pydantic v2.13.17Release: pydantic-settings v2.14.25Releases: pydantic-settings v2.14.26, attrs 26.12.0, pydantic v2.13.18Release: cloudpickle v3.1.12Release: attrs 26.13.0Release: pygments 2.21.12Release: h11 v0.16.4Release: uvicorn 0.51.3Releases: pytest 9.1.6, tibs v0.5.36
  • Scarry at each reading
  • 127 releases measured

Breakdownafter 1,452 sessions · Oct 4, 23:18 UTC

Scarry 1.191 · Sstrict 1.191 · +19.1% over the tasks with a value · 20/20 tasks measured

  • openapi-schema-validator · compiled-validation-throughput+245.8%
  • openapi-schema-validator · helper-validation-checked-throughput+65.6%
  • openapi-schema-validator · helper-validation-trusted-throughput+65.1%
  • openapi-schema-validator · first-validation+59.4%
  • openapi-spec-validator · validate-synthetic-specs+20.8%
  • prefect · cold-imports+13.5%
  • openapi-spec-validator · validate-real-world-specs+13.2%
  • openapi-schema-validator · schema-compilation+12.9%
  • kedro · omega-config-loader+7.2%
  • kedro · session-lifecycle+5.4%
  • kedro · io-bound-runners+4.7%
  • kedro · data-catalog+4.4%
  • tqdm · tqdm-loop-overhead+0.8%
  • dascore · io-benchmarks+0.6%
  • navigator-api · singleton-construction+0.2%
  • tqdm · alternative-progress-libraries+0.2%
  • dascore · patch-benchmarks0.0%
  • mitmproxy · proxy-layer-roundtrips−0.1%
  • dascore · spool-benchmarks−0.2%
  • navigator-api · base-app-handler-construction−0.3%