Benchmarks / SciHorizon Self-Evolution Leaderboard
SciHorizon Self-Evolution Leaderboard
What SciHorizon Self-Evolution Leaderboard measures, in its makers' own words, and the claim on the rsi.watch ledger that cites it. We do not list scores; its own page has those.
- What it measures
- “Across multi-disciplinary, multi-type scientific tasks, we systematically evaluate how LLM-based agents achieve self-evolution through memory updates, harness optimization and improver iteration, study how their capabilities transfer across disciplines and task types on independent test tasks, and compare the effectiveness and transferability of different self-evolution mechanisms.”
- Tasks
- “Each agent first evolves on evolution tasks, then its state is frozen and it is tested on test tasks that do not overlap the evolution tasks; testing never writes back to the agent's state.”
- How it is scored
- “Score of one run, scaled against the reference solution to a range of 0–1; 0.5 matches the reference solution; higher scores are better.”
Before comparing numbers
- The main track tests on 20 held-out modeling targets across four domains: physics, life sciences, earth & space and chemistry. A cross-type track evolves on analysis tasks. A cross-domain track, evolving on one discipline, is marked as coming soon and has no results yet.
- Gain is the mean per-task percentage change from the agent's initial state. A run counts only if it is a valid submission within 4 times the reference solution's distance from the optimum.
- All agents use one base model, which the pages we saved do not name. The technical report is marked "Stay tuned".
On the ledger
- SciHorizon (Computer Network Information Center, Chinese Academy of Sciences): SciHorizon RSI leaderboard 2026-10-26On the main track, in the leaderboard data as fetched on 2026-10-29: EvoScientist (memory and skill library) gains 1.42%, with 9 wins and 11 losses on 20 tasks; HyperAgents (meta-agent rewrites the task agent) -1.64%; SIA-H (harness self-improvement) -3.34%.
Release date of Edition 1 in the leaderboard page's own code.
Other benchmarks the ledger cites Aider Polyglot, KernelBench, RSI-Inference Bench, SWE-bench Verified, Terminal-Bench 2.0