Self-improving AI / R²Orch
R²Orch
“we propose R2Orch, a new paradigm of Recursive Self-Play Orchestration: an inner loop of self-play that improves the orchestrator on self-chosen tasks, and an outer loop that improves how the inner loop improves.”
R²Orch is one of the projects on the self-improving AI map. Below is what it changes about itself, whether the change lasts, and whether it improves the improver, each with the line from its own source.
Where it sits
Columns: what it changes, from the surface to the deepest change. Rows: whether the change outlasts the run. A number is how many projects sit in that square. See it on the 3D map
- What it changes
- Its model weights“failures are distilled into skills, skills into parameters”
- Does the change last
- Becomes the next version“the two loops form one cycle: failures are distilled into skills, skills into parameters, and withdrawals lead the Proposer to new failures”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“An LLM editor revises skills, the least helpful first, and distills new ones from groups in which both parts fail, while withdrawals follow Eq. (14).”
- Who sets the goals
- people set the aim, the system picks tasks“Each candidate list yields two distinct picks”
- How it was tested
- tested on held-out tasks“is best on all twelve benchmarks against six families of baselines, and its lead holds on the held-out half at zero skills”
Date is arXiv v1. What the outer loop changes is the skill set: a fixed Bayes rule withdraws, admits, or restores skills, and an LLM editor revises skills and distills new ones from groups where both parts fail. The paper's case for the outer loop improving the inner loop is that this skill set also sets the Proposer's reward and the Director's training context; the reward formula, the withdrawal rule, and the update steps stay as written in Algorithm 1, so recursion is task. Director, Proposer, and Workers start from Qwen3.5-9B; the Director and Proposer are trained (Table 4 lists LoRA rank 64), and the Workers are frozen. The trained Proposer learns to skip solved candidates, but its reward and update rule stay fixed. The Proposer picks from candidate lists of tasks drawn from the six training benchmarks. The initial skills are customized by hand for each dataset.
Does it count as RSI?
It depends on the definition. Under the four used on What counts as RSI, by what its own source says:
- Loose (Eliezer Yudkowsky, Intelligence Explosion Microeconomics (2013)): counts
- Durable (Ren et al., self-improvement survey (2026)): counts
- Strict (Roman Yampolskiy, From Seed AI to Technological Singularity (2015)): does not count
- Autonomous (Anthropic, When AI builds itself): does not count
How it got here
Earlier work in this catalog that R²Orch cites, oldest first. Under each: where that work sits on the map, then where R²Orch sits (changes in bold), and what R²Orch's own paper says about it. Its whole family on the citation map
Cites Darwin Gödel Machine 2025, Hyperagents 2026, Weco AIDE² 2026
In the same square
CoEvo-RSI, ECG-RSI, MTA-Zero, Sakana MASS (Multi-Agent Self-Supervision), Self-HPO, Self-Rewarding Language Models, SPADE, STaR: Self-Taught Reasoner, Terminal-RSI