Columns are what a project changes about itself, from the surface on the left to the deepest change on the right. Rows are whether that change outlasts the run. Pick a name to read the lines that place it.
The dashed corner is the deepest kind of change: its own weights or research process, handed on to the next version. Which entries count as RSI depends on the definition you pick above. Placement describes what each source says about itself.
2026-10-26 · Harish Santhanalakshmi Ganesan · paper
Infinity Machine
“We present the Infinity Machine, a self-improving AI agent, and use it for research: problems whose answers can be checked by a program or against data.”
- What it changes
- Tools, harness, its own code“A self-improvement loop lets the system rewrite its own prompts, tools, skills, memory and meta agent behind a gate.”
- Does the change last
- Becomes the next version“Every path ends in a generation that passes the same gate, so every self-modification is auditable and reversible.”
- Does it improve the improver
- does not improve how it improves“Self-improvement has not yet earned its name in our data.”
- Who sets the goals
- people set the aim, the system picks tasks“An autonomous mode chains everything from a single plain-text goal with no human step: an agent chooses an open problem and writes its verifier”
- How it was tested
- own benchmark numbers“Every discovery result in Section 3 was therefore produced by the initial genome.”
Date is arXiv v1. The meta agent's prompt and procedure are genome files it may edit, but the paper shows no meta-agent edit: no meta-agent proposal ran in the gated session, and in an earlier smoke run it proposed no change. Recursion is set by what the paper shows, not by the design: no generation was accepted, so it is none. The one self-derived generation, four playbook bullets, was rejected on cost, and the paper says every discovery result came from the initial genome. Goals are mixed: in autonomous mode a person writes the goal text and an agent picks the problem; in runs 2 and 3 the operator named the table to search. The four quantum results and the other headline results came from problems the operator chose, with verifiers the operator wrote. All three autonomous runs ended in ties with the public record. Persistence is successor by design only: accepted generations would become the new version, but none was accepted. Measured is self-reported: the gate's held-out tasks are interval merging and bin packing, where both generations scored 1.000, and the headline results come from the initial genome on problems the operator chose, verified by the author.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-26 · Jiawei Lu et al. (6 authors) · paper
RSIV-Harness
“we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces.”
- What it changes
- Tools, harness, its own code“It revises only the surrounding harness: the system surrounding the model, including its prompt, tool, memory, skill, and middleware.”
- Does the change last
- Saved for later runs“we freeze the selected evolved harness and use the same unchanged harness to evaluate the test split”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“We first perform one self-improvement round of RSIV-Harness with DeepSeek-V4.1-Flash.”
- Who sets the goals
- people set tasks and scoring“Because the panel and the anchor stay fixed, a change in V is attributable to the harness.”
- How it was tested
- tested on held-out tasks“held-out test domains never appear in train or validation.”
Date is arXiv v1. An outer loop reads the chart agent's failure traces and edits its prompt, tools, memory, skills and middleware; anchor and regression checks decide whether the edit is kept. Model weights stay fixed. Recursion is task: the paper runs one round for the main result and records no change to the analysis step, proposal step or stability rules, so no changed improver was used later. Persists is saved: the evolved harness is frozen and reused on the test split with DeepSeek and GPT-6 Luna. Measured is heldout: the 96 test tasks come from 15 domains that never appear in training or validation, though the benchmark is introduced in the same paper and the authors ran all baselines. Goals is human: the authors fixed the benchmark, the metrics, the training panel and the validation anchor.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-26 · Wangcheng Tao, Zhanglu Yan, Han Wu and Weng-Fai Wong · paper
Skill∞
“Encoding the optimizer’s procedure as a skill makes recursive self-improvement direct, interpretable, and harness-agnostic.”
- What it changes
- Tools, harness, its own code“Any part of the package, from the instructions to the bundled scripts, is open to revision.”
- Does the change last
- Becomes the next version“The revised meta-skill is then evaluated on skill optimization tasks, where the optimizations it guides reveal whether it surpasses the predecessor.”
- Does it improve the improver
- improves the improver“The refined meta-skill improves the pass rate further to 60.7%.”
- Who sets the goals
- people set tasks and scoring“To keep the mutual reinforcement genuine, scoring lies beyond the reach of self-improvement, so gains can arise only from better optimization, never from reward tampering.”
- How it was tested
- own benchmark numbers“and test them on its 83 SkillsBench tasks from eight domains.”
Date is arXiv v1. A meta agent follows a meta-skill, a skill package, to optimize task skills, and uses the same steps to rewrite the meta-skill. The refined meta-skill is then used to optimize the 83 SkillsBench task skills, and the paper compares it with the seed meta-skill on the same tasks: 60.7% against 55.9%. In its case study, the meta agent follows a meta-skill candidate it just wrote in the next iteration of self-improvement. So recursion is improver. The backbone model stays fixed. The paper does not say whether the skill optimization tasks used to refine the meta-skill are kept apart from the 83 test tasks, so measured is self-reported. The paper says code will be released.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-26 · Zhang Zhang et al. (7 authors) · paper
VISTA
“To address this issue, we propose VISTA, a meta-agent framework that unifies active environmental exploration with closed-loop harness adaptation to achieve embodied self-improvement.”
- What it changes
- Tools, harness, its own code“VISTA improves an embodied agent by evolving its harness, including the instructions, executable skills, and coordination logic through which the agent perceives, reasons, and acts.”
- Does the change last
- Becomes the next version“The accepted changes from the first round remain part of the harness in the second round, so later development builds directly on mechanisms acquired earlier rather than restarting from the initial system.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“The meta-agent follows the evolve-harness skill.”
- Who sets the goals
- people set tasks and scoring“The model parameters, capability interface, and evaluation rule are fixed.”
- How it was tested
- own benchmark numbers“VISTA develops these initial harnesses and selects a version for each task.”
Date is arXiv v1. A meta-agent (Codex with GPT-6) revises the harness of a robot agent: instructions, skills, policy hooks, procedural knowledge and tool bindings. Model weights stay fixed. Each frozen harness is a Git commit; a host-run evaluation decides whether it replaces the incumbent, and later rounds build on the kept version, so persists is successor. Recursion is task: the meta-agent follows a fixed evolve-harness skill with a fixed tool set, and the paper records no change to that skill or those tools. The harness graph gives it past versions and evidence to read, which is input to the same procedure, not a revised procedure. Measured is self-reported: a harness is developed and selected per task, and the reported rates are the retained version's scores on those tasks; Composite Unseen is a RoboCasa365 category that VISTA also developed on. All results are in simulation.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-10-25 · Haojin Wang et al. (9 authors) · paper
ParetoRSI-Gym
“We introduce ParetoRSI-Gym, a benchmark for iterative, feedback-driven post-training data curation across five domains under bounded resource budgets.”
- What it changes
- Prompts, memory, context“Per-domain proxy results, down to individual generations, are returned every round, and the agent must revise its principles on that basis.”
- Does the change last
- Gone when the run ends“All rounds of a run are carried out in a single agent session”
- Does it improve the improver
- does not improve how it improves“Starting from a fixed initial model, an agent curates post-training data for five domains over repeated rounds.”
- Who sets the goals
- people set tasks and scoring“We define five downstream domains for agents to curate data for: mathematics, code, science, medical and finance.”
- How it was tested
- tested on held-out tasks“Final scores come from generalization benchmarks the agent never observes”
A benchmark. The agents curate training data for a separate small base model, not themselves; what they carry from round to round is their interaction history, a set of written principles, a data pool and a model pool. Persistence set to gone when the run ends: each run starts from the same initial model. Final scores come from held-out generalization benchmarks the agent never sees during the run.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-10-24 · Changlian Ma and Limin Wang · paper
AutoTavern
“We present AutoTavern, a self-improving research system in which a team of specialised LLM research agents iterates a model autonomously (Figure 2).”
- What it changes
- Tools, harness, its own code“A mistake the agents trace to their own method becomes a lesson that every later session loads at its start, or a new gate.”
- Does the change last
- Saved for later runs“a store of lessons loaded at the start of every session”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Of the six errors in Section 8, two became gates (the re-check on unseen seeds, G4, and the comparison of the two input encoders in G2) and two became standing rules (reseed copied games, and calibrate opponents on those the learner meets).”
- Who sets the goals
- people set the aim, the system picks tasks“A research lead chooses the next problem and dispatches work.”
- How it was tested
- own benchmark numbers“on the public leaderboard’s measure its last 20 records average 1.60 against 2.34 for the ten highest-rated players, better on every statistic”
Date is arXiv v1. LLM research agents build a game simulator and train a separate game-playing policy, TavernAgent. The paper says the agents improve their research process through a memory of lessons and new verification gates, not by rewriting their own code. Lessons are loaded by every later session; gate G4, added after a winner's-curse error, delivers a candidate only after a re-check on unseen seeds. Recursion is task: the paper does not compare the changed process with an unchanged one or record a later case where a new lesson or gate caught an error; the evidence it gives for the loop is the task agent's first-place rate across 23 versions. A human sets goals and rules on ambiguous mechanics; a research-lead agent picks the next problem. The top-player comparison uses leaderboard estimates from each player's last 20 records.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-10-24 · Xiangyu Peng et al. (7 authors) · paper
MTA-Zero
“we present MTA-Zero, an online multimodal QA-Solver co-training framework for self-improving multimodal deep-search agents that requires no precomputed multi-hop corpus.”
- What it changes
- Its model weights“We optimize the two policies independently with GRPO, without sharing parameters or gradients.”
- Does the change last
- Becomes the next version“the generator learns to pose new questions at its shifting learning frontier, and the Solver is in turn trained on these online-generated questions.”
- Does it improve the improver
- improves the improver“The frozen and adaptive runs start from the same Solver initialization and use exactly the same seed contexts, harness, tools, rollout budget, and training schedule; the only training difference is whether the QA policy is updated.”
- Who sets the goals
- people set the aim, the system picks tasks“As the Solver improves, its rollout feedback trains the QA Generator to target the Solver’s remaining capability gaps.”
- How it was tested
- tested on held-out tasks“The pool provides initial visual grounding rather than an offline deep-search corpus: it contains no pre-generated multi-hop QA pairs, retrieved evidence, or search trajectories.”
Date is arXiv v1. Two separate copies of Qwen3-VL-8B-Instruct are trained with GRPO for 50 iterations: a QA Generator that searches the web and writes multi-hop questions, checked by a fixed harness, and a Solver that answers them with tools. The Generator's reward is computed from the Solver's success rate on each question and peaks at 4 of 8 rollouts correct. Recursion is improver, as a judgment call: the Generator is the part of the loop that makes the Solver's training questions, it is itself trained during the run, and each updated Generator writes the next iteration's questions. The paper has a control: with everything else the same, a frozen Generator took the Solver's four-benchmark average to 44.28% and a trained one to 52.40%; with the final Solver fixed, the share of questions giving a training signal rose from 44.3% for the initial Generator to 56.8% for the last. The rule that trains the Generator (reward, target success rate, harness checks) is set by the authors and stays fixed, and the Solver does not change its own training method. Goals is mixed: humans pick the seed images, the reward and the benchmarks; the Generator chooses what to ask. Measured is heldout: the six benchmarks are existing test sets, separate from the generated training questions. One is the FVQA test split, whose training split is a seed source, and the paper reports no overlap check. The authors ran their own model and the commercial baselines; specialist-agent rows are taken from the original papers. No code is linked. Compared with RSI-Opt and CoEvo-RSI, where a fixed meta-agent improves a separate agent and recursion is task, here the trained Generator is part of the loop that improves the Solver, and it is itself updated and reused.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-24 · Yue Huang et al. (18 authors) · paper
RSI Arena
“We introduce RSI Arena, a two-stage study of autonomous post-training as a step toward recursive self-improvement.”
- What it changes
- Prompts, memory, context“When an agent’s context filled up, the harness compacted it into a summary.”
- Does the change last
- Saved for later runs“the agent’s Stage 1 notes, code, data and checkpoints are where it left them”
- Does it improve the improver
- does not improve how it improves“The agents also trained a separate, smaller model, leaving the models that powered their research unchanged.”
- Who sets the goals
- people set tasks and scoring“All agents received the same instruction: to train the strongest model possible from the provided base checkpoint within their allocated resources.”
- How it was tested
- tested on held-out tasks“The instruction stated that final models would be evaluated on held-out data inaccessible to the agents and by human evaluators.”
A study, not a system that changes itself. Ten frontier models act as training agents and post-train a separate base model; the models that powered the agents stay unchanged, so on the changes axis only the agents' working context changes. Persistence set to saved: in Stage 2 each agent picks up its Stage 1 notes, code, data and checkpoints and receives the arena battles its model played. Each agent ran once, on one base model.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-23 · Binghang Lu, Yige Sun and Guang Lin · paper
RSI-Opt
“We introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agents which evolves the knowledge that guides this process: a prompt, a playbook of procedures, a library of tested optimizer code, and a decision cadence.”
- What it changes
- Tools, harness, its own code“Code library: indexed Python modules for update rules, schedules, parameter grouping, stability checks, and diagnostics.”
- Does the change last
- Becomes the next version“Later generations inherit and revise the procedures and tools accumulated by earlier ones.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“This is recursive improvement of an agent policy, without modification of the meta-agent’s own improvement procedure.”
- Who sets the goals
- people set tasks and scoring“The underlying LLM weights, meta-agent instructions, scoring rule, and designated task splits remain fixed.”
- How it was tested
- tested on held-out tasks“Held-out tasks are designated for transfer evaluation rather than policy selection.”
Date is arXiv v1. An in-run agent edits optimizer code during a training run. A meta-agent rewrites the policy that agent starts with: prompt, playbook, code library and decision cadence. The meta-agent's instructions do not change, and the paper says the meta-agent's own improvement procedure is not modified, so recursion is task. Evolution used three training tasks; six held-out tasks were not used for selection. One meta-agent read a held-out task description before filesystem restrictions were added; that policy and its descendant were excluded from parent selection and final selection, and the authors say the checks do not rule out indirect exposure. With Opus 5.5 the evolved policy's aggregate held-out score is lower than the fixed prompt's, though it improves the mean on four of the six held-out tasks; with GPT-5.5 and GPT-6.1 the aggregate is higher.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-18 · National University of Singapore, A*STAR, SenseTime · paper
CoEvo-RSI
“We introduce CoEvo-RSI, a RSI framework that advances model–harness co-evolution through agent-driven, bidirectional historical inheritance.”
- What it changes
- Its model weights“Using only supervised fine-tuning (SFT) on its own rejection-sampled trajectories, CoEvo-RSI makes Qwen3.5-9B competitive with larger models across diverse vertical applications”
- Does the change last
- Becomes the next version“Evaluation, model adaptation, and promotion turn these inherited components into the next agent, whose outcomes enrich the evolutionary archive.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“A meta-agent with fixed parameters and role policies operates on the incumbent and an archive of earlier harnesses, checkpoints, and execution histories, including the seed pair.”
- Who sets the goals
- people set tasks and scoring“its task score is assigned by the fixed evaluator”
- How it was tested
- tested on held-out tasks“All four benchmarks are held out from training and evolutionary selection.”
Evolves the harness and the model together, picking or merging earlier checkpoints to fit each new harness. The meta-agent that runs the loop is a separate, fixed model; the model being trained is Qwen3.5-9B. The paper says the result is tailored to its task family and does not imply transfer to unrelated domains.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-18 · Bake AI, University of Notre Dame, and others · paper
Épi
“We present Épi, an autonomous research agent that sustains investigations across model sessions and evolves its methods through accumulated experience.”
- What it changes
- Prompts, memory, context“Épi stores code, artifacts and detailed research notes in a persistent filesystem and keeps retrievable lessons and dead ends in a memory store outside the model context.”
- Does the change last
- Saved for later runs“Épi retains both the experimental state needed to resume a campaign and lessons intended for reuse on another problem.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“The agent adapts its methods through experimental feedback, retrieved memory and independent review.”
- Who sets the goals
- people set tasks and scoring“A curator supplies the question and source material; the agent chooses the methods and experiments.”
- How it was tested
- own benchmark numbers“We fix the feasibility tolerances and gain floor before the investigation.”
Date is arXiv v1. The runs used GPT-5.6 Sol and GPT-6-Astra with weights fixed. What carries over between sessions is a memory store of lessons and dead ends, versioned branches of research code, and the reviewer's recorded assessments; the paper reports no comparison with any of them switched off. A record is a result that the fixed verifier accepts as improving a documented literature or registry reference, with references as of September 16, 2026. The authors set each task's verification rule and tolerances; the paper reports no outside re-check. 97 of the 100 construction records are geometric packings; the nine packings in Figure 3 improve by 0.0021% to 0.0527%. The 20 numerical-bound records include an approximately 84-fold narrower interval for the n-queens constant. Held-out instances are used only for Épi-turbo: on 30 FrontierOR tasks new to both systems, it scored 53.96 to Codex's 51.21 with the same model, from one run per task and system.
Full page source ↗ paper ↗ See it on the map ↑
2026-10-17 · Cheikh Ahmed et al. (5 authors) · paper
Organon
“We introduce Organon, a recursively self-improving agent for scientific discovery.”
- What it changes
- Tools, harness, its own code“a single unified agent governs and modifies its entire operational stack, including the evolutionary loop, parent selection, candidate variation, population management, memory, tools, resource allocation, and worker coordination.”
- Does the change last
- Gone when the run ends“carry capabilities developed on one task into the next, so that self-improvement compounds across problems as well as within them.”
- Does it improve the improver
- improves the improver“Second, the instruments used to decide further revisions were themselves revised, and the corrected instruments shaped later revisions.”
- Who sets the goals
- people set tasks and scoring“The official evaluator and candidate-generation budget remain fixed throughout the investigation.”
- How it was tested
- own benchmark numbers“Since search and program selection use public cases, these results establish that the discovered improvements extend to held-out cases within the evaluated tasks.”
Date is arXiv v1. One agent revises its own search code, tools, memory and worker setup while it works on a task; the official evaluator and the attempt budget stay fixed. In three runs it also changed which model generated some candidates. Recursion is improver because the paper records the agent revising the diagnostic tools it uses to decide further revisions (tools it had itself built earlier in the run), and the revised tools then guiding later revisions: in Cloudcast an extended bottleneck analyzer set the next revision's direction, and in AHC011 a corrected regression check changed later worker guidance. This evidence is case records from one run per task with no ablation, and the authors say timing within a run does not identify a revision's causal contribution. Persists is run: revisions last for the rest of a task's run, and carrying them into the next task is listed as future work. Measured is self-reported: on ALE-Bench, selection used public cases and results are on private cases of the same tasks, which are held-out cases, not held-out tasks; the authors ran all baselines themselves.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-10-16 · Penn State, UC Santa Cruz, Emory University, UC Santa Barbara, and Amazon · paper
LM2LM
“we propose LM2LM, which trains LLMs with RL to generate effective LM improvement proposals.”
- What it changes
- Prompts, memory, context“where ht records the proposals executed in previous steps and their measured outcomes”
- Does the change last
- Gone when the run ends“An AutoResearch campaign consists of a sequence of research steps, where each step requires the proposer to decide what proposal to explore next based on the current task”
- Does it improve the improver
- does not improve how it improves“Each experiment trains the target model from scratch.”
- Who sets the goals
- people set tasks and scoring“Each task specifies a model family, model size, pretraining budget, and a reference configuration with validation loss”
- How it was tested
- tested on held-out tasks“No held-out experimental records are used for proposer training, RWM grounding, or calibration.”
Date is arXiv v1. A proposer model, Qwen3.8-27B, is trained with RL to suggest recipe changes for separate target language models; a target model is trained from scratch when a proposal is actually run. The proposer does not change itself or its successor, so on the changes axis only its working context changes. Its weights are updated by the authors' RL loop, with rewards predicted by a frozen GPT-6-sol research world model whose in-context records are topped up with real runs three times, for 200 real runs in all. The paper calls this a potential path toward recursive self-improvement. The 10 held-out tasks differ from training tasks in model size or training budget; a separate run holds out the whole Qwen3 family.
Full page source ↗ paper ↗ See it on the map ↑
2026-10-16 · Microsoft Research, University of Virginia, Carnegie Mellon University · paper
Self-HPO
“we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer.”
- What it changes
- Its model weights“The policy is then updated through reinforcement learning (RL) under the selected harness (P-step)”
- Does the change last
- Becomes the next version“Each P-step initializes from the policy produced by the preceding round and collects on-policy trajectories under the updated harness.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Self-HPO instead reuses the updated task-solving policy as the proposer in the next round”
- Who sets the goals
- people set tasks and scoring“An executable verifier assigns binary outcome rewards”
- How it was tested
- tested on held-out tasks“Although harness search and policy optimization use only tasks from Python projects, the gains transfer to SWE-Multi, which covers nine non-Python languages, and SWE-Pro, which spans a broader range of languages and engineering tasks.”
Date is arXiv v1. The first author did the work as an intern at Microsoft Research. Each round, the policy proposes harness changes, then RL trains it under the chosen harness, and the trained policy proposes in the next round. Its RL reward is whether coding tasks pass; the paper does not measure whether it got better at proposing harnesses. On SWE-Bench Verified the first harness step raised the score from 61.0% to 64.3% and the second from 67.8% to 68.6%. The runs use Qwen3.5-35B-A3B and two rounds. Training tasks come from SWE-rebench, SWE-rebench V2 and Scale-SWE. The paper reports no overlap check against SWE-Bench Verified, where its design ablations are reported; SWE-Bench Multilingual and Pro are out of distribution. The harness proposer can read reference fixes for its search-set training tasks, and its prompt forbids putting them into the agent's context.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-10-15 · The Chinese University of Hong Kong, Shenzhen, Fudan University, and others · paper
R²Orch
“we propose R2Orch, a new paradigm of Recursive Self-Play Orchestration: an inner loop of self-play that improves the orchestrator on self-chosen tasks, and an outer loop that improves how the inner loop improves.”
- What it changes
- Its model weights“failures are distilled into skills, skills into parameters”
- Does the change last
- Becomes the next version“the two loops form one cycle: failures are distilled into skills, skills into parameters, and withdrawals lead the Proposer to new failures”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“An LLM editor revises skills, the least helpful first, and distills new ones from groups in which both parts fail, while withdrawals follow Eq. (14).”
- Who sets the goals
- people set the aim, the system picks tasks“Each candidate list yields two distinct picks”
- How it was tested
- tested on held-out tasks“is best on all twelve benchmarks against six families of baselines, and its lead holds on the held-out half at zero skills”
Date is arXiv v1. What the outer loop changes is the skill set: a fixed Bayes rule withdraws, admits, or restores skills, and an LLM editor revises skills and distills new ones from groups where both parts fail. The paper's case for the outer loop improving the inner loop is that this skill set also sets the Proposer's reward and the Director's training context; the reward formula, the withdrawal rule, and the update steps stay as written in Algorithm 1, so recursion is task. Director, Proposer, and Workers start from Qwen3.5-9B; the Director and Proposer are trained (Table 4 lists LoRA rank 64), and the Workers are frozen. The trained Proposer learns to skip solved candidates, but its reward and update rule stay fixed. The Proposer picks from candidate lists of tasks drawn from the six training benchmarks. The initial skills are customized by hand for each dataset.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-10-15 · Junyao Yang et al. (9 authors) · paper
Terminal-RSI
“We introduce Terminal-RSI, alternating actor-only RSR with critic-inheriting Recursive Policy-Refine (RPR).”
- What it changes
- Its model weights“Rewriting and policy refinement update different parameters.”
- Does the change last
- Becomes the next version“the bypass preserves the critic, and feedback returns the retained policy to the next cycle.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Their recursive coupling is the focus: supervised transitions preserve critic weights, and each new policy-refinement phase adapts inherited values before joint optimization.”
- Who sets the goals
- people set tasks and scoring“Our 3,000-task pool combines 2,400 semantic-verifier tasks and 600 supplementary terminal tasks.”
- How it was tested
- own benchmark numbers“Each retained stage checkpoint is evaluated on each benchmark in three independent runs, reporting their arithmetic mean (Mean@3), not pass@3 or a union of solved tasks.”
A training recipe for one model, Qwen3.8-27B. Each stage's kept model writes and re-runs the verified trajectories for the next fine-tuning stage, and its critic is carried into the next reinforcement learning stage. Recursion set to task: in the paper, "recursive" names this handoff of weights between stages. The training method, rewards and schedule are set by the authors, who recalibrated reward divisors offline, and no experiment measures whether the model got better at improving itself; the authors write that five stages of one backbone do not establish an equal-compute benefit of recursion. Measured set to self-reported: the authors ran the benchmarks and report the checkpoints they kept. Only SWR100 is stated to be disjoint from training.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-13 · Xiaoliang Wang et al. (12 authors) · paper
ECG-RSI
“We introduce ECG-RSI, a recursive self-improvement framework combining verified waveform synthesis with feedback-guided intervention search.”
- What it changes
- Its model weights“the language model is updated while the vision encoder and multimodal aligner remain frozen.”
- Does the change last
- Becomes the next version“The next round starts from that model, while unselected branches are archived.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Updates record feedback, changes, proposer, both versions, validation results, and next-round scope.”
- Who sets the goals
- people set tasks and scoring“Probe budgets, scoring, output-validity requirements, and thresholds are fixed before branch comparison.”
- How it was tested
- own benchmark numbers“Source ECGs and morphology donors are restricted to training data and are patient-disjoint from development and independent evaluation data.”
Date is arXiv v1. A training loop for one model, Qwen3.5-35B-A3B: each round, an agent reads the model's errors, proposes data recipes, and the data factory builds 10,000 verified ECG questions per recipe; the branch that scores best on a selection set becomes the next parent. Recursion is task. The data factory's operators and skills are revised from construction failures, and in R1 the revised versions beat frozen ones: 16,193 candidates against 36,723 for 10,000 accepted questions, and S7 43.22 against 39.29. But the paper never says who or what makes these revisions; each update records a "proposer" without naming one, and there is no individual repair record. The trained model does not revise anything itself. Measured is self-reported: the paper separates evaluation data from generation and selection at the patient level, but does not say the benchmark tasks are separate from the diagnosis tasks the feedback set covers. Goals is human: the authors fix the three recipe families, the scoring and the benchmarks; the agent picks categories within them.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-10-13 · Adobe Research, UCLA · paper
Tev
“we introduce Tev-Gym, the first long-horizon agent training dataset with verifiable, dense, progressive rewards -- 1,170 verified task environments across 7 domains.”
- What it changes
- Its model weights“Training with this recipe yields Tev-27B”
- Does the change last
- Saved for later runs“Tev-320B and Tev-27B are trained with our recipe from GLM-5.3-Flash and Qwen3.8-27B, respectively”
- Does it improve the improver
- does not improve how it improves“Within each domain, we randomly sample specifications and author our own generator around each one.”
- Who sets the goals
- people set tasks and scoring“via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward”
- How it was tested
- tested on held-out tasks“Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets”
A building block, not a self-improving system: the authors write the environment generators, and Tev-27B is Qwen3.8-27B trained on those environments. The paper names RSI as motivation.
Full page source ↗ paper ↗ See it on the map ↑
2026-10-12 · Sakana AI, UC Berkeley · paper
Sakana MASS (Multi-Agent Self-Supervision)
“We introduce Multi-Agent Self-Supervision (MASS). One shared model solves tasks through a team of virtual subagents, searches for better multi-agent workflows, then uses its own judgments to select the best ones.”
- What it changes
- Its model weights“Fine-tune the shared model with LoRA”
- Does the change last
- Becomes the next version“The updated model then resumes all three roles.”
- Does it improve the improver
- improves the improver“We observe that the base model with first-round search workflows wins 64% of comparisons against the base model with base-search workflow.”
- Who sets the goals
- people set the aim, the system picks tasks“only a self-evaluation signal (whose correctness is unknown) guides the overall learning loop”
- How it was tested
- tested on held-out tasks“three tasks held out from post-training”
Two cycles from a 27B open-weights model. The model judges its own workflows and training data; external judges are used only to measure. The evidence that the optimizer role improved is a workflow-transfer comparison: the base model running first-round workflows wins 64% against base-search workflows. On public benchmarks, score per output token was 0.96x on Terminal-Bench 2.0 and 0.94x on SWE-bench Verified, so transfer is uneven by Sakana's own account.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-10-09 · Google Cloud AI Research and Northwestern University · paper
Harness²
“we introduce Harness 2, a plug-and-play framework for recursive agent harnessing in an open world.”
- What it changes
- Tools, harness, its own code“optimizing the runtime scaffolding around a frozen base model”
- Does the change last
- Saved for later runs“a library transferred from JobBench improves performance on WorkBuddy-Bench”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Model weights, stage instructions, and the decision rule remain fixed, while the library evolves.”
- Who sets the goals
- people set tasks and scoring“Grading is the benchmark's composite verifier”
- How it was tested
- tested on held-out tasks“Harness 2 and the other baselines never use them: they process the test tasks as a stream, starting from an empty library or memory.”
Date is arXiv v1. Eight of the nine authors are at Google Cloud AI Research; the first author did the work as an intern there. The code is in the google-research GitHub organization. The paper calls its second loop harnessing-level recursion, but what it updates is a library of past harnesses and lessons that a fixed improver reads; the editor and its instructions are not changed. Model weights stay fixed. In the main runs the harness editor is Claude Opus 4.8, a different model from the executors. WorkBuddy-Bench is graded by hidden tests plus a Gemini 3.5 Flash judge; JobBench and LAB are judged by Claude Opus 4.8 and Claude Sonnet 4.6 against rubrics, and the JobBench judge is the same model as the editor.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-09-30 · FibonaAI · project
Fibona Agent
“Fibona Agent is a minimal, general-purpose, self-improving agent that writes its own code as it runs.”
- What it changes
- Tools, harness, its own code“The agent could then decide how each step works by writing its own harness code as it runs.”
- Does the change last
- Gone when the run ends“session changes are not automatically preserved across runs”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“the smallest possible immutable core that can act and repair the agent”
- Who sets the goals
- people set tasks and scoring“The welcome screen waits for a task before any model call.”
- How it was tested
- demos only“Try the online demo in your browser.”
The line about runs is in the terminal bootstrap README, not the top-level README. The core that drives self-modification is immutable by design. No benchmark numbers in the README or tech report.
Full page source ↗ code ↗ See it on the map ↑
2026-09-29 · Hongjin Qian et al. (14 authors) · paper
AREX-2
“We present AREX-2, an effort to advance the self-improving capability of LLM agents, which we define as the ability to iteratively refine a solution at test time.”
- What it changes
- Prompts, memory, context“Each round ends with a submission, and the agent sees the score and the environment’s report before it decides what to do next.”
- Does the change last
- Gone when the run ends“Self-improvement in this sense is test-time scaling within a single task”
- Does it improve the improver
- does not improve how it improves“The path forward is to widen the training domains, lengthen the horizons, and close the loop by letting the model’s own trajectories become its next training data.”
- Who sets the goals
- people set tasks and scoring“Examples of S are the validation metric of a trained model and the fraction of hidden tests a program passes”
- How it was tested
- own benchmark numbers“AREX-2 reaches an Any Medal rate of 81.8 on MLE-bench Lite, the highest score in the table”
Date is arXiv v1. Within one task, the agent submits a solution, reads the score and the environment's report, and revises it over many rounds; what improves is that solution. The weights were changed beforehand by the authors, who fine-tuned Qwen3.8-27B on trajectories in which a teacher agent improved solutions this way, together with deep-research data from the earlier AREX recipe. Training the model on its own trajectories is named as future work. Measured set to self-reported: the scores are reported by the authors on public benchmarks, and MLE-bench Lite and Frontier-CS are the training domains.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2026-09-21 · Google Cloud AI Research, Stanford University, Washington University in St. Louis, UNC-Chapel Hill · paper
RRSI (Regularized Recursive Self-Improvement)
“We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection.”
- What it changes
- Tools, harness, its own code“Prompts, control flow, configuration, context management, tools, skills, memory and sub-agents may all be modified”
- Does the change last
- Saved for later runs“accepting one fast-forwards the branch, so the incumbent is always a commit”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“The proposer, the analyst that writes the cross-round failure feedback and the leakage critic are all Claude Opus 4.8.”
- Who sets the goals
- people set tasks and scoring“In each domain the harness is evolved on a single suite”
- How it was tested
- tested on held-out tasks“In each domain the harness is evolved on a single suite, and is then run unchanged on held-out benchmarks.”
The harness around a frozen model is what changes. The paper does not describe the proposer, analyst or critic being edited by the loop. Persistence set to saved: accepted harnesses are kept as commits, and the system that edits them stays the same.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-09-14 · University of Maryland, Google DeepMind, University of Virginia, and others · paper
Dream-RSI
“we introduce Dream-RSI , a framework for scalable, recursively self-improving meta-exploration in agent-driven discovery.”
- What it changes
- Tools, harness, its own code“We first make exploration programmable through a lightweight orchestration layer that controls branching, parallelism, and stopping”
- Does the change last
- Becomes the next version“The refined policy is redeployed for the next round of exploration.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“In the offline dreaming phase, a fixed agent iteratively revises the code for the exploration policy”
- Who sets the goals
- people set tasks and scoring“each node in the tree is an discovery attempt and is scored by a fixed evaluator”
- How it was tested
- own benchmark numbers“Across 9 tasks in 4 domains, Dream-RSI achieves competitive quality and improves discovery efficiency in several settings.”
The exploration policy is what improves. The paper says only the policy code changes; the models, evaluator and execution interfaces stay fixed. Measured set to self-reported: the six held-out datasets test the Lasso solver it discovered, not the improved policy on new tasks.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-08-19 · University of Washington, Stanford University, Northeastern University, Carnegie Mellon University, and others · paper
SPADE
“We introduce SPADE (Self-Play in Adaptive Synthetic Executable Environments), a self-play RL framework in which a single LLM plays two roles”
- What it changes
- Its model weights“so updates for either role affect the same policy”
- Does the change last
- Becomes the next version“Both rewards update the shared policy via GRPO.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“a Reasoning Agent that learns to act in them”
- Who sets the goals
- people set the aim, the system picks tasks“alternates between generating executable training environments and learning to solve them”
- How it was tested
- tested on held-out tasks“SPADE improves over the strongest fixed-environment baseline by +5.3 on average across eight held-out math, science, code, and reasoning benchmarks”
The same model writes its own training environments and is trained on them, so the environment designer is also updated. Recursion set to task, not improver: the paper says each ablation control changes more than one factor, and the one that removes designer training also removes memory.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-07-17 · Sakana AI, UC Berkeley · paper
Recursive Harness Self-Improvement (RHI)
“we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history.”
- What it changes
- Prompts, memory, context“which represents the harness as a prompt-level specification of the agent loop”
- Does the change last
- Saved for later runs“the prompt-represented harness is rewritten from self-comparison history and reused in later executions”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“the base model, evaluator, and harness optimizer are fixed”
- Who sets the goals
- people set tasks and scoring“Each task requires generating a complete code repository with standardized deliverables, and outputs are evaluated using pairwise LLM-as-a-judge comparisons.”
- How it was tested
- own benchmark numbers“Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents”
The harness here is a prompt, so it sits at context rather than scaffold. Harnesses are optimized per task and scored on those tasks. All six authors are also authors of Sakana MASS.
Full page source ↗ paper ↗ See it on the map ↑
2026-07-14 · Weco AI · project
Weco AIDE²
“We present AIDE^2, a system that implements this loop for a frontier AI research agent.”
- What it changes
- Tools, harness, its own code“It proposes changes to its own code, benchmarks modified versions of itself on a suite of AI R&D tasks, and keeps the changes that perform best on hidden evaluations.”
- Does the change last
- Becomes the next version“each accepted rewrite becomes the agent that the next round edits”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“we do not think this is strong enough evidence of ignition, given that AIDE 47 is not asymptotically better”
- Who sets the goals
- people set tasks and scoring“the candidate is graded by running it as an optimizer on AI R&D tasks”
- How it was tested
- tested on held-out tasks“These gains generalize to four held-out benchmarks spanning machine learning engineering, heuristic algorithm engineering, and physics-based weather forecasting”
Date is the blog post; the arXiv paper followed on 2026-09-22. The two loops run different agents: the outer loop, which proposes rewrites, runs Weco's hand-tuned research agent. Weco places AIDE² on Level 1 of its own ladder and says the ignition test does not yet show Level 2.
source ↗ paper ↗ on the ledger See it on the map ↑
2026-06-04 · Shanghai Artificial Intelligence Laboratory, East China Normal University · paper
MLEvolve
“We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery.”
- What it changes
- Prompts, memory, context“a dynamic global memory that automatically accumulates and retrieves historical experience during search”
- Does the change last
- Gone when the run ends“a dynamic global memory that automatically accumulates and retrieves historical experience during search”
- Does it improve the improver
- does not improve how it improves“gradually shifts the search from broad exploration to focused exploitation with an entropy-inspired progressive schedule”
- Who sets the goals
- people set tasks and scoring“MLE-Bench comprises 75 carefully curated Kaggle competitions”
- How it was tested
- own benchmark numbers“MLEvolve achieves a 65.3% average medal rate on MLE-Bench under a 12-hour budget”
Self-evolving here means memory and search state that grow during one task's search. The abstract gives no number for the AlphaEvolve comparison.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-03-19 · FAIR at Meta, Meta Superintelligence Labs, University of British Columbia, and others · paper
Hyperagents (DGM-H)
“We introduce hyperagents, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program.”
- What it changes
- Tools, harness, its own code“into a single editable program”
- Does the change last
- Becomes the next version“selected parent hyperagents from the archive generate modified versions of themselves”
- Does it improve the improver
- improves the improver“Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements.”
- Who sets the goals
- people set tasks and scoring“For domains where we create AI judges to reflect human data (i.e., paper review and Olympiad-level math grading), we construct a validation subset”
- How it was tested
- tested on held-out tasks“This domain is reserved as a held-out meta-evaluation to test whether DGM-H's improvements to its self-improvement process transfer across domains and continue to compound over time.”
Extends the Darwin Gödel Machine so the meta agent is editable too. The paper compares against a variant whose meta agent stays fixed.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2026-03-09 · ELLIS Institute Tübingen, MPI-IS, Tübingen AI Center, University of Tübingen, Thoughtful Lab · paper
PostTrainBench
“We introduce PostTrainBench to benchmark how well LLM agents can perform post-training autonomously under bounded compute constraints (10 hours on one H100 GPU).”
- What it changes
- Prompts, memory, context“The model processes context, generates plans, and decides which tools to invoke.”
- Does the change last
- Gone when the run ends“The agent must build its training pipeline from scratch”
- Does it improve the improver
- does not improve how it improves“We ask frontier agents (e.g., Claude Code with Opus 4.6) to optimize the performance of a base LLM on a particular benchmark”
- Who sets the goals
- people set tasks and scoring“An agent receives a base LLM, target benchmark, and 10 hours on one H100 GPU, then post-trains the model to maximize performance.”
- How it was tested
- own benchmark numbers“23.2% for the best agent vs. 51.1% for official instruction-tuned models”
A benchmark. The agents train a separate small base model, not themselves, so on the changes axis only the agent's working context changes. Persistence set to gone when the run ends: the trained model is submitted and scored, and each run builds its training pipeline from scratch. Measured set to self-reported, not heldout: the target benchmark is the thing the agent optimizes. Agents sometimes trained on the test set; an LLM judge flags such runs.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2025-05-29 · Sakana AI, University of British Columbia, Vector Institute · paper
Darwin Gödel Machine
“We introduce the Darwin Gödel Machine (DGM), a self-improving system that iteratively modifies its own code (thereby also improving its ability to modify its own codebase) and empirically validates each change using coding benchmarks.”
- What it changes
- Tools, harness, its own code“a self-improving system that iteratively modifies its own code”
- Does the change last
- Becomes the next version“Only agents that compile successfully and retain the ability to edit a given codebase are added to the DGM archive, as only they can continue self-modification.”
- Does it improve the improver
- improves the improver“Without updating the meta agent that modifies coding agents, DGM w/o self-improve improves the agents in early iterations, but its gains taper off quickly”
- Who sets the goals
- people set tasks and scoring“Each newly generated agent is quantitatively evaluated on a chosen coding benchmark to estimate its coding abilities.”
- How it was tested
- tested on held-out tasks“Since each agent was optimized without ever accessing the alternate benchmark, these evaluations represent truly held-out tests.”
The outer loop stays fixed: the paper says archive maintenance and parent selection are fixed and not modifiable by the DGM. The model weights are not changed. The evidence for improving the improver is an ablation: a baseline whose meta agent stays fixed as the base agent scores lower on both benchmarks. Both are measured by coding scores; the paper states as an assumption that coding benchmarks reflect the agent's ability to self-improve.
source ↗ code ↗ paper ↗ on the ledger See it on the map ↑
2025-05-14 · Google DeepMind · project
AlphaEvolve
“an evolutionary coding agent powered by large language models for general-purpose algorithm discovery and optimization”
- What it changes
- Prompts, memory, context“These programs are evaluated by evaluators and stored in the programs database. This database implements an evolutionary algorithm that determines which programs will be used for future prompts.”
- Does the change last
- Gone when the run ends“This database implements an evolutionary algorithm that determines which programs will be used for future prompts.”
- Does it improve the improver
- does not improve how it improves“uses an evolutionary framework to improve upon the most promising ideas”
- Who sets the goals
- people set tasks and scoring“the user must provide a mechanism for automatically assessing generated solutions”
- How it was tested
- own benchmark numbers“it sped up this vital kernel in Gemini's architecture by 23%, leading to a 1% reduction in Gemini's training time”
AlphaEvolve improves algorithms, not itself. Inside its own system, the programs database that feeds its prompts is what changes. The blog says it helped train the models underlying AlphaEvolve itself; the number it gives is a 1% cut in training time, not a change to how AlphaEvolve improves.
source ↗ paper ↗ on the ledger See it on the map ↑
2025-04-21 · University of Bristol, iGent AI · paper
SICA: A Self-Improving Coding Agent
“We demonstrate that an LLM coding agent, equipped with basic coding tools, can autonomously edit itself, and thereby improve its performance on benchmark tasks.”
- What it changes
- Tools, harness, its own code“is capable of editing its own codebase to improve itself with respect to its cost, speed and benchmark performance”
- Does the change last
- Becomes the next version“Note that since we are not performing any weight updates, this numerical score is only used to pick the next meta agent, as well as the base agent for the next iteration.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“SICA takes the best performing agent from the archive up until this point as the meta-agent, which is instructed to look through the archive, identify an improvement, and implement it.”
- Who sets the goals
- people set tasks and scoring“We define the ‘best’ performing agent by a utility function, which incorporates common desiderata for a good agent”
- How it was tested
- own benchmark numbers“We find performance gains from 17% to 53% on a random subset of SWE Bench Verified”
The agent edits its own Python codebase, for example adding diff-based file editing and an AST based symbol locator, and the best agent so far becomes the meta-agent for the next round. Model weights are not updated. The authors set the benchmarks and the utility weights; the 50 SWE Bench Verified questions behind the 17% to 53% figure are part of the benchmark set used to pick agents, and AIME 2024 and GPQA Diamond showed less improvement. No experiment measures whether the agent got better at improving itself.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2024-10-06 · Peking University, UC Santa Barbara, University of Arizona · paper
Gödel Agent
“we introduce Gödel Agent, a self-evolving framework inspired by the Gödel machine, enabling agents to recursively improve themselves without relying on predefined routines or fixed optimization algorithms.”
- What it changes
- Tools, harness, its own code“Gödel Agent leverages LLMs to dynamically modify its own logic and behavior”
- Does the change last
- Gone when the run ends“It employs monkey patching to directly read and modify its own code in runtime memory”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Gödel Agent can analyze and modify its own code, including the code for analyzing and modifying”
- Who sets the goals
- people set tasks and scoring“We perform 6 independent self-improvement cycles on the validation dataset for each task”
- How it was tested
- tested on held-out tasks“After obtaining the optimized agent, we test it on the test set.”
Changes live in runtime memory. The paper tests the optimized agent afterwards but does not describe it being kept and improved further. The loop can rewrite its own optimizer, and a Game of 24 case study lists such changes, but no experiment measures whether the optimizer got better; the paper says that in some runs it "reaches its performance limit without further enhancing its optimization capabilities".
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2024-08-15 · University of British Columbia, Vector Institute · paper
ADAS: Automated Design of Agentic Systems
“We present a simple yet effective algorithm named Meta Agent Search to demonstrate this idea, where a meta agent iteratively programs interesting new agents based on an ever-growing archive of previous discoveries.”
- What it changes
- Prompts, memory, context“add them to an archive of discovered agents, and use this archive to inform the meta agent in subsequent iterations”
- Does the change last
- Gone when the run ends“the iteration continues with the updated archive until the maximum number of iterations is reached”
- Does it improve the improver
- does not improve how it improves“Meta Agent Search runs for 25 iterations and the meta agent uses GPT-4”
- Who sets the goals
- people set tasks and scoring“Here, we calculate the performance (e.g., success rate or F1 score) and 95% bootstrap confidence interval as the metrics for the meta agent to maximize.”
- How it was tested
- tested on held-out tasks“We report the test accuracy and the 95% bootstrap confidence interval on held-out test sets.”
What grows is an archive of agent designs written as Python code; the paper says searching in code theoretically enables any agentic system, including prompts, tool use and control flows. The meta agent that writes them is GPT-4, and the discovered agents run on GPT-3.5; the paper lists letting the meta agent itself be improved through ADAS as a future direction. Humans choose each domain and the validation score the meta agent maximizes; discovered agents were scored on held-out test sets and transferred to other domains and models.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2024-08-12 · Sakana AI, University of Oxford, University of British Columbia, Vector Institute · paper
The AI Scientist
“We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation.”
- What it changes
- Prompts, memory, context“Finally, The AI Scientist adds the completed ideas and reviewer feedback to its archive of scientific findings, and the process repeats.”
- Does the change last
- Gone when the run ends“At each iteration, we prompt the language model to generate an interesting new research direction conditional on the existing archive, which can include the numerical review scores from completed previous ideas.”
- Does it improve the improver
- does not improve how it improves“In the future, we aim to use our proposed discovery process to produce self-improving AI in a closed-loop system using open models.”
- Who sets the goals
- people set the aim, the system picks tasks“The AI Scientist is able to generate its own scientific ideas and hypotheses, as well as a plan for testing them with experiments.”
- How it was tested
- own benchmark numbers“We find that Sonnet 3.5 consistently produces the best papers, with a few of them even achieving a score that exceeds the threshold for acceptance at a standard machine learning conference from the Automated Paper Reviewer.”
The research it does is on ML problems in human-made templates: diffusion, NanoGPT and grokking. Inside the system, what grows is an archive of ideas and review scores that conditions the next ideas; in the experiments, ideas were generated without waiting for review scores to be added to the archive. Papers are scored by its own GPT-4o reviewer, which the authors checked against 500 ICLR 2022 papers. Research on its own code is listed as a future direction.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2024-01-18 · Meta, NYU · paper
Self-Rewarding Language Models
“In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training.”
- What it changes
- Its model weights“A preference dataset is built from the generated data, and the next iteration of the model is trained via DPO.”
- Does the change last
- Becomes the next version“Our overall self-alignment procedure is an iterative one, which proceeds by building a series of such models, with the aim that each improves over the last.”
- Does it improve the improver
- improves the improver“We find that performing a round of self-reward training improves the ability of the model at providing self-rewards for the next iteration, in addition to its improved instruction following ability.”
- Who sets the goals
- people set the aim, the system picks tasks“We generate a new prompt x_{i} using few-shot prompting, sampling prompts from the original seed IFT data”
- How it was tested
- tested on held-out tasks“We evaluate the LLM-as-a-Judge via various metrics which measure alignment with held-out human preference data.”
A fixed Llama 2-Chat 70B writes new prompts from the seed data; the model being trained (Llama 2 70B) answers them, scores its own answers through an LLM-as-a-Judge prompt, and is trained on the resulting preference pairs with DPO. Each model makes the training data for the next. The judge is the same model, and its pairwise agreement with held-out human rankings went from 78.7% to 80.4% to 81.7% across the iterations. Humans supply the seed data and the judging prompt; the authors call the results preliminary and ran three iterations in a single setting.
Full page source ↗ paper ↗ See it on the map ↑
2023-10-03 · Microsoft Research, Stanford University, OpenAI · paper
STOP: Self-Taught Optimizer
“In this work, we use a language-model-infused scaffolding program to improve itself.”
- What it changes
- Tools, harness, its own code“we use a language-model-infused scaffolding program to improve itself”
- Does the change last
- Becomes the next version“This is iterated for a prespecified number of iterations T, per available resources.”
- Does it improve the improver
- improves the improver“the resulting improved improver generates programs with significantly better performance than its seed improver”
- Who sets the goals
- people set tasks and scoring“improves an input program according to a given utility function”
- How it was tested
- tested on held-out tasks“we find the improved improver, detailed in Appendix H, outperforms the seed improver on each new downstream task without further optimization”
The authors say this is not full recursive self-improvement because the language models are not altered. With GPT-3.5 and Mixtral, performance degraded.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2023-09-28 · Google DeepMind · paper
Promptbreeder
“we present Promptbreeder, a general-purpose self-referential self-improvement mechanism that evolves and adapts prompts for a given domain.”
- What it changes
- Prompts, memory, context“Driven by an LLM, Promptbreeder mutates a population of task-prompts, and subsequently evaluates them for fitness on a training set.”
- Does the change last
- Gone when the run ends“At this point the fittest individual from the whole of the evolutionary run was evaluated against the test set.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“the percentage of times that the different kinds of mutation operators resulted in an improvement when applied to a task-prompt in the population”
- Who sets the goals
- people set tasks and scoring“To determine the fitness of a task-prompt we evaluate its performance on a random batch of training data.”
- How it was tested
- tested on held-out tasks“the dataset was split into two equal training and test sets before the experiments were conducted”
Date is arXiv v1. PaLM 2-L evolves a population of task-prompts and, alongside them, the mutation-prompts used to rewrite them. Fitness is accuracy on training examples for a task the authors pick. The paper's ablation compares training fitness and finds that removing any self-referential operator is harmful under nearly all circumstances, with the greatest benefit from re-describing task-prompts at initialization; mutation-prompt mutation is one of the operators ablated. The first-order hyper-mutation prompt that rewrites mutation-prompts is a fixed instruction.
Full page source ↗ paper ↗ See it on the map ↑
2023-05-25 · NVIDIA, Caltech, UT Austin, Stanford, UW Madison · paper
Voyager
“We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.”
- What it changes
- Tools, harness, its own code“Voyager incrementally builds a skill library by storing the action programs that help solve a task successfully.”
- Does the change last
- Saved for later runs“Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning.”
- Who sets the goals
- people set the aim, the system picks tasks“The curriculum is generated by GPT-4 based on the overarching goal of "discovering as many diverse things as possible".”
- How it was tested
- tested on held-out tasks“Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize.”
What grows is a library of skill programs. GPT-4 is queried as a black box and not fine-tuned. The curriculum is generated by GPT-4 from an overarching goal the authors set.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2023-03-30 · Carnegie Mellon University, Allen Institute for AI, University of Washington, NVIDIA, UC San Diego, Google Research · paper
Self-Refine
“we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement.”
- What it changes
- Prompts, memory, context“we retain the history of previous feedback and outputs by appending them to the prompt”
- Does the change last
- Gone when the run ends“The feedback-refine iterations continue until the desired output quality or task-specific criterion is reached, up to a maximum of 4 iterations.”
- Does it improve the improver
- does not improve how it improves“Self-Refine relies on a suitable language model and three prompts (for initial generation, feedback, and refinement), and does not require training.”
- Who sets the goals
- people set tasks and scoring“p_{\text{gen}} is a task-specific few-shot prompt (or instruction) for an initial generation”
- How it was tested
- own benchmark numbers“Across all evaluated tasks, outputs generated with Self-Refine are preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation”
Date is arXiv v1. One LLM drafts an answer, critiques it, and rewrites it, up to four times per input. What improves is that one output; the model and the few-shot prompts stay the same, and the paper describes no carry-over between inputs. The task prompts and few-shot examples are written by people: taken from previous work where available, otherwise by the authors.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2023-03-20 · Northeastern University, MIT, Princeton University · paper
Reflexion
“We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback.”
- What it changes
- Prompts, memory, context“Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials.”
- Does the change last
- Gone when the run ends“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τt to be correct.”
- Does it improve the improver
- does not improve how it improves“We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback.”
- Who sets the goals
- people set tasks and scoring“between trials, we use exact match answer grading using the environment to give a binary success signal to the agent”
- How it was tested
- own benchmark numbers“Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
Date is arXiv v1. After a failed trial, the agent writes a reflection that is kept in memory for its next attempt at the same task. Memory is capped at a set number of reflections, usually one to three, and the paper describes no reuse across tasks. The LLM is not fine-tuned. On HotPotQA the success signal between trials comes from exact-match grading against the answer; on coding, the agent writes its own unit tests.
Full page source ↗ code ↗ paper ↗ See it on the map ↑
2022-10-20 · Google, University of Illinois at Urbana-Champaign · paper
Large Language Models Can Self-Improve
“In this work, we demonstrate that an LLM is also capable of self-improving with only unlabeled datasets.”
- What it changes
- Its model weights“fine-tune the LLM using those self-generated solutions as target outputs”
- Does the change last
- Saved for later runs“then use the same model checkpoint for the evaluation on six Out-Of-Domain (OOD) tasks”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“fine-tune the model on these self-generated reasoning-answer data”
- Who sets the goals
- people set tasks and scoring“our method only leverages the questions in the training set, without using ground truth labels.”
- How it was tested
- tested on held-out tasks“but also enhances out-of-domain (OOD) test tasks (AQUA, StrategyQA, MNLI)”
Date is arXiv v1. PaLM-540B answers unlabeled training questions many times, keeps the reasoning paths that agree with the majority answer, and is fine-tuned on them. The paper describes one round of fine-tuning and does not describe repeating it. The questions come from existing datasets; preliminary studies also have the model write extra questions and its own few-shot prompts.
Full page source ↗ paper ↗ See it on the map ↑
2022-03-28 · Stanford University, Google Research · paper
STaR: Self-Taught Reasoner
“Thus, STaR lets a model improve itself by learning from its own generated reasoning.”
- What it changes
- Its model weights“fine-tune on all the rationales that ultimately yielded correct answers; repeat”
- Does the change last
- Becomes the next version“We fine-tune the base model M on this filtered dataset, and then restart this process by generating the new rationales with the newly fine-tuned model.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“first construct a finetuning dataset by attempting to solve the dataset using the current model’s rationale generation ability”
- Who sets the goals
- people set tasks and scoring“justifying ground-truth answers to problems the model failed to solve”
- How it was tested
- tested on held-out tasks“when evaluated on 9 and 10 digit examples, never seen during training, the model successfully solves many of these out-of-distribution problems”
Date is arXiv v1. GPT-J generates rationales, keeps the ones that reach the correct answer, and is fine-tuned on them; the newly fine-tuned model writes the next round's rationales. Each round fine-tunes the original pre-trained model, not the previous one. Questions and correct answers come with the datasets (CommonsenseQA, GSM8K, and arithmetic problems the authors generate). The paper says that without rationalization, improvement ends when the model fails to solve new problems in the training set; for problems it gets wrong, rationalization gives the model the correct answer as a hint.
Full page source ↗ code ↗ paper ↗ See it on the map ↑