Self-improving AI / Self-HPO
Self-HPO
“we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer.”
Self-HPO is one of the projects on the self-improving AI map. Below is what it changes about itself, whether the change lasts, and whether it improves the improver, each with the line from its own source.
Where it sits
Columns: what it changes, from the surface to the deepest change. Rows: whether the change outlasts the run. A number is how many projects sit in that square. See it on the 3D map
- What it changes
- Its model weights“The policy is then updated through reinforcement learning (RL) under the selected harness (P-step)”
- Does the change last
- Becomes the next version“Each P-step initializes from the policy produced by the preceding round and collects on-policy trajectories under the updated harness.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“Self-HPO instead reuses the updated task-solving policy as the proposer in the next round”
- Who sets the goals
- people set tasks and scoring“An executable verifier assigns binary outcome rewards”
- How it was tested
- tested on held-out tasks“Although harness search and policy optimization use only tasks from Python projects, the gains transfer to SWE-Multi, which covers nine non-Python languages, and SWE-Pro, which spans a broader range of languages and engineering tasks.”
Date is arXiv v1. The first author did the work as an intern at Microsoft Research. Each round, the policy proposes harness changes, then RL trains it under the chosen harness, and the trained policy proposes in the next round. Its RL reward is whether coding tasks pass; the paper does not measure whether it got better at proposing harnesses. On SWE-Bench Verified the first harness step raised the score from 61.0% to 64.3% and the second from 67.8% to 68.6%. The runs use Qwen3.5-35B-A3B and two rounds. Training tasks come from SWE-rebench, SWE-rebench V2 and Scale-SWE. The paper reports no overlap check against SWE-Bench Verified, where its design ablations are reported; SWE-Bench Multilingual and Pro are out of distribution. The harness proposer can read reference fixes for its search-set training tasks, and its prompt forbids putting them into the agent's context.
Does it count as RSI?
It depends on the definition. Under the four used on What counts as RSI, by what its own source says:
- Loose (Eliezer Yudkowsky, Intelligence Explosion Microeconomics (2013)): counts
- Durable (Ren et al., self-improvement survey (2026)): counts
- Strict (Roman Yampolskiy, From Seed AI to Technological Singularity (2015)): does not count
- Autonomous (Anthropic, When AI builds itself): does not count
How it got here
Earlier work in this catalog that Self-HPO cites, oldest first. Under each: where that work sits on the map, then where Self-HPO sits (changes in bold), and what Self-HPO's own paper says about it. Its whole family on the citation map
2024 ADAS sets itself apart
Prompts, memory → Model weights · gone when the run ends → becomes the next version · does not improve the improver → improves the task agent
“Harness optimization holds the policy fixed and searches over prompts, workflows, or agent code (Hu et al., 2024; Lee et al., 2026; Zhang et al., 2026a), whereas policy optimization holds the harness fixed and updates the model from successful trajectories or verifiable rewards (Shao et al., 2024; Guo et al., 2025; Wei et al., 2025a).”
“However, harness-only optimization cannot improve the underlying model, while policy-only training can learn only from the trajectories that its fixed harness elicits.”
2026 Self-HPO
Model weights · becomes the next version · improves the task agent
Also cites STaR 2022, Reflexion 2023, Self-Refine 2023, Voyager 2023, Promptbreeder 2023, STOP 2023, Self-Rewarding Language Models 2024, AlphaEvolve 2025, Darwin Gödel Machine 2025, Hyperagents 2026, RRSI 2026, Harness² 2026
In the same square
CoEvo-RSI, ECG-RSI, MTA-Zero, R²Orch, Sakana MASS (Multi-Agent Self-Supervision), Self-Rewarding Language Models, SPADE, STaR: Self-Taught Reasoner, Terminal-RSI