Erfan Darzi et al. · 2026-10-21
Recursive Self-Improvement on a Budget: Scoring Code Edits Before They Run
“Twelve runs of GPT-4o return a node at 87.0% on held-out HumanEval+, against 84.2% for the parent-rate ablation.”
- Who
- Erfan Darzi et al.
- Date
- 2026-10-21
- Evidence
- Very low certainty Starts at Low: the developer’s own numbers.
Very lowLowModerateHigh
Low ↓ imprecise → Very low
- ↓ imprecise. Twelve runs scored on 100 held-out instances. The gain over the parent-rate ablation has a 95% interval of [0.65,5.01] points, and the interval against Huxley-Gödel Machine selection includes zero. “The paired difference against Huxley–Gödel Machine selection is 1.92 points, with interval [-0.87,4.70] and permutation probability 368/4096.”
- Replication
- No rerun found. We found no public code and no rerun by another group. Searched up to 2026-10-26. all claims
- Checked
- 2026-10-26, against the source page
- Links
- Source Paper
Also on the page
- The agent edits its own scaffold, a system prompt and a loop that writes a solution, runs the unit tests, and retries once. A linear score ranks each proposed edit before it is run.
- With GPT-4o on held-out HumanEval+, Huxley-Gödel Machine selection returns 85.1% and the starting scaffold 81.2%. With Claude Sonnet 4.5 the method returns 86.6% against 86.5% for Huxley-Gödel Machine selection.
- The paper's limits: the score is linear and fit on 96 edits labeled on 16 instances, with two generators and two benchmarks.