rsi.watch

Mohsen Arjmandi · 2026-10-28

Recursive Self-Improvement for ARC-AGI-3: Hill-Climbing to 78.95 RHAE with Haiku 5.5 and ASSAY

“After two rounds, the newest version on its own won 17 of 25 games with Claude Haiku 5.5, at a set RHAE of 76.26, with one medium-effort pass over all games and one xhigh pass over the 13 it lost.”
Who
Mohsen Arjmandi
Date
2026-10-28 Publication date on the Zenodo record.
Evidence
Very low certainty Starts at Low: the developer’s own numbers.
  • ↓ risk of bias. Both analyses read the record of the same 25 games the next pass was scored on, using Sonnet's solved records of those games as the account of their mechanics, and there are no held-out games. The quoted 17 of 25 counts the better of two attempts per game, a medium pass and an xhigh pass on the 13 losses. “The fixes were designed on the games they were scored on.”
  • ↓ imprecise. The tests cover 25 games with one run each. The p = 0.016 is for seven wins gained and none lost at medium effort; the paper says its intervals cover the sampling of games only. “Every cell is one run per game, and run-to-run variance is not measured.”
how we rate evidence
Replication
No rerun found. The paper and code are public, and the project page links the experiment branch. We found no rerun by another person. Searched up to 2026-10-29. all claims
Checked
2026-10-29, against the source page
Links
Source Paper Code

Also on the page

  • Haiku 5.5 plays the 25 public ARC-AGI-3 games inside ASSAY, his reasoning harness; the model's weights do not change. Between versions an analyst model, Claude Fable 5.1, "reads every loss through the layers and recommends code changes at the layer each loss points to". "The author adopts every recommendation on a tagged branch, and the loop runs again." Two rounds produced fix 1 and fix 2.
  • At medium effort, one run per game, wins went 5, 8 and 12 across the three versions, RHAE 39.89, 48.52 and 60.24: "The two rounds together added 20.4 RHAE", "exact McNemar p = 0.016". The paper adds: "Neither round alone is distinguishable from game-sampling noise". Claude Code alone at medium scored 37.18 with 5 wins, with up to five sessions a game "against the harness passes’ three, which favours the baseline"; on Haiku the unmodified harness "made no measurable difference" against it.
  • The headline 78.95 is a pinned record: "The pinned record over six passes, which keeps each game’s best verified run, is 17 of 25 at 78.95."
  • His project page: "Scores are computed locally over several attempts on the public games. They are not comparable to ARC Prize's single-attempt verified results."
  • The author's limits: "The fixes were designed on the games they were scored on." "There are no held-out games to test this on", and "transfer to unseen games is untested". "Each round shipped its changes together and no ablation was run, so no change’s share of the gain is measured." Both analyses "used Sonnet’s solved records of those games as the account of each game’s real mechanics". On the pinned record: "If pins re-win at a lower rate, the record overstates what replaying the pins would score."

In the self-improving AI catalog

How it got here

Earlier work in this catalog that ASSAY RSI loop on ARC-AGI-3 cites, oldest first. Under each: where that work sits on the map, then where ASSAY RSI loop on ARC-AGI-3 sits (changes in bold), and what ASSAY RSI loop on ARC-AGI-3's own paper says about it. Its whole family on the citation map

Cites Schmidhuber, the Gödel machine 2003, Voyager 2023, STOP 2023, ADAS 2024, Gödel Agent 2024, SICA 2025, Darwin Gödel Machine 2025

← Ledger Report an error