Junyao Yang et al. (9 authors) · 2026-10-15
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
“The five-stage sequence from Qwen3.8-27B base model through RSR-1, RPR-1, RSR-2, and RPR-2 yields Terminal-RSI-27B: 19.84% on Terminal-Bench 4.0, 86.52% on Terminal-Bench 2.1, 16.95% on Terminal-Bench 3.0, and 79.00% on Terminal-Bench Hard.”
- Who
- Junyao Yang et al. (9 authors)
- Date
- 2026-10-15 Version 1 was posted on 2026-10-15. The text we archived and checked is version 2, posted on 2026-10-25.
- Evidence
- Very low certainty Starts at Low: the developer’s own numbers.
Very lowLowModerateHigh
Low ↓ selective reporting ↓ risk of bias ↓ inconsistent results ↓ imprecise → Very low
- ↓ selective reporting. The paper reports the checkpoints it kept, and discarded gray branches appear in its results figure. It records "retained checkpoints rather than validating a prospective selection rule". “The reported sequence records retained checkpoints rather than validating a prospective selection rule.”
- ↓ risk of bias. Only SWR100 is stated to be disjoint from training. For the Terminal-Bench suites, overlap with training is not stated. “SWR100 is training-disjoint; broader repository and pretraining overlap remain incompletely audited.”
- ↓ inconsistent results. The nominal task counts are 70 for Terminal-Bench 3.0 and 63 for Terminal-Bench 4.0. The authors say task manifests and run-level results must be reconciled before assuming identical coverage. “Earlier records instead use 74 Terminal-Bench 3.0 tasks and 66 Terminal-Bench 4.0 tasks.”
- ↓ imprecise. No repeated training seeds, three evaluation runs per checkpoint, and no spread reported. LHTB strict completion is a mean of 5 of 46 tasks. “Three evaluations do not replace independent training seeds or establish statistical superiority.”
- Replication
- No rerun found. The paper links no code or weights. We found no rerun by another group. Searched up to 2026-10-28. all claims
- Checked
- 2026-10-28, against the source page
- Links
- Source Paper
Also on the page
- The model being trained is Qwen3.8-27B. From the base model, four training stages alternate fine-tuning on verified trajectories that the current model rewrote and re-ran (RSR) with PPO reinforcement learning (RPR) on a 3,000-task pool. The authors ran all benchmarks themselves, three runs per checkpoint, and report the mean.
- On Terminal-Bench 2.1 the base model scored 51.70%. The 86.52% was first reached at RSR-2; the final stage left it unchanged. The four-suite mean rose from 20.68% to 50.58%.
- On LHTB (46 long tasks), a task counts as strictly completed only at score 1.00. The mean count of strictly completed tasks went from 0/46 to 5/46, and process reward, which gives partial credit, from 0.21 to 0.37. SWR100, the one suite the paper calls training-disjoint, went from 3.00% to 6.00%, "twice Base but unchanged after RPR-1".
- The authors' limits: "Three evaluations do not replace independent training seeds or establish statistical superiority." "Checkpoint selection requires protected validation, not adaptation to final benchmarks."
In the self-improving AI catalog
- Terminal-RSI: what it changes, whether the change lasts, and how it was tested
How it got here
Earlier work in this catalog that Terminal-RSI cites, oldest first. Under each: where that work sits on the map, then where Terminal-RSI sits (changes in bold), and what Terminal-RSI's own paper says about it. Its whole family on the citation map