Self-improving AI / STaR: Self-Taught Reasoner
STaR: Self-Taught Reasoner
“Thus, STaR lets a model improve itself by learning from its own generated reasoning.”
STaR: Self-Taught Reasoner is one of the projects on the self-improving AI map. Below is what it changes about itself, whether the change lasts, and whether it improves the improver, each with the line from its own source.
Where it sits
Columns: what it changes, from the surface to the deepest change. Rows: whether the change outlasts the run. A number is how many projects sit in that square. See it on the 3D map
- What it changes
- Its model weights“fine-tune on all the rationales that ultimately yielded correct answers; repeat”
- Does the change last
- Becomes the next version“We fine-tune the base model M on this filtered dataset, and then restart this process by generating the new rationales with the newly fine-tuned model.”
- Does it improve the improver
- improves the task agent; no evidence the improver got better“first construct a finetuning dataset by attempting to solve the dataset using the current model’s rationale generation ability”
- Who sets the goals
- people set tasks and scoring“justifying ground-truth answers to problems the model failed to solve”
- How it was tested
- tested on held-out tasks“when evaluated on 9 and 10 digit examples, never seen during training, the model successfully solves many of these out-of-distribution problems”
Date is arXiv v1. GPT-J generates rationales, keeps the ones that reach the correct answer, and is fine-tuned on them; the newly fine-tuned model writes the next round's rationales. Each round fine-tunes the original pre-trained model, not the previous one. Questions and correct answers come with the datasets (CommonsenseQA, GSM8K, and arithmetic problems the authors generate). The paper says that without rationalization, improvement ends when the model fails to solve new problems in the training set; for problems it gets wrong, rationalization gives the model the correct answer as a hint.
Does it count as RSI?
It depends on the definition. Under the four used on What counts as RSI, by what its own source says:
- Loose (Eliezer Yudkowsky, Intelligence Explosion Microeconomics (2013)): counts
- Durable (Ren et al., self-improvement survey (2026)): counts
- Strict (Roman Yampolskiy, From Seed AI to Technological Singularity (2015)): does not count
- Autonomous (Anthropic, When AI builds itself): does not count
How it got here
Earlier work in this catalog that STaR cites, oldest first. Under each: where that work sits on the map, then where STaR sits (changes in bold), and what STaR's own paper says about it. Its whole family on the citation map
It cites nothing in this catalog, or our copy has no reference list.
Cited by Large Language Models Can Self-Improve 2022, Promptbreeder 2023, STOP 2023, SICA 2025, Hyperagents 2026, Recursive Harness Self-Improvement 2026, AREX-2 2026, ECG-RSI 2026, Terminal-RSI 2026, LM2LM 2026, Self-HPO 2026
In the same square
CoEvo-RSI, ECG-RSI, MTA-Zero, R²Orch, Sakana MASS (Multi-Agent Self-Supervision), Self-HPO, Self-Rewarding Language Models, SPADE, Terminal-RSI