Self-improving AI / PostTrainBench
PostTrainBench
“We introduce PostTrainBench to benchmark how well LLM agents can perform post-training autonomously under bounded compute constraints (10 hours on one H100 GPU).”
PostTrainBench is one of the projects on the self-improving AI map. Below is what it changes about itself, whether the change lasts, and whether it improves the improver, each with the line from its own source.
Where it sits
Columns: what it changes, from the surface to the deepest change. Rows: whether the change outlasts the run. A number is how many projects sit in that square. See it on the 3D map
- What it changes
- Prompts, memory, context“The model processes context, generates plans, and decides which tools to invoke.”
- Does the change last
- Gone when the run ends“The agent must build its training pipeline from scratch”
- Does it improve the improver
- does not improve how it improves“We ask frontier agents (e.g., Claude Code with Opus 4.6) to optimize the performance of a base LLM on a particular benchmark”
- Who sets the goals
- people set tasks and scoring“An agent receives a base LLM, target benchmark, and 10 hours on one H100 GPU, then post-trains the model to maximize performance.”
- How it was tested
- own benchmark numbers“23.2% for the best agent vs. 51.1% for official instruction-tuned models”
A benchmark. The agents train a separate small base model, not themselves, so on the changes axis only the agent's working context changes. Persistence set to gone when the run ends: the trained model is submitted and scored, and each run builds its training pipeline from scratch. Measured set to self-reported, not heldout: the target benchmark is the thing the agent optimizes. Agents sometimes trained on the test set; an LLM judge flags such runs.
Does it count as RSI?
It depends on the definition. Under the four used on What counts as RSI, by what its own source says:
- Loose (Eliezer Yudkowsky, Intelligence Explosion Microeconomics (2013)): does not count
- Durable (Ren et al., self-improvement survey (2026)): does not count
- Strict (Roman Yampolskiy, From Seed AI to Technological Singularity (2015)): does not count
- Autonomous (Anthropic, When AI builds itself): does not count
How it got here
Earlier work in this catalog that PostTrainBench cites, oldest first. Under each: where that work sits on the map, then where PostTrainBench sits (changes in bold), and what PostTrainBench's own paper says about it. Its whole family on the citation map
Cites The AI Scientist 2024, AlphaEvolve 2025, Darwin Gödel Machine 2025
Cited by RSI Arena 2026, ParetoRSI-Gym 2026, Pu, recursive self-improvement 2026
In the same square
ADAS: Automated Design of Agentic Systems, AlphaEvolve, AREX-2, LM2LM, MLEvolve, ParetoRSI-Gym, Promptbreeder, Reflexion, Self-Refine, The AI Scientist