rsi.watch

Hongjin Qian et al. (14 authors) · 2026-09-29

AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

“Trained on this data, our agent, built on Qwen3.8-27B, achieves strong results on MLE-bench Lite (81.8) and Frontier-CS (70.7), transfers to deep research with 84.0 on BrowseComp, 52.6 on HLE, 92.2 on GAIA, and 93.8 on DeepSearchQA, and keeps improving as its budget of rounds grows.”
Who
Hongjin Qian et al. (14 authors)
Date
2026-09-29
Evidence
Very low certainty Starts at Low: the developer’s own numbers.
  • ↓ risk of bias. AREX-2 was scored on MLE-bench Lite with general and task-specific skills in its context. Its task-specific skills are "built by searching for information related to a task". The paper says "MLE-Lite results are reproduced and reported by OpenMLE", and the paper does not say whether the other systems had skills. In the ablation, skills and more rounds alone raise the base model from 28.8 to 68.2; with the same skills and rounds, the trained model scores 13.6 points higher. “On MLE-Lite, AREX-2 is evaluated with skills in its context”
  • ↓ imprecise. No spread across the three seeds is reported, and the other benchmark scores are given as single numbers. “Any Medal is the percentage of competitions in which the agent earns a medal, averaged over three seeds.”
  • ↑ others can rerun it. The evaluation runners are on GitHub and the 27B weights are on Hugging Face under Apache 2.0. The README covers evaluation only; it lists no training code or training data and does not mention the skills used for MLE-bench Lite. “This repository contains the evaluation runners and dataset definitions for the six reported tracks.”
how we rate evidence
Replication
No rerun found. The evaluation code and weights are public. We found no rerun by another group. Searched up to 2026-10-28. all claims
Checked
2026-10-28, against the source page
Links
Source Paper Code

Also on the page

  • What improves is the solution to one task: the paper calls self-improvement "test-time scaling within a single task". The authors fine-tuned Qwen3.8-27B on trajectories in which a teacher agent improved solutions over many rounds, "together with the deep-research data of the AREX recipe". Letting "the model's own trajectories become its next training data" is named as the path forward.
  • The 81.8 is the Any Medal rate on MLE-bench Lite, averaged over three seeds, with skills in the agent's context. It is the highest MLE-bench Lite score in the paper's table, 8.1 points above Naive-N0.5-Flash.
  • On BrowseComp (84.0), text-only HLE (52.6) and DeepSearchQA, it leads the reported models with at most 40B parameters; on GAIA it ranks behind XYZ-Aquila-mini and Agents-A1. The authors: "A gap nevertheless remains on the strongest BrowseComp and HLE results."

In the self-improving AI catalog

  • AREX-2: what it changes, whether the change lasts, and how it was tested

How it got here

Earlier work in this catalog that AREX-2 cites, oldest first. Under each: where that work sits on the map, then where AREX-2 sits (changes in bold), and what AREX-2's own paper says about it. Its whole family on the citation map

Cites STaR 2022, The AI Scientist 2024, AlphaEvolve 2025

← Ledger Report an error