rsi.watch

National University of Singapore, A*STAR, SenseTime · 2026-10-18

CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?

“It surpasses DeepSeek-V4-Pro on TraceElephant (+14.16 agent accuracy) and GLM-5.3 on HomeBench VM (+4.90 success rate)”
Who
National University of Singapore, A*STAR, SenseTime
Date
2026-10-18
Evidence
Moderate certainty Starts at Moderate: the developer’s paper, tested on held-out tasks, code public.
  • ↓ imprecise. No spread across the runs is reported, and the HomeBench test set has 342 instances. “Results for CoEvo-RSI are averaged over three runs.”
  • ↑ others can rerun it. The code is linked from the paper, and the trained Qwen3.5-9B weights and evolved harness for the failure-attribution pair (TraceElephant) are on Hugging Face. “model weights (config.json, *.safetensors, tokenizer, ...)”
how we rate evidence
Replication
No rerun found. Code and weights are public. We found no rerun by another group. Searched up to 2026-10-26. all claims
Checked
2026-10-26, against the source page
Links
Source Paper Code

Also on the page

  • The model being trained is Qwen3.5-9B, fine-tuned only on its own verifier-accepted trajectories. The meta-agent that runs the loop is DeepSeek-V4.1-Flash.
  • On HomeBench VM the closest system is DeepSeek-V4-Pro-0813 at 75.10%, against 75.51%. On Who&When Algo. it reaches 62.17% Who accuracy, below the strongest baseline's 68.25%.
  • The authors' own reading: "We view this as a practical form of RSI". The evolved pair is tailored to its task family and "does not imply transfer to unrelated domains."

← Ledger Report an error