rsi.watch

Jun He et al. (10 authors) · 2026-10-30

From Evaluation to Evolution: Benchmarking Image Editing and Evolving Image Models

“To support the experience-acquisition component of recursive self-improvement (RSI), we develop WeGenEvoStudio, a unified multi-agent harness for RSI-oriented learning that integrates model-conditioned failure diagnosis, retrieval-grounded data construction, supervised adaptation, and re-evaluation.”
Who
Jun He et al. (10 authors)
Date
2026-10-30
Evidence
Very low certainty Starts at Low: the developer’s own numbers.
  • ↓ selective reporting. The 1.083 to 3.517 and 13.2% figures are on target sets the report selected. The 0.190 drop is full-benchmark GEdit Overall. Table 4 lists -0.035 under the GEdit column headed O on the selected row. “on their report-selected Top-5 single-image targets, FLUX.2 [dev] improves Overall from 1.083 to 3.517”
  • ↓ imprecise. The evolution numbers come from one adaptation round. The paper gives no repeated training runs. “Such trade-offs after a single adaptation round motivate re-evaluation and the design of multi-round RSI-oriented feedback loops.”
how we rate evidence
Replication
No rerun found. The paper links no code. We found no rerun by another group. Searched up to 2026-11-03. all claims
Checked
2026-11-03, against the source page
Links
Source Paper

Also on the page

  • The page lists Sun Yat-sen University, WeChat Vision, Tencent Inc., Xidian University, Fudan University, and Tsinghua Shenzhen International Graduate School. WeGenEditBench has 2,740 instances and 148 task types. Among 15 evaluated configurations, GPT-Image-2 leads single-image editing at 4.076 Overall, and Seedream-5.0-Pro leads text-intensive editing at 4.291.
  • On their report-selected Top-5 single-image targets, FLUX.2 [dev] improves Overall from 1.083 to 3.517, and Qwen-Image-Edit-2511 gains 13.2%. On WeGenEditBench-SingleImageEdit, Qwen gains 0.373 Overall (13.2%). On its selected GEdit categories, PQ improves by 0.109 while SC declines by 0.343. Table 4 lists -0.035 under the GEdit column headed O on that row. Separately, "Table 10 separates targeted gains from full-benchmark changes: Qwen gains 0.670 WeGenBench F but loses 0.190 GEdit Overall."
  • Qwen's single-image gains "concentrate in EG (+0.461) and VQ (+0.605), while PR remains stable and DS declines by 0.128." On the selected GEdit categories, PQ improves by 0.109 while SC declines by 0.343.
  • The run is one pass: "Such trade-offs after a single adaptation round motivate re-evaluation and the design of multi-round RSI-oriented feedback loops."

In the self-improving AI catalog

  • WeGenEvoStudio: what it changes, whether the change lasts, and how it was tested

How it got here

Earlier work in this catalog that WeGenEvoStudio cites, oldest first. Under each: where that work sits on the map, then where WeGenEvoStudio sits (changes in bold), and what WeGenEvoStudio's own paper says about it. Its whole family on the citation map

Cites Reflexion 2023, Self-Refine 2023, STOP 2023, Gödel Agent 2024, Darwin Gödel Machine 2025

← Ledger Report an error