rsi.watch

ChengAo Shen et al. (10 authors) · 2026-10-29

TSFLab: A Self-Improving Agent-Agnostic Workbench for Time Series Forecasting Research

“In reproduction and benchmarking, TSFLab improves task completion by 20-163% and cuts fresh tokens per completed task by 11-66% across baselines.”
Who
ChengAo Shen et al. (10 authors)
Date
2026-10-29
Evidence
Very low certainty Starts at Low: the developer’s own numbers.
  • ↓ imprecise. The ablation uses three attempts on each of four reproduction tasks. The abstract ranges, 20-163% and 11-66%, are given without intervals. “we remove one module at a time on four reproduction tasks with three attempts each”
  • ↓ selective reporting. The headline figures are task completion and fresh tokens on the authors' reproduction and benchmarking tasks. On method development, the paper says it has not yet produced a new method that substantially outperforms all leading forecasting models. “method development has not yet produced a new method that substantially outperforms all leading forecasting models”
how we rate evidence
Replication
No rerun found. The paper links its code. We found no rerun by another group. Searched up to 2026-11-03. all claims
Checked
2026-11-03, against the source page
Links
Source Paper Code

Also on the page

  • What they count as self-improvement: "incorporating verified research outputs and fixes into the workbench's shared resources and workflows so later agents can reuse them." The workbench "leaves the agent's model and harness unchanged."
  • The main settings use Codex CLI with GPT-6 Luna. TSFLab completes all 12 reproduction attempts and 42 of 48 benchmark results. It reproduces 73% of the 96 scheduled results within 10% of the paper's MSE, against 21–46% for the other settings.
  • A pilot of agent-driven method development "shows improved forecasting accuracy via TSFLab, with 32.8-73.6% fewer fresh tokens and 9.2-68.8% less development time than the other settings."
  • On maintenance, "all 17 initial usability failures were repaired before integration." The authors add that these cases "do not isolate the effect of individual checks."
  • The authors' limits: "Within the limited search time, method development has not yet produced a new method that substantially outperforms all leading forecasting models. The reported improvements therefore demonstrate support for the evaluated development tasks rather than a broadly superior forecasting method." The code is at https://github.com/D2I-Group/TSFLab.

In the self-improving AI catalog

  • TSFLab: what it changes, whether the change lasts, and how it was tested

How it got here

Earlier work in this catalog that TSFLab cites, oldest first. Under each: where that work sits on the map, then where TSFLab sits (changes in bold), and what TSFLab's own paper says about it. Its whole family on the citation map

Cites Voyager 2023, STOP 2023, Darwin Gödel Machine 2025

← Ledger Report an error