Dream-RSI: What 162x Fewer Agent Calls Really Means

By Lazar Milicevic · Published October 23, 2026 · 9 min read

You're running an agent that explores, evaluates, and retries, and the call count keeps climbing because it keeps walking paths that already failed. Google-linked researchers just published a method that attacks exactly that waste, with a headline number of 162x fewer discovery-agent calls. Before you copy it into your stack, it's worth reading what the number measures, because it's much narrower than the headline suggests.

What Dream-RSI is, in plain terms

Dream-RSI is a research framework that lets a discovery agent improve its search policy by replaying its own past search history instead of running new expensive evaluations. It is a lightweight orchestration layer: the underlying base agent stays unchanged, and no model weights are updated.

The paper is titled "Dream-RSI: Recursive Self-Improvement through Evolving Worlds" and is on arXiv as 2609.14858. Per secondary coverage, it was posted on 14 September 2026 by a team spanning the University of Maryland, Google DeepMind, and the University of Virginia. VentureBeat's write-up describes it as Google researchers, so I'll stay with the arXiv framing: researchers from Google DeepMind and two universities.

"Discovery agent" has a specific meaning here. These are agents that search for better artifacts: faster algorithms, better optimization solutions, quicker GPU kernels. Each candidate gets generated by a model and then scored. That generate-and-score loop is the expensive part, and it's where the savings come from.

The experiments used Gemini-3.1-Pro and Gemini-3.7-Flash, per the coverage, with no fine-tuning. That detail matters for practitioners: the gain comes from how the loop is organized, not from a smarter model.

The mechanism: dreaming over your own history

The core idea is that accumulated discovery history works as a replay simulator. The agent evaluates and refines its exploration policies against that simulator, which gives cheap off-policy feedback without repeating costly online evaluation.

The paper describes a three-stage loop:

1. Online exploration. The agent runs live searches and logs every trace.
2. Replay simulator construction. The logged traces, organized as a discovery tree, become a pool of simulators.
3. Dreaming-based policy improvement. The agent tries out candidate exploration policies against the simulators, offline, and keeps the better ones.

The improved policy is then redeployed online. New online runs add more traces to the pool, so the simulators grow as the policy improves. That's the "recursive self-improvement" in the name.

Here is the same loop as pseudocode. This is my sketch of the shape of the idea, not code from the paper or its repository:

# Illustrative sketch of the loop shape - not the paper's implementation.

history = []          # every (state, action, score) the agent has ever tried
policy = initial_exploration_policy()

for round_ in range(num_rounds):
    # Stage 1: expensive, live. Run the agent and log everything.
    traces = run_online_exploration(policy, budget=live_budget)
    history.extend(traces)

    # Stage 2: turn logged traces into replayable "worlds".
    simulators = build_replay_pool(history)

    # Stage 3: cheap, offline. Score candidate policies against
    # simulators instead of calling the real evaluator again.
    candidates = propose_policy_variants(policy)
    scored = [(p, replay_score(p, simulators)) for p in candidates]
    policy = max(scored, key=lambda x: x[1])[0]

The key asymmetry is that stage 1 costs real model calls and real evaluations, while stage 3 reuses answers you already paid for. The more of your budget moves from stage 1 to stage 3, the bigger the saving.

Reading the numbers without fooling yourself

The 162x figure is real, but it comes from one task under one comparison, so it is a ceiling and not a typical result. Here is every number the sources support:

Four things to take from this table.

The 162x is not like-for-like. The 317-call figure ran on Gemini-3.1-Pro. The 51,200-generation baseline ran on a different model, gpt-oss-120b. Part of that gap could come from the model difference rather than from replay.

The honest aggregate is closer to 1.7x. Against fixed-exploration baselines, the reported saving is about 1.7x. That's still a meaningful cut, but it's a different story than 162x.

Results vary by domain. GPU kernels saw 1.79x to 2.43x. Math optimization claimed over 50x. The method clearly helps more when the search space is one where past traces predict future paths well.

It's the authors' own evidence. The results come from the paper's own tests, on eight discovery tasks across three domains (algorithm engineering, mathematical optimization, GPU kernels), from a single lab. I found no independent reproduction, and the paper is a preprint. Treat the numbers as promising, not settled.

Also worth stating: none of the sources report dollar, token-cost, or pricing figures. VentureBeat's framing about tens of thousands of tokens is context, not a measured result. Don't convert 162x into a dollar savings figure for your own budget.

Where this applies and where it doesn't

Dream-RSI is a research framework for discovery agents, not a drop-in cache for typical business agents. If your agent triages email, drafts invoices, or qualifies leads, the paper does not claim anything about your workload.

The method fits a specific problem shape:

• You run many candidate attempts against an evaluator that returns a score.
• Each attempt is expensive (model call plus evaluation).
• Attempts are related, so a failed path tells you something about nearby paths.
• You keep the full history, including failures.

Workloads that match that shape in a small business are rarer than you'd think, but they exist: tuning a prompt against a test set, searching for a workable data-cleaning script, optimizing a query or scheduling heuristic, iterating on an ad-copy variant against a scoring function.

Workloads that don't match: one-shot classification, extraction, and routing. Those have no exploration tree to replay. A different tool (plain caching, or a cheaper model) is the right answer there.

There's also a built-in limit. Offline replay only covers search topologies already in the discovery history. If the best answer lies in a direction the agent has never tried, replay can't find it. Genuinely new directions still need live rollouts. So the method reallocates budget; it doesn't remove the need to explore.

The transferable pattern: stop paying twice for the same answer

You don't need Dream-RSI to steal its core discipline: log every attempt and its outcome, then query that log before spending new calls. Most agent stacks I see throw this data away.

A minimal version is a trace log plus a "have I been here" check. This is a generic sketch you can adapt, not the paper's method:

import hashlib, json, sqlite3

db = sqlite3.connect("agent_traces.db")
db.execute("""CREATE TABLE IF NOT EXISTS traces (
    key TEXT PRIMARY KEY, task TEXT, candidate TEXT,
    score REAL, ok INTEGER, ts DEFAULT CURRENT_TIMESTAMP)""")

def trace_key(task: str, candidate: dict) -> str:
    blob = json.dumps([task, candidate], sort_keys=True)
    return hashlib.sha256(blob.encode()).hexdigest()

def evaluate_with_memory(task, candidate, evaluator):
    k = trace_key(task, candidate)
    row = db.execute("SELECT score, ok FROM traces WHERE key=?", (k,)).fetchone()
    if row:                       # already paid for this one
        return row[0], bool(row[1])
    score, ok = evaluator(task, candidate)   # the expensive call
    db.execute("INSERT INTO traces (key, task, candidate, score, ok) "
               "VALUES (?,?,?,?,?)", (k, task, json.dumps(candidate), score, int(ok)))
    db.commit()
    return score, ok

That only deduplicates exact repeats, which is the cheapest tier of reuse. The paper's contribution sits a level above: using the structure of past attempts (the discovery tree) to evaluate new exploration policies offline. If you want to see how that's done, the official repository is on GitHub at zhengkid/Dream-RSI. Code availability was reported as time-sensitive (one outlet said the code wasn't out as of 16 September 2026, while later coverage says it's public), so check the repo's current state yourself before building on it.

A practical decision framework

Before investing in anything Dream-RSI-shaped, run your workload through these checks in order.

1. Measure your repeat rate first. Log a week of agent calls with a hash of task plus input. If a large share are exact repeats, plain caching wins and you don't need research-grade machinery. If almost none repeat, caching won't help, and you should look at the next question.

2. Check whether your attempts are related. If failed attempts inform later ones (iterative search, optimization, tuning), a replay-style approach has something to work with. If every call is independent, it doesn't.

3. Check whether you have a cheap, trustworthy evaluator. Replay only helps if scoring past attempts is meaningful. If "success" is a human's subjective judgment, you can't simulate it offline.

4. Count your exploration budget honestly. The gains in the paper come from moving budget from live evaluation to offline replay. If your live evaluation is already cheap, there's little to move.

5. Plan for the replay blind spot. Keep a slice of budget for genuinely new directions. If you replay only what you've seen, you converge on local optima.

6. Pilot against a fixed baseline, with the same model. The 162x headline mixed models. When you test, hold the model constant so you measure the method and not the model swap.

If you pass checks 2 through 4, a pilot is reasonable. If you fail them, spend your effort on boring wins: caching, cheaper models for easy steps, and tighter stop conditions on loops.

Controlling runaway loops first

The cheapest fix for wasteful exploration is usually a budget and a stop rule, not a new algorithm. Hard caps catch most of the pain before you need anything sophisticated:

# agent-loop guardrails (illustrative config)
max_calls_per_run: 200
max_calls_per_candidate_family: 20
stop_if_no_improvement_in: 30      # consecutive calls
dedupe_identical_candidates: true
log_all_attempts: true             # needed for any future replay

Note the last line. Logging every attempt costs almost nothing now and is the precondition for any replay-based improvement later. If you're not logging failed paths today, start there; it's the one change that keeps the door open to this whole family of techniques.

Where this fits in my work

I build practical AI automations for solopreneurs and small teams, and the part of this paper I find most useful is the unglamorous one: log your history and query it before you pay for new calls. Dream-RSI itself is a research framework for discovery-style search, and I'm not claiming it's something I ship as a product. If you try any of this on your own stack, start with logging and simple call budgets, and treat the paper's numbers as a pointer to where the waste might be hiding, not as a result you should expect to reproduce.

Work with BizFlowAI

If you'd rather have this built for you, that's what we do: production AI automation for solo founders and small teams — agents, integrations, and document pipelines that actually ship.

Request a free discovery call — 30 minutes, we map the highest-ROI automation in your workflow. No pitch deck, just engineering.

More guides like this on the BizFlowAI blog.