42h Rebuild vs I/O 2026: 6.2 Retries Became 1.8

By Lazar Milicevic · Published July 20, 2026 · 8 min read

Google ranked its top three AI tooling updates at I/O 2026. I spent a weekend rebuilding a production lead-scoring agent against all three, pointed at 180 real prospect sites with cookie banners, shadow DOM, lazy loads and auth walls. Two updates broke in production. The one Google buried at number three cut retries from 6.2 per site to 1.8.

If your agents only ever touch clean staging pages, ignore this post. If they hit the actual web, the keynote ranking is inverted.

The setup: one dirty corpus, three configs, same hardware

The only benchmark that matters for a scraping agent is a corpus that looks like the sites you actually scrape. I ran all three configurations against the same 180 prospect URLs from an active lead-scoring pipeline. Not curated. Not staged. Whatever the sales list contained.

Corpus breakdown:

Hardware: my home server, WSL Ubuntu, running two other agent stacks in parallel because that's the real load profile. Runtime was Playwright 1.49 with a Chromium build patched for each I/O update. Baseline (no new tooling, just the scraper I've shipped for a year): 6.2 retries per site average, ~11h wall time for the full 180-site run.

That 6.2 is the number every I/O update had to beat by a meaningful margin. Anything less than a 50% drop and it's noise dressed up as progress.

Update one: the modern web guidance layer (34% DOM misread rate)

The pitch was clean. Feed the agent page structure, it infers intent, it picks the right selectors instead of you writing them. On the keynote demo pages it worked. On my 180 sites it misread the DOM 34% of the time.

The failure pattern was consistent:

• Shadow DOM: the guidance layer read the light-DOM tree, missed the shadow root entirely, then confidently returned a selector that pointed at nothing.
• React portals: modal content rendered outside the visual parent tree confused the intent inference. It scored the wrong container as the "main content region."
• Post-auth dashboards: this was the worst. The guidance model was clearly trained on public marketing pages. Dashboards inside login walls have a different shape — nav-heavy, low prose, high interactivity — and the model kept classifying dashboard views as "navigation pages" and returning zero data selectors.

Retry rate went from 6.2 to 5.8. I gave it twelve hours to see if the failure modes were fixable with prompt hints and hybrid selector fallbacks. They weren't fixable in a way that beat what I already had. Cut it.

# what I wanted to write
selectors = guidance.infer(page, intent="pricing_table")
data = page.query_all(selectors)

# what I actually had to write to make it not-terrible
selectors = guidance.infer(page, intent="pricing_table")
if not selectors or page.is_shadow_heavy() or page.is_authed():
    selectors = legacy_selector_bank["pricing_table"]  # my old code
data = page.query_all(selectors)

When the fallback path runs on more than a third of your corpus, the new tool isn't the tool. Your old code is still the tool.

Update two: the new Chrome DevTools protocol (180MB leaked per session)

I wanted this one to win. The protocol is well-designed, the API surface is small and sensible, and the docs don't make you guess. I wired it into the stack on Saturday night around 10pm.

By 8am Sunday, memory was gone.

Each session was leaking roughly 180MB after ~40 pages. On a box with 32GB shared across three agent stacks, that's a countdown. Worker crashed hard at page 121 of the run. I tried three teardown patterns:

// pattern 1: explicit session.detach() after each page
await session.detach();  // memory: still climbing

// pattern 2: full browser context per N pages
if (pageCount % 20 === 0) {
  await context.close();
  context = await browser.newContext();
}  // slowed the crash, didn't stop it

// pattern 3: full browser restart per 50 pages
// worked, but killed throughput by 40% — worse than baseline

None of them released the memory cleanly. Could be a bug that ships in a patch. Could be a design constraint of the new event streaming layer. Either way, in production right now, it's a liability. Killed it Sunday night, roughly 22 hours in. That's the honest cost of chasing a keynote update: two nights and a rewrite of my worker pool for something I had to rip out.

If you're evaluating this update, run it against a stress corpus of at least 200 pages in a single session before you decide. The leak doesn't show up at 30 pages, which is exactly how long Google's demos ran.

Update three: AI-assisted DevTools as a retry fallback (6.2 → 1.8)

Google put this at number three on the keynote slide. It's not agent-facing. It's an assistant inside DevTools that helps a human debug faster. Which is why nobody talked about it — it doesn't have a demo video where an agent does something impressive alone on stage.

I wired it into the retry loop, not the main path. Every time a selector failed, before triggering a full retry, the failure state (rendered DOM snapshot, network trace, event log) was passed to the DevTools assistance layer and asked one question: what is actually blocking this interaction right now?

The answers were mundane and correct:

• "A cookie consent iframe is intercepting the click at coordinates X,Y."
• "The target element is not yet in the DOM. It renders after a fetch that completes ~840ms after DOMContentLoaded."
• "The button is inside a closed shadow root on <my-widget>. Query through .shadowRoot first."

None of that is genius. It's what a competent human would spot in DevTools in ten seconds. The point is the agent couldn't spot it, and now it can, and it can loop the answer back into a retry with actual signal instead of blind exponential backoff.

Numbers from the full 180-site run:

The reason it wins is boring. On real sites, a selector failure is almost never a selector problem. It's a cookie banner, a lazy paint, a shadow root, an overlay z-index. The guidance layer (update 1) tries to predict selectors up front and gets it wrong on messy DOM. The DevTools protocol (update 2) gives you more raw signal but doesn't tell you what it means. The assistance layer looks at the rendered state after the failure and tells the agent what's actually in the way. That's the whole game on the real web.

This one is now default in the stack I ship to clients.

The two-hour benchmark that saves the 42-hour weekend

Every I/O, every OpenAI dev day, every Anthropic launch — the same trap. You watch the keynote, the demo is clean, you spend a weekend rebuilding. If I had run a benchmark first, I'd have killed update one in 90 minutes and update two in three hours.

Here's the protocol. It takes two hours. Do it before you adopt anything from a keynote:

• Assemble a dirty corpus of 20-50 real target URLs. Not your best pages. The messiest ones your agent actually hits. Auth walls, banners, single-page-app junk.
• Measure baseline retry rate. One number. Retries per site with your current stack.
• Swap in exactly one new feature. Not three. One. Rerun.
• Apply the 50% rule. If retries don't drop by at least half, or if wall time doesn't improve meaningfully, the feature is a demo, not a tool. Cut it.
• Stress the memory profile at 200+ pages in a single session. Leaks under load are invisible in demos and fatal in production.

For the assistance layer, my two-hour benchmark against 25 sites showed retries dropping from 6.4 to 2.1 before I committed to the full 180-site rebuild. That was the signal.

Keynote-ranking heuristic that keeps being right

• Slide 1 features have the best demo video. Optimized for the audience watching the stream.
• Slide 3 features are often written by the team that uses the product internally. Optimized for load.
• Default to testing keynote lists in reverse order. It's been right more often than wrong.

What this means if you run agents against real sites

If your agent stack only touches your own clean pages — internal dashboards you control, staging environments, well-behaved SaaS APIs — update one is fine. It'll save you selector work. Adopt it.

If you're running against the open web, treat the ranking as inverted. The features vendors bury are the ones written for operators who have to keep something running at 3am when a client's target site rolls out a new consent modal. Those features rarely demo well because "our agent retried 4.4 fewer times per site" is not a stage moment. But 4.4 fewer retries across 180 sites is seven hours of my Sunday back.

Where bizflowai.io fits into this

The lead-scoring pipeline in this post is one of the stacks bizflowai.io runs 24/7 against real prospect sites for clients — cookie banners, auth walls, shadow DOM, all of it. The dirty-corpus benchmark and the assistance-layer retry loop described above are how new tooling gets evaluated before it ever touches a production run. If a feature can't cut retries by 50% on the messiest 25 sites in a client's corpus, it doesn't ship. That filter is why the stack survives contact with the actual web instead of only the demo version of it.

Want more like this?

I publish practical AI automation, GenAI engineering, and faceless content workflows on YouTube every week.

Subscribe to bizflowai.io on YouTube — never miss a new tutorial.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn — Lazar Milicevic, GenAI Engineer & bizflowai.io Founder.

Visit bizflowai.io for our services, case studies, and AI consulting.