Agent Costs Are Rising Even as Tokens Get Cheaper

By Lazar Milicevic · Published October 17, 2026 · 11 min read

Your per-token price dropped, your agent bill didn't. If you've shipped an agent that loops through tools, rereads its own context, and retries on failure, you've probably seen the invoice grow while the pricing page got friendlier. Gartner now has a name for it and a forecast, and the fix is mostly plumbing you control: caching, routing, loop limits, and budgets.

What Gartner actually predicted (and what it didn't)

Gartner's press release of August 17, 2026 predicts that AI inference costs per agentic workflow will increase more than fivefold through 2028. Gartner calls this the "Inference Paradox": better unit economics escalate the overall cost of AI without a clear path to commensurate, predictable value.

Read the claim carefully before you panic or dismiss it:

• It is a per-workflow inference cost forecast, not a forecast of your invoice. The prediction refers to inference cost per agentic workflow, so "your bill will rise 5x" is not a supportable reading of it.
• It is not a steady climb. Gartner analyst Will Sommer told VentureBeat that first-wave agentic capabilities drove a large spending increase in the spring, and that enterprises are currently in a belt-tightening phase.
• It coexists with falling provider costs. Gartner's earlier March 2026 forecast says that by 2030, inference on a 1-trillion-parameter LLM will cost GenAI providers over 90% less than in 2025. That is a provider-cost forecast, not a customer price forecast.

Gartner names three drivers: foundation-model economics are improving rapidly; that efficiency unlocks more powerful and more expensive models; and sophisticated AI workflows use far more tokens than simple chatbot interactions.

So cheaper tokens do not cancel the trend. They change what people build, and what people build consumes more tokens.

Why agents burn money differently than chatbots

A chatbot turn is one request, one response. An agent turn is a loop: plan, call a tool, read the result, re-plan, call another tool, and so on, with the growing context re-sent each time. Gartner estimates that routing a task to an agentic reasoning model raises provider inference costs by at least 5x compared with a basic chatbot interaction, and often much more as task complexity grows. Sommer told VentureBeat that agents need 5-30x more tokens per task than basic chatbots for the same task.

Here is where the extra tokens come from in a typical small-business agent:

1. Re-sent context. Every loop iteration includes the system prompt, tool definitions, and the full history. Iteration 8 pays for iterations 1-7 again.
2. Reasoning tokens. Reasoning models generate "thinking" you pay for as output, and output is the expensive side of the price list.
3. Oversized tool results. A tool that returns a whole 40-page PDF or a 500-row JSON blob puts all of it into context for every subsequent step.
4. Retries and dead ends. An agent that retries a failing tool call six times is a very expensive way to produce an error message.
5. Wrong model for the step. Using a frontier model to decide "is this email a receipt or a lead?" is paying a surgeon to take your temperature.

Notice that none of these are price problems. They are design problems. That's the good news: design problems are fixable without waiting on any vendor.

Step 1: Instrument before you optimize

You cannot tune what you cannot see. Before changing anything, log tokens per step, per workflow, per outcome. The metric that matters is cost per successful task, not cost per call.

A minimal wrapper, assuming the Anthropic Python SDK:

import time, json
from anthropic import Anthropic

client = Anthropic()

def tracked_call(workflow: str, step: str, **kwargs):
    start = time.time()
    resp = client.messages.create(**kwargs)
    u = resp.usage
    record = {
        "workflow": workflow,
        "step": step,
        "model": kwargs["model"],
        "input_tokens": u.input_tokens,
        "output_tokens": u.output_tokens,
        "cache_read": getattr(u, "cache_read_input_tokens", 0),
        "cache_write": getattr(u, "cache_creation_input_tokens", 0),
        "latency_s": round(time.time() - start, 2),
    }
    with open("usage.jsonl", "a") as f:
        f.write(json.dumps(record) + "\n")
    return resp

Once you have a week of this, group by workflow and step. In most agent systems I've looked at, one or two steps account for the bulk of spend, and they're rarely the ones you'd guess. Fix those first. Don't touch the rest.

Tag each run with an outcome (success, failed, escalated_to_human). A cheap workflow that fails 30% of the time is not cheap.

Step 2: Turn on prompt caching for the stable prefix

Caching is the highest-leverage change for loop-heavy agents because the repeated part (system prompt, tool schemas, reference documents) is exactly what gets re-sent on every iteration.

Anthropic's prompt caching pricing works like this:

The read discount varies by model. According to Anthropic's pricing page, a cache hit costs 2.5% of base input on Claude Fable 5.1 and Mythos 5.1, and 5% on Claude Opus 5.5 and Sonnet 5.5, while all other models use 0.1x. Check the current page for your specific model before doing math.

The break-even is short. Per Anthropic's docs, caching pays off after one cache read for the 5-minute duration and after two cache reads for the 1-hour duration. An agent that loops five times against the same prefix clears that easily. The docs also state that caching multipliers stack with other pricing modifiers such as the Batch API discount, long-context pricing and data residency.

Basic usage: mark the end of the stable prefix with cache_control.

resp = client.messages.create(
    model="claude-sonnet-5-5",  # check the docs for current model IDs
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LONG_SYSTEM_PROMPT_AND_POLICIES,
            "cache_control": {"type": "ephemeral"},
        }
    ],
    tools=TOOL_DEFINITIONS,
    messages=conversation,
)

Practical rules that save you debugging time:

• Put stable content first, volatile content last. A cache hit requires an identical prefix. A timestamp or user name at the top of your system prompt kills it.
• Don't reorder tools between calls. Tool definitions are part of the prefix.
• Watch cache_read_input_tokens in the response. If it's zero on iteration 2, your prefix isn't matching. Fix that before anything else.
• Check the minimum cacheable length for your model. Sources conflict on the threshold, so read the current docs rather than trusting a number from a blog.

Step 3: Route by task, not by habit

Anthropic's price list makes the spread obvious. Per the docs linked above, Claude Fable 5.1 is listed at $10 input / $50 output per million tokens, Claude Sonnet 5.5 at $2 / $10, and Claude Haiku 5.5, which Anthropic positions for high-volume work like classification, extraction and routing, at $0.10 per MTok input for prompts up to 100,000 tokens (with a higher rate above that).

Gartner's own conclusion points the same way: to get acceptable ROI from reasoning agents, companies need either exponentially higher returns relative to basic models or highly optimized inference tiering, routing and orchestration, per VARindia's coverage.

Illustrative math with round numbers (an example, not a benchmark): suppose a step sends 10,000 input tokens. At $10/MTok that's $0.10 per call; at $2/MTok it's $0.02; at $0.10/MTok it's $0.001. Run it 1,000 times a day and the difference between the top and bottom tier is the difference between roughly $100/day and $1/day on input alone. You will not be able to move every step down a tier, but you probably can move most of them.

A simple three-tier routing table that works for small-business agents:

routes:
  classify_intake:        # is this a lead, invoice, support, spam?
    model: haiku-tier
    max_output_tokens: 50
  extract_fields:         # pull amounts, dates, vendor from a document
    model: haiku-tier
    max_output_tokens: 400
  draft_reply:            # write a customer-facing email
    model: sonnet-tier
    max_output_tokens: 600
  plan_multi_step:        # only when the task is genuinely ambiguous
    model: frontier-tier
    max_output_tokens: 1500
escalate_to:
  frontier-tier: when haiku/sonnet returns low confidence or fails validation

The pattern is cheap first, escalate on failure. Have the cheap model return a confidence flag or pass a deterministic validator (does the extracted total match the line items? is the date parseable?). Only failures climb the ladder.

How common is this? In a September VB Pulse survey of 109 qualified respondents, cited by VentureBeat, 31% of enterprises using an orchestration platform send routine, high-volume agent tasks to cheaper models. That means roughly two-thirds don't, which is where the savings are hiding.

Step 4: Keep agent loops tight

Every extra iteration re-pays for the whole context. Tight loops are the unglamorous half of cost control.

Hard caps. Set a maximum number of iterations and a maximum token budget per run. When the cap hits, the run stops and escalates to a human or a fallback path. It does not "try a bit harder."

MAX_STEPS = 8
MAX_RUN_TOKENS = 60_000

def run_agent(task):
    spent = 0
    for step in range(MAX_STEPS):
        resp = tracked_call("invoice_agent", f"step_{step}", **build_request(task))
        spent += resp.usage.input_tokens + resp.usage.output_tokens
        if is_done(resp):
            return finalize(resp)
        if spent > MAX_RUN_TOKENS:
            return escalate(task, reason="token_budget")
    return escalate(task, reason="max_steps")

Trim tool output before it enters context. If a tool returns 200 rows and the agent needs 3 fields from 1 row, filter in code. Code is free; context is not.

def search_invoices(query):
    rows = db.search(query)              # may return hundreds
    return [
        {"id": r.id, "vendor": r.vendor, "total": r.total}
        for r in rows[:10]               # cap rows, drop unused columns
    ]

Stop retrying blindly. One retry with a validated error message is fine. Six identical retries is a bug. Detect repeated identical tool calls and break out.

Summarize, don't accumulate. For long-running sessions, compress older turns into a short state summary instead of carrying the raw transcript.

Replace reasoning with code where you can. If the step is deterministic (format a date, sum a column, look up a customer ID), do it in a function, not a model call.

Step 5: Set budgets that stop the bleeding automatically

Here is the uncomfortable number from that September VB Pulse survey: 21% of enterprises using an orchestration platform have nothing that automatically stops agents' token spending. A runaway loop at 2 a.m. is a real failure mode, and "we'll notice on the invoice" is not a control.

Build three layers:

1. Per-run cap (shown above): tokens and steps.
2. Per-workflow daily cap: a counter in your database or a spend-tracking proxy. When the day's budget is hit, the workflow pauses and alerts you.
3. Account-level alert and hard limit at your provider, if available. Check your provider's console for spend limits and alerting options.

# nightly cost report from the usage log (requires jq)
jq -s '
  group_by(.workflow)
  | map({
      workflow: .[0].workflow,
      calls: length,
      in_tokens: (map(.input_tokens) | add),
      out_tokens: (map(.output_tokens) | add),
      cache_read: (map(.cache_read) | add)
    })
' usage.jsonl

Pipe that into an email or Slack message. The goal isn't precision; it's making a doubling visible within a day instead of at month end.

Step 6: Price the workflow before you approve it

VentureBeat's first recommended architectural change is to set success thresholds, compliance overhead and risk-mitigation costs upfront, and not to approve deployments until scenarios are stress-tested against token-price swings.

For a small business, a lightweight version of that is a one-page model per agent workflow:

If the workflow only pencils out at today's prices with zero margin, it is fragile. If it still clears the bar when usage per task doubles, ship it. If it doesn't, redesign the loop or route more steps to cheaper tiers before you scale volume, not after.

This also tells you what not to automate. An agent that costs more per task than the human step it replaces, with no quality gain, is a science project.

A practical order of operations

If you only have a weekend, do these in this order:

1. Add usage logging with outcomes (Step 1).
2. Find the top two costly steps.
3. Move stable prefixes behind caching and confirm cache reads are non-zero (Step 2).
4. Move classification and extraction steps to the cheapest tier with a validator and escalation path (Step 3).
5. Add step and token caps to every loop (Step 4).
6. Add a daily budget with an alert (Step 5).
7. Write the one-page cost model (Step 6) for any workflow you plan to scale.

None of this requires a bigger budget or a platform migration. It requires treating the agent like any other production system: measured, capped, and priced.

Where this fits in my work

I build AI automations for small teams and keep an eye on cost while doing it. In practice that means the pieces above: logging cost per successful task, caching stable prefixes, routing cheap steps to cheap models with validators and escalation, and putting hard caps on agent loops so a bad run can't become a bad invoice.

If you already have agent workflows running and the spend is drifting, the fastest next step is to go through them step by step and see where the tokens are actually going. The usage log from Step 1 will show you most of it.

Work with BizFlowAI

If you'd rather have this built for you, that's what we do: production AI automation for solo founders and small teams — agents, integrations, and document pipelines that actually ship.

Request a free discovery call — 30 minutes, we map the highest-ROI automation in your workflow. No pitch deck, just engineering.

More guides like this on the BizFlowAI blog.