Why More GPUs and Tokens Won’t Fix Your AI Agent

Your lead-follow-up agent drafts polished emails, consumes more tokens than expected, and still misses the prospects who need a reply. Giving it a stronger model or a larger compute budget might help—but first you need to know whether the failure is in its reasoning, its access to the right data, or the workflow around it.
More compute is an input. A completed business task is an outcome. The engineering job is to connect the two with evidence.
Define “done” in the system, not the agent’s reply
An AI agent succeeds when the intended change has happened in the business system and can be verified there. “I followed up with the lead” is not proof that the correct lead received an approved message, that the CRM reflects the action, or that a human took over when the agent could not proceed safely.
That distinction is central to Anthropic’s guidance on agent evaluations: evaluate the final state of the environment, not merely the agent’s statement that it finished. For a small business, the “environment” is usually a set of ordinary tools—an inbox, CRM, calendar, shared document, or invoicing system.
Take a lead-follow-up workflow. A usable definition of success might require all of these conditions:
- The agent identified the right contact and associated the message with the right opportunity.
- It used information the business actually has, rather than inventing a requirement, quote, or availability.
- An approved message was sent, or a draft was routed to a person when approval was required.
- The CRM record reflects what happened.
- The lead was not contacted twice because a job retried.
These conditions are deliberately stricter than “the draft sounds good.” They also expose different failure classes. A good draft sent to the wrong person is a failure. A correctly identified lead left in a draft queue with no owner is unfinished work. A safe handoff can be the correct outcome when the available information is insufficient.
Write the acceptance criteria before changing models or adding agents. Otherwise, you may pay for better prose while the underlying task remains incomplete.
Find the bottleneck before buying more reasoning
More tokens or GPUs can help when the agent genuinely needs more capacity to work through a difficult task. They cannot repair a missing customer record, an unreliable tool response, an undefined approval rule, or an integration that reports success before a write is confirmed.
For practical workflow automation, trace a failed run backward from its final state:
| What you observe | Likely question to investigate | First change to try |
|---|---|---|
| Plausible answer, wrong customer | Did retrieval or record matching select the wrong account? | Improve identity matching and require a record identifier before writing. |
| Correct draft, no action in the system | Did the tool call fail, time out, or stop at a draft? | Check the tool response and verify the downstream state. |
| Duplicate email or CRM update | Was the job retried after an uncertain result? | Check for an existing action before retrying a write. |
| Confident answer with missing details | Did the agent have the required source data? | Make missing information a handoff condition. |
| Task completed, but policy violated | Were permissions or approval gates absent? | Put authorization checks outside the model’s judgment. |
This is why an agent trace matters more than a screenshot of its final response. You need to see which records it read, which tools it called, what parameters it supplied, what each tool returned, and whether the intended state change appeared afterward. Anthropic identifies outcome verification, tool and parameter checks, and analysis of turns and token usage as useful parts of code-based agent evaluation.
Do not log sensitive customer content by default just because a detailed trace would be convenient. Preserve the identifiers, tool results, decision points, and error categories needed to investigate a run; restrict access to anything that contains customer data.
The infrastructure argument is worth keeping in proportion. In a WEKA-presented VentureBeat article, WEKA chief AI officer Val Bercovici argues for measuring multi-agent systems by cost per completed outcome. That is a useful question for an operator. It is not evidence that a particular storage architecture will lower the cost of your lead-follow-up workflow.
Treat multi-agent teams as an experiment, not an upgrade
Adding agents can improve a workflow when the work splits into genuinely useful roles, but it also introduces coordination, additional tool calls, and more points of failure. Compare a team with a capable single agent on the same tasks before treating the extra activity as progress.
The clearest caution in the supplied research comes from Vals AI’s web-app study. It compared GPT 6 Sol and Claude Opus 5.5 as single agents and teams on 50 web-app tasks, at medium and maximum reasoning effort. Teams cost 1.8 to 5.1 times as much as single agents, while only one of four team-versus-single score differences was statistically significant.
That significant result matters: the GPT 6 Sol team at medium effort scored 7.3 percentage points above its single-agent setup, with p = 0.005. The study does not show that teams never pay off.
It does show why “more agents” is not a purchasing criterion by itself. For Claude Opus 5.5 at medium effort, the single agent scored 91.5% at a mean cost of $4.08 per app; the team scored 91.2% at $9.10. Those are benchmark measurements, not current API prices or a forecast for your business.
The measurement also has boundaries. Vals AI scored the share of each app’s UI tests that passed, averaged across the 50 apps, and ran each of its eight configurations once. The authors caution that results from this full-stack web-app benchmark may not transfer to other work. UI-test pass share is not the same thing as a verified lead followed up, an invoice reconciled, or a customer issue resolved.
For an SMB workflow, start with a single agent and clear tool access unless you can name the handoff a second agent will improve. A separate reviewer may be worth testing for a high-stakes approval step. A separate researcher may help when source gathering and action-taking need distinct permissions. But give each role a testable responsibility; otherwise, you have made the run more expensive and harder to debug without knowing what changed.
Build an evaluation set from real workflow failures
An agent evaluation should contain representative inputs, an expected safe action, and a check of the resulting system state. Start with examples of routine work, then add the cases that have already broken—or would be costly if handled incorrectly.
For lead follow-up, useful cases include a complete lead record, two contacts with similar names, an email with missing requirements, a customer asking not to be contacted, a stale CRM status, a tool timeout after an uncertain write, and a request that needs a human-approved commitment. These are not trivia questions for the model. They test whether the workflow behaves correctly.
Store each case in a format your test runner can read. For example:
{
"case": "follow_up_after_ambiguous_tool_result",
"input_fixture": "lead_with_existing_outbound_message",
"expected": {
"duplicate_message_sent": false,
"crm_record_matches_outbound_action": true,
"requires_human_review_if_send_status_unknown": true
}
}
The fixture represents a controlled test environment, not a live customer account. The expected fields describe observable results. If your tooling cannot determine whether a message was sent, the case should not pass simply because the agent says it sent one.
Evaluate several layers separately:
- End state: Did the intended record, message, or handoff exist?
- Tool use: Were the right tools called with the right identifiers and parameters?
- Safety boundaries: Did the run avoid unauthorized sends, duplicates, and unsupported claims?
- Interaction quality: Was the message appropriate for the customer and useful to the human reviewing it?
- Resources: How many tool calls, turns, and tokens did the run require, and how long did it take?
A workflow can pass one layer and fail another. A concise response is not a win if it skipped the CRM update. A completed update is not a win if it was applied to the wrong account. For conversational agents, Anthropic likewise recommends checking verifiable end states alongside task completion and interaction quality.
Keep the cases stable when comparing configurations. If you change the model, prompt, tools, and test cases together, you cannot tell what caused a better result. Add new failure cases as you discover them, but preserve a comparable set for before-and-after decisions.
Design retries and handoffs before increasing autonomy
Reliable agents need explicit behavior for uncertainty. The useful question is not only “Can the model make this decision?” but “What happens when a tool fails, the source is incomplete, or the action may already have occurred?”
A read operation is usually straightforward to retry. A write operation needs more care. If a send request times out, the absence of a success response does not prove that no email was sent. Blindly issuing the same request again may create the duplicate you were trying to avoid.
Put that rule in the workflow, not just in a prompt:
def follow_up(lead, message, mailer, records):
if not lead.contact_allowed or not lead.identity_confirmed:
return "human_review"
if records.outbound_message_exists(lead.id, message.action_id):
return "already_completed"
result = mailer.send(
recipient=lead.email,
body=message.body,
action_id=message.action_id,
)
if result.confirmed:
records.mark_followed_up(lead.id, message.action_id)
return "completed"
return "verify_send_status"
This is illustrative control flow, not a drop-in integration. Its important property is that an uncertain send goes to verification rather than another automatic send. A production implementation also needs to account for the capabilities and failure modes of the actual mail and CRM tools.
Human handoffs deserve the same precision. “Ask a human” is not a completed workflow if nobody receives the task or knows what to decide. A useful handoff includes the relevant record, the proposed action, the reason automation stopped, and a place for the reviewer’s decision. The agent should not continue past that point merely because it can produce a plausible answer.
These controls may feel less exciting than adding reasoning effort. They are also what make an agent usable when the founder is busy and cannot watch every run.
Decide whether more compute earns its place
The right metric is the cost of verified completed work at an acceptable error rate, not the cost of a model call in isolation. Compare configurations on the same evaluation cases, and count unfinished tasks, manual rescue, duplicate actions, and review effort rather than hiding them outside the measurement.
Use a simple decision record for each candidate configuration:
| Measure | What to record |
|---|---|
| Verified outcomes | Cases whose required end state was observed |
| Unsafe or incorrect actions | Wrong recipient, duplicate write, unauthorized action, or unsupported claim |
| Human intervention | Cases requiring review or repair, and why |
| Resource use | Model usage, tool calls, and elapsed time |
| Failure recovery | Whether uncertain results were verified or safely handed off |
Cost per completed outcome is useful only if “completed” means the same thing across candidates. A cheaper run that leaves work in an unowned queue has shifted cost to a person. A more expensive run may be justified if it completes meaningfully more work without adding unacceptable risk. Conversely, if the larger configuration produces a better explanation but the same system state, there is no outcome improvement to pay for.
Increase reasoning effort or add a specialist agent only after the baseline reveals a failure those changes could plausibly address. If the agent repeatedly misinterprets a complex request despite having the right data and functioning tools, test more reasoning. If it fails because the CRM lookup returns ambiguous records, fix identification first. If it makes the right decision but cannot confirm a send, improve verification.
The goal is not to minimize tokens at all costs. It is to spend them where they change a result the business can observe.
A practical way to assess this
Define the completed workflow first, then inspect tool use, failure handling, and the final state before considering more model capacity or agent roles. For small-business automation, make routine actions verifiable and give uncertain cases a clear human handoff.
A discussion of a specific process can help identify the goal and how to check whether the workflow meets it. It is not a promise that more automation—or less compute—will improve every outcome.
Work with BizFlowAI
If you'd rather have this built for you, that's what we do: production AI automation for solo founders and small teams — agents, integrations, and document pipelines that actually ship.
Request a free discovery call — 30 minutes, we map the highest-ROI automation in your workflow. No pitch deck, just engineering.
More guides like this on the BizFlowAI blog.
Frequently asked questions
How do I know if my AI agent actually completed a task?
Check the final state of the business system, not the agent’s reply. For a lead follow-up, verify that the right contact received an approved message, the CRM was updated, and no duplicate was sent. If the agent could not proceed safely, a documented handoff to a person can be the correct outcome.
Should I give my failing AI agent more tokens or a stronger model?
First trace a failed run to find where it broke. More reasoning capacity may help with a difficult decision, but it will not fix a missing record, failed tool call, or undefined approval rule. Set acceptance criteria and verify downstream changes before increasing the compute budget.
When should I use multiple AI agents instead of one?
Use multiple agents when each has a distinct, testable responsibility that improves a handoff, such as separating research from an action requiring different permissions. Compare the team against a capable single agent on the same tasks. Extra agents add coordination and cost, so more activity alone is not evidence of a better outcome.
What should I include in an AI agent evaluation set?
Include representative inputs, the expected safe action, and checks of the resulting system state. Mix routine cases with failures such as similar contact names, missing information, uncertain tool results, and requests requiring approval. Score end state, tool use, safety, interaction quality, and resource use separately.
How do I stop an AI follow-up agent from sending duplicate emails?
Before retrying a send, check whether an outbound message already exists for the lead and action. Treat a timeout or unclear tool response as an uncertain result, not proof that nothing happened. If the send status cannot be verified, route the case for human review rather than sending again.