Enterprise AI Sovereignty: Own the Full Agent Stack

By Lazar Milicevic · Published September 23, 2026 · 10 min read

At VB Transform 2026 in Menlo Park, Cohere's VP of product engineering Rachad Alao made a claim that hit harder than most conference soundbites: real enterprise AI sovereignty means controlling the entire agent stack — model, runtime, tools, data plane, and observability. If you're a founder or a small ops team trying to ship AI features this quarter, that's a big ask. You don't have a platform team. You don't have a private cloud contract. But the underlying principle still applies to you, and ignoring it is how you end up locked into a vendor's roadmap with your customer data as collateral.

This post breaks down what "controlling the full agent stack" actually means, what parts matter for a 1–10 person business, and where the pragmatic shortcuts are.

What Cohere means by "the full agent stack"

The full agent stack is the set of components an AI agent needs to execute real work: the language model, the inference runtime, the tool interfaces (search, code, database, APIs), the memory and data layer, the orchestration/planning logic, and the observability and guardrails around all of it. Sovereignty means you decide where each piece runs, what data it sees, and how it's audited — not the vendor.

Alao's argument to VentureBeat's Matt Marshall was that enterprises can't outsource this to a single API and call it a strategy. The moment you offload orchestration, retrieval, and tool-calling to a black-box provider, you lose three things at once: portability (you can't swap models), auditability (you can't prove what the agent saw), and cost control (you pay whatever they charge for tokens plus margin on tools).

For enterprises, that's a compliance problem. For a solopreneur or a 5-person SaaS, it's a survival problem — because vendor lock-in on the agent layer is worse than lock-in at the database layer. Your prompts, tools, and evals are the product.

The seven layers you actually need to think about

Here's a plain-language decomposition of the stack. Not all of it needs to be self-hosted. But you need to know who owns each layer.

The takeaway: even if you use a hosted model, layers 3–7 are yours whether you plan them or not. Most SMB AI projects fail because the team ships layer 1 and pretends layers 3–7 don't exist. Then a customer asks "why did the agent send that email?" and there's no trace.

Why sovereignty matters for a 5-person business, not just banks

The enterprise framing makes sovereignty sound like a Fortune 500 concern. It isn't. Three concrete failure modes I've seen at small companies:

1. Model deprecation resets your product. A vendor sunsets a model. Your prompts, which were tuned for its quirks, now underperform. If you don't own your eval set and your prompt versioning, you're re-doing the work from scratch. This happened repeatedly in 2024–2025 across all major providers.
2. Pricing shifts erase margin. Agent workflows are token-hungry — a single customer support agent can burn 30–80k tokens per resolution once you add tools and memory. If the vendor raises prices 2x, and you priced your SaaS on the old rate, you're now selling dollars for eighty cents.
3. Data leakage kills B2B deals. The first serious enterprise customer will ask where their data goes, whether it trains a model, and who can see it. If your answer is "I POST it to a vendor and hope," you lose the deal.

Sovereignty isn't about running everything on-prem. It's about being able to answer those three questions with specifics.

The pragmatic sovereignty stack for SMBs

You don't need Kubernetes. You need a boring architecture that keeps the expensive-to-move parts under your control. Here's the setup I recommend and use with clients:

• Model: hosted API (Claude, GPT, or Command). Fine. Just don't couple your code to one.
• Runtime abstraction: a thin wrapper so you can swap models in one file.
• Tools: MCP servers you write, running in your process or your VPC.
• Memory: Postgres + pgvector, or SQLite for tiny deployments. Your database, your rules.
• Orchestrator: plain Python with explicit state. Skip heavy frameworks until you feel pain.
• Guardrails: a pre/post-processing function you own, plus the vendor's safety layer.
• Observability: structured logs to your own storage, plus Langfuse or similar for traces.

Here's the minimal model abstraction — 20 lines that save you from lock-in:

from typing import Protocol

class LLM(Protocol):
    def complete(self, messages: list[dict], tools: list[dict] | None = None) -> dict: ...

class ClaudeLLM:
    def __init__(self, client, model="claude-sonnet-latest"):
        self.client, self.model = client, model
    def complete(self, messages, tools=None):
        resp = self.client.messages.create(
            model=self.model, messages=messages, tools=tools or [], max_tokens=4096
        )
        return {"text": resp.content[0].text, "stop_reason": resp.stop_reason,
                "usage": {"in": resp.usage.input_tokens, "out": resp.usage.output_tokens}}

class OpenAILLM:
    def __init__(self, client, model="gpt-5"):
        self.client, self.model = client, model
    def complete(self, messages, tools=None):
        resp = self.client.chat.completions.create(
            model=self.model, messages=messages, tools=tools or [])
        return {"text": resp.choices[0].message.content,
                "stop_reason": resp.choices[0].finish_reason,
                "usage": {"in": resp.usage.prompt_tokens, "out": resp.usage.completion_tokens}}

Every call in your codebase goes through the LLM protocol. Swap providers by changing one line at startup. This is the single highest-leverage decision in any AI project under 10k lines.

MCP: the sovereignty layer for tools

Model Context Protocol (MCP), Anthropic's open spec for tool-calling, is the piece that changes the math for small teams. Before MCP, every tool integration was bespoke per-provider function-calling code. After MCP, your tools are stand-alone servers that any compliant model can call. That means the tool layer — where most of your business logic actually lives — becomes portable.

A minimal MCP server for, say, a "get customer by email" tool looks like this:

from mcp.server.fastmcp import FastMCP
import psycopg

mcp = FastMCP("crm-tools")

@mcp.tool()
def get_customer(email: str) -> dict:
    """Look up a customer by email address."""
    with psycopg.connect(os.environ["DB_URL"]) as conn:
        row = conn.execute(
            "SELECT id, name, plan, created_at FROM customers WHERE email = %s",
            (email,)
        ).fetchone()
        return dict(zip(["id", "name", "plan", "created_at"], row)) if row else {}

if __name__ == "__main__":
    mcp.run()

Three things worth noting:

• The tool runs in your process, hitting your database. No customer PII leaves your network to a vendor's tool sandbox.
• The same server works with Claude Desktop, Claude Code, your production agent, and any future MCP-compatible model.
• You can log every tool call at the server boundary — a clean audit line for "what did the agent actually do?"

That last point is what enterprise sovereignty is about, at any company size. If a customer asks "prove the agent didn't touch account X," you can answer.

Orchestration: keep it boring, keep it yours

The temptation with agents is to reach for a framework. Resist it for the first version. A production agent for a small business usually needs: a system prompt, a loop that calls the model, a tool dispatcher, and a stop condition. That's ~80 lines.

def run_agent(user_input: str, llm: LLM, tools: dict, max_steps: int = 8):
    messages = [{"role": "user", "content": user_input}]
    trace = []
    for step in range(max_steps):
        resp = llm.complete(messages, tools=[t.schema for t in tools.values()])
        trace.append({"step": step, "usage": resp["usage"]})
        if resp["stop_reason"] == "end_turn":
            return {"output": resp["text"], "trace": trace}
        for call in resp.get("tool_calls", []):
            result = tools[call["name"]].run(**call["args"])
            messages.append({"role": "tool", "content": result, "id": call["id"]})
    return {"output": "step limit reached", "trace": trace}

You own the loop. You control retries, tool timeouts, cost caps, and every trace entry. When the agent misbehaves — and it will — you can print the trace and see exactly which step went sideways. Most framework debugging sessions I've watched are engineers trying to understand what their framework did for them. Skip that phase.

Move to LangGraph or a heavier orchestrator when you actually have parallel branches, human-in-the-loop checkpoints, or complex state machines. Not before.

Observability and evals: the part everyone skips

Sovereignty without observability is theater. If you can't see what the agent is doing, you don't own it — you're just hosting the illusion. The minimum viable observability setup:

• Structured logs per step: model, tokens in/out, latency, tool calls, cost estimate.
• Trace ID per user interaction: so you can reconstruct a full session.
• A tiny eval set: 20–50 real examples with expected behavior, run before every prompt change.
• A cost dashboard: even a daily SQL query counts. Know your $/resolution.

Log shape I use in production:

{
  "trace_id": "01HZ...",
  "step": 2,
  "model": "claude-sonnet-latest",
  "tokens_in": 4210,
  "tokens_out": 380,
  "cost_usd": 0.019,
  "tool_calls": [{"name": "get_customer", "duration_ms": 42}],
  "latency_ms": 1830
}

Ship that to Postgres, or a file, or Langfuse. It doesn't matter where — it matters that you have it. Without it, you cannot answer "is this agent actually helping customers, or slowly getting worse as the model shifts?" And that question comes up in month three of every deployment.

For evals: don't over-engineer. A pytest file with 30 real cases and an LLM-as-judge scoring function catches 80% of regressions. Run it on every deploy.

Where hosted models still make sense (and where they don't)

Sovereignty doesn't mean self-hosting Llama on a rented A100. For most SMBs, that's a bad trade — you pay in engineering time what you'd save in tokens, and the frontier models are still meaningfully ahead on complex reasoning and tool use.

A rough decision framework:

The trap is skipping the abstraction because "we're just prototyping." Prototypes ship. The two-hour cost of writing the LLM protocol on day one saves weeks later.

How BizFlowAI approaches this

The Cohere framing is correct but incomplete for small teams. "Control the full stack" assumes you have engineers to assemble it. Most solopreneurs and small ops teams don't — they have a business to run and a backlog of automation ideas that never ship because the stack decisions feel too weighty.

What I build for clients is the boring, sovereign version of this stack: a Claude-based agent runtime with a thin model abstraction, MCP servers wired to their actual tools (CRM, invoicing, document store, email), a Postgres memory layer they own, and structured traces so they can see every step. Document pipelines for the messy 40% of work — invoice extraction, contract summaries, lead enrichment — where sovereignty over the source data matters most. The result is a system the client owns end-to-end: they can read the code, swap the model, export the traces, and hand it off to another engineer if I disappear tomorrow. If you're staring at the agent-stack decision and want a working system instead of another architecture diagram, book a discovery call and we'll map your top three automations.

The one-week sovereignty checklist

If you're already running an AI feature and want to shore up the sovereignty side, here's what to do this week:

1. Wrap your model calls in a protocol/interface. One afternoon.
2. Move at least one tool to MCP. Even a single tool proves the pattern.
3. Log structured traces for every agent step. Postgres or JSONL is fine.
4. Write 20 eval cases from real user sessions. Run them before every prompt change.
5. Set a cost cap per user session in code, not just in the vendor dashboard.
6. Document your data flow: where user data goes, what's retained, what's logged. One page.
7. Pick a fallback model. Test that your abstraction actually works with it.

None of this requires a platform team. It requires two or three focused days and the discipline to not skip the abstraction for a shortcut you'll pay for later.

Sovereignty at Cohere's scale is a different problem than sovereignty at your scale. But the principle is the same: know where each layer of your agent lives, own the parts that encode your business, and never let a vendor's roadmap decide what your product does next.

Work with BizFlowAI

If you'd rather have this built for you, that's what we do: production AI automation for solo founders and small teams — agents, integrations, and document pipelines that actually ship.

Book a free discovery call — 30 minutes, we map the highest-ROI automation in your workflow. No pitch deck, just engineering.

More guides like this on the BizFlowAI blog.