One Model Swap Shouldn’t Break Your AI Operations

By Lazar Milicevic · Published October 25, 2026 · 9 min read

Your lead-follow-up workflow finally works, and then your model provider announces a retirement date. If the model's raw output feeds straight into your CRM, that swap is not a setting change; it is a rewrite you do under deadline. Here's exactly how I design the handoff so a model change becomes a test run instead of an outage.

The news hook, and what it does not prove

On October 22, 2026, MIT Technology Review published "Designing an AI serving harness built to last". Its argument: the next competitive advantage in enterprise AI depends less on which model you choose than on the engineering system around it, and that system can turn a model change into an evaluation and configuration decision instead of an architectural reset.

Two caveats. This is not evidence that models have stopped improving, or that choosing one no longer matters. And it is one perspective, not settled consensus, so read the original and judge the argument yourself. I'm borrowing the idea, not the enterprise platform.

My working definition of a serving harness: the surrounding software that prepares requests, calls the model, checks responses, and manages failures. Everything below is a design I'd propose for a small team. It is not a reported result from the article.

Why swaps are not hypothetical

Model retirements are on the calendar right now. These are the dated ones from the providers' own documentation:

Retirement dates for the same model can differ across platforms (the Claude API versus cloud marketplaces, for example), so check the page for the platform you actually call. The sources are the OpenAI deprecations page, the Claude Platform release notes, and the Gemini deprecations page.

The swap can also break code, not just quality. Anthropic's docs state that on Claude Opus 4.7 and later models, setting temperature, top_p, or top_k to a non-default value returns a 400 error, and they say to omit those parameters and guide behavior through prompting. If your integration hardcodes temperature=0, a model upgrade can fail on the first request. See the model deprecations docs.

Anthropic's own advice is to test your applications thoroughly with the replacement model well before the retirement date, to measure how the new model performs on your tasks (source). That's the right instinct. The rest of this post is how to make that testing cheap enough to actually do.

The job: one inquiry in, one traceable action out

A potential customer emails: "Can you quote for twelve locations? We need to start next month."

If email, invoicing, lead follow-up, and reporting take a big chunk of your day, this is the exact moment that eats your afternoon. Hiring is slow and expensive. Generic SaaS misses your workflow. Gmail, your CRM, accounting, chat, and documents don't talk to each other. "We tried ChatGPT, but it didn't actually integrate with anything" describes the gap.

The workflow needs to do four things:

1. Identify the contact.
2. Extract the request.
3. Find the right customer record.
4. Prepare the next step.

The model can help with the interpretation in steps 1, 2, and 4. It should not decide which record to update, which fields are allowed to change, or what happens when the contact is ambiguous. Those decisions belong to ordinary code you control.

Step one: write the output contract before you pick the model

If the model sits directly in charge of your handoffs, a changed response format becomes an operational failure. A missing field stops an update. A guessed customer match puts a request on the wrong account. Manual cleanup brings back the work you wanted to remove, with data-entry mistakes that can create audit problems.

So define the handoff first. The model returns a small structured result and nothing else:

{
  "contact_email": "string or null",
  "requested_service": "string or null",
  "requested_start_date": "ISO date string or null",
  "location_count": "integer or null",
  "supporting_excerpt": "string, verbatim from the message, or null",
  "draft_reply": "string"
}

Rules that go with the contract:

• Unknown values stay null. The model never fills a gap with a plausible guess.
• A request for twelve locations stays twelve locations. It cannot quietly become twelve employees. That's why location_count is its own field and supporting_excerpt must quote the message verbatim.
• Every field has one meaning that doesn't change between models.

Both major vendors offer schema-constrained output. OpenAI says Structured Outputs is the evolution of JSON mode: both produce valid JSON, but only Structured Outputs ensures the output adheres to your schema, enabled with a json_schema response format and strict: true (docs). The Claude API supports the same idea, with the schema sent in output_config.format of type json_schema (docs).

Use them, but understand what they guarantee. The vendor promise is that the output matches your schema. It is not a promise that the extracted facts are correct. A model can return perfectly valid JSON with the wrong start date.

Schema support also has limits that differ by platform. Google's documentation for Claude structured outputs on Vertex says objects need additionalProperties set to false, and that numerical and string-length constraints are not supported (docs). So a rule like "location count between 1 and 500" has to live in your own validator, not in the schema. That's a good argument for keeping validation in your code regardless of vendor.

Step two: ordinary code checks the result

The model's output is untrusted input. A plain validation layer sits between it and your CRM:

from dataclasses import dataclass
from datetime import date

@dataclass
class Decision:
    action: str   # "sessions_spawn" | "ask_clarification" | "skip_duplicate" | "review"
    reason: str

def decide(result: dict, crm, processed_ids: set, message_id: str) -> Decision:
    # 1. Idempotency: never process the same message twice
    if message_id in processed_ids:
        return Decision("skip_duplicate", "message already processed")

    # 2. Shape and business rules (the schema can't enforce these everywhere)
    email = result.get("contact_email")
    if not email:
        return Decision("review", "no contact email extracted")

    count = result.get("location_count")
    if count is not None and not (1 <= count <= 500):
        return Decision("review", f"location_count out of range: {count}")

    start = result.get("requested_start_date")
    if start is not None:
        try:
            date.fromisoformat(start)
        except ValueError:
            return Decision("review", f"bad date: {start}")

    # 3. The excerpt must actually appear in the original message
    # (checked by the caller against the raw email body)

    # 4. Exactly one CRM match, or a human decides
    matches = crm.find_contacts(email=email)
    if len(matches) != 1:
        return Decision("review", f"{len(matches)} CRM matches for {email}")

    # 5. Missing key fields -> ask, don't guess
    if result.get("requested_service") is None or start is None:
        return Decision("ask_clarification", "missing service or start date")

    return Decision("sessions_spawn", "all checks passed")

Four outcomes only:

• sessions_spawn: checks pass; the integration creates an internal task and saves the draft reply.
• ask_clarification: the request is real but incomplete; the draft asks for the missing piece.
• skip_duplicate: already processed; do nothing.
• review: anything ambiguous goes to a visible exception queue for a human.

Sending the customer an email stays a separate approval step in this starting design. The workflow prepares; a person (or a later, deliberately granted permission) sends.

This is the whole point of the harness idea at small-business scale. The model's job is interpretation. Your code's job is deciding whether that interpretation becomes an action.

Step three: build a ten-example test set you can rerun

Here's something you can do today, with no tooling:

1. Write the output contract above in a document.
2. Collect ten representative inquiries you're allowed to use for testing.
3. Make sure the set includes a clear request, a vague request, a missing date, and a duplicate.
4. Beside each one, write the expected action: create a task, ask for clarification, or do nothing because it was already processed.

Then run your current model against the ten examples with writes to live records disabled. Record, per example:

• Missing fields
• Incorrect facts
• Which review decision the validator made
• Processing time
• Cost

Keep the observed numbers. Ten examples won't establish production reliability, and I won't pretend they do. What they do is expose a broken handoff before a customer does, and they give you a baseline.

A minimal runner looks like this:

import json, time

def run_suite(cases, extract_fn, crm_stub):
    rows = []
    for case in cases:
        t0 = time.time()
        result = extract_fn(case["email_body"])      # model call
        decision = decide(result, crm_stub, case["seen_ids"], case["id"])
        rows.append({
            "id": case["id"],
            "expected": case["expected_action"],
            "got": decision.action,
            "match": decision.action == case["expected_action"],
            "seconds": round(time.time() - t0, 2),
            "result": result,
        })
    return rows

rows = run_suite(CASES, extract_with_current_model, CRM_STUB)
print(f"{sum(r['match'] for r in rows)}/{len(rows)} matched expected action")
json.dump(rows, open("baseline_current_model.json", "w"), indent=2)

Note that extract_fn is the only thing that touches a vendor. Everything else is yours.

Step four: swap the model and compare against the same evidence

When your provider announces a retirement, or you want to try a cheaper model, you now have something more useful than "this answer sounds better." You rerun the same ten examples through the same validator and compare to the saved baseline.

Keep the vendor-specific parts in one thin adapter so the swap touches one file:

# adapter.py - the only place that knows about a vendor
MODEL_ID = "your-pinned-model-id"   # one line to change

def extract(email_body: str) -> dict:
    # build request, call the API, parse the structured result.
    # Omit sampling parameters (temperature/top_p/top_k) unless the
    # target model documents support for them - some newer Claude
    # models return a 400 on non-default values.
    ...

What to look for in the comparison:

• Contract violations: schema failures, wrong field types, new nulls where there were values.
• Factual drift: the same inquiry now yields a different location count or start date.
• Routing changes: the validator sends more (or fewer) items to review.
• Latency and cost: record both; don't trust a vendor's headline numbers for your workload.

Model differences may still require changes. A prompt that worked for one model may need adjusting for another. A stable contract doesn't remove that work. It gives you a defined place to catch it, and a pass/fail answer instead of a vibe.

Pin the model identifier explicitly and treat any change to it as a deploy that requires the test suite to pass. Also note that aliases and auto-routing behavior differ across providers; I haven't verified any provider's current alias policy, so check each vendor's changelog for the model you depend on.

The approach I recommend

The approach I recommend, and the one I'd start from when building AI automations: let a model handle the interpretation of an incoming email or document, while ordinary code owns the contract, the validation, the duplicate check, the CRM write, and a visible exception queue. Tying your tools together is the unglamorous part that determines whether a workflow survives its first model change, so that's where the effort should go.

What I'd do this week

Start with one workflow, one output contract, and one visible exception queue. Don't copy an enterprise-sized platform. Write the contract, collect your ten inquiries, run them against your current model with live writes off, and save the results. Then, when a retirement notice lands in your inbox, you'll be running a test instead of starting a rebuild.

A better model can improve the interpretation. The surrounding system determines whether that interpretation becomes a correct, traceable action. That's the part your business has to live with after the demo ends.

Want more like this?

I write about practical AI automation and GenAI engineering for solopreneurs and small teams.

Planning an AI automation project or need a second opinion on your architecture?

Connect with me on LinkedIn: Lazar Milićević, senior engineer.

Visit bizflowai.io for more on practical AI automation.