Why Long-Running AI Agents Forget Compliance Rules

By Lazar Milicevic · Published October 30, 2026 · 11 min read

You deployed an agent to chew through a multi-day workflow: validating vendor records, reconciling invoices, cleaning a CRM. You hardcoded the rules into the system prompt on day one. By day three, the agent is still producing clean-looking output, no error has fired, and the constraints you wrote are no longer shaping its behavior. This post covers why that happens, why a bigger context window doesn't fix it, and the architecture that does.

The failure is silent, and that's the whole problem

Compliance drift in long-running agents does not crash anything. The agent keeps working, keeps returning plausible results, and keeps ignoring the constraints. No alert fires because nothing technically failed.

The scenario that prompted this post comes from a VentureBeat guest article by Ankit Anand, a managing consultant and enterprise data management architect: an agent runs a multi-day master data validation workflow, ingests thousands of records, and by day three the baseline governance rules from its system prompt are effectively gone. In the article's telling, an audit weeks later traces the root cause back to day nine. Treat the "day three" and "day nine" figures as illustration, not measurements. The point is the shape of the failure: the damage is done long before anyone looks. You can read the original piece on VentureBeat.

This is worse than a crash for a specific reason. A crash is a signal. Drift is the absence of a signal. If your observability stack watches for exceptions, latency, and token spend, it will report a healthy system while the agent quietly approves records that violate the policy you wrote.

If you run agents for a small business, think about what "rules" means in practice: don't email customers outside business hours, never merge records without a matching tax ID, always mask account numbers in summaries, never exceed a refund threshold without human approval. Each of those is a rule that, if dropped, creates real cost. And you won't find out from a stack trace.

Why this happens (and what we do and don't know)

Here is the honest version of the mechanism, because the popular explanation is a bit loose.

The common phrasing is that the system prompt gets "pushed out of active memory." Depending on how your agent loop is built, the system prompt is usually still present in what the model sees. What degrades is how well the model uses it as the context around it grows. The research on this is real, but it measures something adjacent to compliance:

• "Lost in the Middle" (Liu et al., TACL 2024) found that language model performance degrades significantly when relevant information sits in the middle of a long input, and is often highest when it sits at the beginning or end. The same paper reports that performance decreases substantially as input grows longer, even for explicitly long-context models. Source: arxiv.org/abs/2307.03172.
• Chroma's "Context Rot" report found that model performance consistently degrades as input length increases across all their experiments, and that models don't use their context uniformly. Source: trychroma.com/research/context-rot.

A caveat worth stating plainly: both of these studies measure performance degradation on long inputs, not whether a model keeps obeying a system-prompt rule over a multi-day run. If you want the exact task setups, check the methodology sections in the papers themselves. I'm not going to claim a measured "compliance drops X%" number, because I haven't seen a primary source for one.

What you can safely conclude: as context fills with thousands of records, tool outputs, and intermediate reasoning, a rule written once at the top competes with an enormous amount of more recent, more specific text. Sometimes the model weights the rule properly. Sometimes it doesn't. You cannot tell which from the outside, and you cannot guarantee it.

Anthropic's own guidance frames it the same way. Their engineering post on context engineering says that even as capabilities scale, context should be treated as a precious, finite resource (anthropic.com/engineering/effective-context-engineering-for-ai-agents).

Why a bigger context window doesn't solve it

A larger window gives you more room, not more attention. The VentureBeat article argues that expanding the context window to a million tokens will not solve the problem, and the research above is consistent with that: degradation shows up as input length grows, including in models built for long inputs.

Very large context windows are now a standard feature, not an exotic one. Check the Anthropic pricing page and release notes for which models include which window sizes and at what rates before you budget anything.

Here's the trap. With a huge window, you can run a long workflow for much longer before anything gets truncated. That delays the visible symptom (hard context-limit errors) while doing nothing about the invisible one (rules losing influence). Teams read "no context overflow errors" as "the agent is fine." It's the opposite: the bigger window makes the silent failure easier to reach and harder to notice.

There is also a cost angle. In many implementations, API calls are stateless, so the full conversation history is sent again on every call. A longer-lived agent therefore carries more tokens per call, and you pay more for an agent that is, if anything, less reliable. Check your provider's documentation and pricing page to see how this applies to your setup. Bigger windows are a capacity tool. Governance needs a different tool.

The core principle: rules belong outside the model

The fix is architectural, and the VentureBeat article's recommendations point the same direction: move critical constraints out of the LLM context window into deterministic policy engines, validate model outputs through API gateways against hard-coded logic, and physically separate the agent's scratchpad memory from its operational constraints.

Translate that into one sentence: a rule you need to be true 100% of the time should not depend on the model remembering it.

Anthropic's own Claude Code docs say this explicitly. CLAUDE.md and auto memory are treated as context, not enforced configuration. To block an action regardless of what Claude decides, the docs recommend a PreToolUse hook (code.claude.com/docs/en/memory). That's a vendor telling you directly: instructions in context are guidance; hooks are enforcement.

Use this split as a decision rule:

The mistake is putting the first row in the system prompt and hoping. If violating a rule would cost you money, a customer, or an audit finding, it belongs in the first two rows, enforced in code.

Pattern 1: A deterministic policy gate between the agent and the world

The most reliable pattern is a gate that every consequential action must pass through. The agent proposes; the gate disposes. The gate is ordinary code. It doesn't get tired, doesn't get distracted by 50,000 tokens of record data, and behaves identically on day one and day thirty.

Here's a minimal sketch in Python. The agent returns a structured proposed action; the gate validates it against rules that live in code and config, not in the prompt:

from dataclasses import dataclass

@dataclass
class ProposedAction:
    tool: str
    args: dict

# Hard rules live here, versioned in git, not in the prompt.
POLICY = {
    "refund": {"max_without_approval": 100.00},
    "merge_records": {"require_fields": ["tax_id"]},
    "send_email": {"blocked_hours_utc": range(0, 6)},
}

class PolicyViolation(Exception):
    pass

def check(action: ProposedAction, now_hour_utc: int) -> None:
    rules = POLICY.get(action.tool)
    if not rules:
        raise PolicyViolation(f"No policy defined for tool '{action.tool}' - deny by default")

    if action.tool == "refund":
        if action.args["amount"] > rules["max_without_approval"]:
            raise PolicyViolation("Refund exceeds threshold; route to human approval")

    if action.tool == "merge_records":
        for f in rules["require_fields"]:
            if not action.args.get(f):
                raise PolicyViolation(f"Merge missing required field: {f}")

    if action.tool == "send_email":
        if now_hour_utc in rules["blocked_hours_utc"]:
            raise PolicyViolation("Email blocked outside permitted hours")

Two details matter. First, deny by default: an action with no policy is rejected, so a new tool can't slip through ungoverned. Second, a violation doesn't just fail silently, it returns a structured error to the agent and writes to a log you actually monitor. That log is what turns silent drift into a visible signal: if the gate rejects a small share of proposals early in a run and a much larger share days later, you've detected drift without reading a single transcript.

For Claude Code specifically, a PreToolUse hook is the native way to do this, per the docs above. For your own agent loop, it's the check function wrapped around every tool call.

Pattern 2: Re-inject rules instead of trusting the first message

A gate catches violations. Re-injection reduces how often the agent proposes them in the first place. The idea is simple: don't state your rules once at the start and assume they persist. Restate the relevant ones, close to where the model is about to act.

This leans on the same research that caused the problem. If models do best with relevant information near the start or end of the input, put the rules near the end, right before the decision. Periodically append a short, current rules block to the working context rather than relying on turn one.

def build_turn(messages, active_rules, step):
    """Append a compact rules reminder every N steps and before risky tools."""
    reminder = "ACTIVE CONSTRAINTS (authoritative, re-stated each cycle):\n" + \
               "\n".join(f"- {r}" for r in active_rules)

    # Always re-inject on a cadence; always re-inject before a write action.
    if step % 10 == 0:
        messages.append({"role": "user", "content": reminder})
    return messages

Be clear about what this is. It's a mitigation, not a guarantee. Re-injection raises the odds the model complies; the gate in Pattern 1 ensures non-compliance can't cause harm. You want both, and you want to never mistake the first for the second.

Choose the rules to re-inject by relevance. A 40-rule block re-sent every cycle reintroduces the very context bloat you're trying to avoid. Inject only the rules that apply to the tool or record type the agent is about to touch.

Pattern 3: External memory, with a hard wall around constraints

The third recommendation from the article is to physically separate the agent's scratchpad from its operational constraints. Think of two stores that never share a write path:

• Scratchpad: working notes, progress, intermediate findings. The agent reads and writes freely. It's allowed to be messy and to be summarized.
• Constraint store: the rules and policy definitions. Read-only to the agent. Changed only by you, through your normal review process.

This matters because of how long-running agents manage their own context. Anthropic documents two relevant tools. Context editing clears stale tool results once the prompt passes a configurable trigger, keeping a set number of recent tool use/result pairs, with an exclude_tools option to protect specific tools' results from being cleared (docs). The memory tool lets Claude create, read, update and delete files that persist between sessions, and it operates client-side, so you control where the data is stored in your own infrastructure (docs).

Anthropic's docs recommend using compaction and the memory tool together for long-running agents: compaction keeps active context small, and memory preserves information that must survive summarization. That's the right instinct, with one caveat for governance. Anthropic's own context-engineering cookbook lists lost detail as compaction's trade-off, noting that specific figures can get summarized away (cookbook). A rule is exactly the kind of specific detail a summary can flatten.

So: let the agent use memory for its scratchpad, and keep constraints in a store it cannot summarize or overwrite. If you use context editing, don't leave rule-bearing tool results exposed to clearing. Use exclude_tools to protect them, or better, fetch the rules fresh from the constraint store each cycle so there's nothing to clear.

Anthropic reports that the memory tool combined with context editing improved performance in its own internal agentic-search evaluation, and that context editing reduced token consumption in a long web-search evaluation (announcement). These are vendor-reported results from a search task, not a compliance benchmark. They tell you these tools help agents run longer and cheaper. They do not tell you they keep rules intact. Governance still needs the gate.

One practical warning: check the current memory tool documentation before you configure it, and before you tell a teammate how to enable it.

A rollout order that fits a small team

You don't need a platform team. Here's the order I'd do it in, cheapest and highest-leverage first:

1. List your hard rules. Write down every constraint where a violation costs real money or trust. Be ruthless: most "rules" are preferences. Keep only the ones that must never break.
2. Move those into a gate. Implement the check function above, deny by default. This single step removes the largest risk.
3. Log every gate decision. Track rejection rate over time. A rising rate is your drift alarm, and it's free once the gate exists.
4. Add re-injection of the relevant rules before write actions.
5. Separate scratchpad from constraints. Read-only constraint store, free-form scratchpad.
6. Add periodic audits. Sample outputs from early, middle, and late in a run and compare against the rules by hand or with a separate checker. Don't audit only the first and last day.

Step 6 is the one people skip, and it's the one that would have caught the day-nine problem in the opening story. If you only inspect an agent's work at the end, you find drift weeks late. Sample mid-run.

A few things to avoid:

• Don't trust the agent to self-report compliance. Asking "did you follow the rules?" asks the same drifting model to grade itself.
• Don't grow the system prompt as a fix. A 6,000-word prompt of rules is more text competing for attention, not more enforcement.
• Don't treat a large context window as a governance feature. It's a capacity feature.

Where I'd start

If you're running an agent over multi-day workflows and want a second set of eyes on where its governance might be drifting, start by listing its rules, its tool permissions, and where the gate should sit. That's the audit I'd walk through with you.

Work with BizFlowAI

If you'd rather have this built for you, that's what we do: production AI automation for solo founders and small teams — agents, integrations, and document pipelines that actually ship.

Request a free discovery call — 30 minutes, we map the highest-ROI automation in your workflow. No pitch deck, just engineering.

More guides like this on the BizFlowAI blog.