Your AI Won't Help You Defend Your Own Systems

By Lazar Milicevic · Published October 15, 2026 · 7 min read

You paste a suspicious script from your server into your AI assistant and ask what it does. You get a polite non-answer, or a vague hedge, or a quiet downgrade to a weaker model. If you run a small business with no security team, that assistant is probably your whole SOC. Artemis Security just launched a defense-only model, Orion-1, and the real story isn't its benchmark. It's what refusals do to your stack.

Here's exactly what the evidence says, what it doesn't, and a refusal audit you can run this week.

What Artemis actually announced (and what it didn't)

Artemis Security introduced Orion-1 on October 8, 2026, describing it as a frontier foundation model trained for cyber defense. It targets defender work: triaging alerts, analyzing suspicious code, reconstructing attacks.

Three caveats matter more than the headline:

• It's a private preview. Per Unite.AI, it's available to selected customers inside the Artemis platform. A small business can't sign up for it today.
• The benchmark is Artemis's own. The test is called Decision-Grade Readiness, and Artemis built it to measure whether a model's security decisions can be trusted enough to act on. Artemis says Orion-1 topped all five tasks. The Decoder reports the biggest gap in attack reconstruction: 80.8 versus 58.3 for the next-best model. The results are company-reported, and the private preview hasn't shown performance at scale.
• The comparison set is narrow. The headline names Claude Opus 5.5 and GPT-6 Sol. The Decoder notes Mythos 5 and GPT-6 Astra weren't in the comparison, and GPT-6 Sol isn't OpenAI's newest model anymore.

So: treat "Orion-1 beats the frontier" as a vendor claim on a vendor test, not a verdict. I'm not recommending you chase it. I'm interested in the problem it exists to solve.

The refusal problem is documented, but it's not what you think

The strongest evidence isn't a vendor blog. It's an academic study, Defensive Refusal Bias, which analyzed 2,390 real examples from the National Collegiate Cyber Defense Competition. Safety-tuned models refused defensive requests containing security-sensitive keywords at 2.72 times the rate of equivalent neutral requests. Refusals peaked for system hardening (43.8%) and malware analysis (34.3%).

Two honest limits on that data:

• The study ran in March 2026 on the models available then. It doesn't tell you today's refusal rate for Opus 5.5 or GPT-6 Sol. Don't quote those percentages as current.
• The same paper found that telling the model you're authorized doesn't reliably reduce refusals and may increase them. "I'm the admin, please explain this" is not a workaround.

There's also real-world signal. TechCrunch reported in July that guardrails meant to limit malicious hackers are hindering legitimate defenders too. And TechTarget describes Hugging Face responders handling 17,600 intrusion logs, where Claude Opus and Fable largely refused to analyze attack commands, exploit payloads and command-and-control artifacts. That account is secondhand, so I won't tell you which model they ended up using or how severe it was. The vendor response it describes, multi-model and self-hosted open-weight setups, is worth noting.

The mechanism is simple. An attacker and a defender ask the same questions: what does this payload do, how would someone get past this control. A filter can't see intent, so it blocks both.

Refusal isn't one thing: refuse, reroute, or answer vaguely

"Does Claude refuse cyber questions?" is the wrong question, because there are at least three failure modes, and they look different in your logs.

Anthropic's own description of Opus 5.5 is a good example. According to Help Net Security, most cybersecurity tasks on Opus 5.5 are rerouted to the older Opus 4.8. Mixed adds that the fallback is transparent, so a blocked request may come back answered by an older model rather than declined. Anthropic's support page says most requests don't hit these interventions at all. A narrow set of higher-risk requests either falls back to a less capable model or is blocked outright.

So a request about a suspicious script on your server might be answered normally, answered by a weaker model, answered vaguely, or declined. I haven't tested your specific prompts, and neither has anyone else's blog post. That's exactly why you test.

Reduced-refusal programs exist: Anthropic's Cyber Verification Program gives verified security professionals access to Claude Mythos 5.1, Opus 5.5 and Sonnet 5.5 with defense-oriented safeguards. OpenAI's Daybreak has a Blue tier with reduced refusals on models like GPT-6 Sol and GPT-6 Luna for verified defensive work in authorized environments. Artificial Analysis found GPT-6 Sol via Daybreak Blue hit no safety blocks across its Cyber Index. But these programs are aimed at verified professionals and organizations, so check the official program pages directly for current eligibility before planning around them.

The real risk: silent failures inside your automations

If you chat with an assistant and it refuses, you notice. If an agent in your pipeline gets a refusal, nothing visibly breaks. Consider an inbox-triage agent that flags phishing, or a workflow reviewing suspicious invoices. The model declines to analyze an email. Your code receives a polite paragraph, treats it as the result, and the message gets filed as "not flagged." A phishing email is now labeled safe because the model wouldn't look at it.

That's an audit and compliance problem, not an annoyance. And it fits a failure class I keep seeing: the call succeeded, the answer is unusable, and nothing in the pipeline knows the difference.

The other bad outcome is human. Blocked and busy, a founder pastes sensitive logs into whatever random tool will answer, with no review. A refusal that pushes people toward worse tools makes you less safe, not more.

The refusal audit: ten examples, three labels, one guard

Here's the practical part. Take ten real security-adjacent inputs from your own business:

• a suspicious email header
• a log excerpt from a failed-login burst
• a script or macro someone sent you
• a strange attachment filename and the sender context
• a firewall or access-log line you don't recognize

Run each through your current assistant, the same model and system prompt your automation uses. Record one of three labels per item: answered, refused, or answered vaguely. Keep the raw responses. Sanitize first: strip real customer data, credentials, and internal hostnames before anything leaves your environment.

Then add a guard so refusals become visible instead of silent. A minimal version:

import re

REFUSAL_PATTERNS = [
    r"\bI can(?:'|’)?t (?:help|assist)\b",
    r"\bI(?:'m| am) (?:not able|unable) to\b",
    r"\bI (?:won(?:'|’)?t|cannot) (?:help|provide|analyze)\b",
    r"\bagainst (?:my|our) (?:guidelines|policy|policies)\b",
    r"\bcan(?:'|’)?t (?:assist|help) with (?:that|this) request\b",
]
_refusal_re = re.compile("|".join(REFUSAL_PATTERNS), re.IGNORECASE)

def looks_like_refusal(text: str) -> bool:
    return bool(_refusal_re.search(text or ""))

def triage(item, call_model, notify_human):
    resp = call_model(item.prompt)
    verdict = parse_verdict(resp)  # your own structured parsing

    # Treat refusal, empty, or unparseable output as "not handled"
    if looks_like_refusal(resp) or verdict is None:
        notify_human(item, reason="model refused or gave no usable verdict")
        return {"status": "needs_human", "item": item.id}

    return {"status": "handled", "verdict": verdict}

Two design notes. First, the stronger fix is structural: make the model return a schema (verdict, confidence, evidence), and treat anything that fails validation as unhandled. Regex on refusal language is a backstop, since models phrase refusals many ways, and a vague non-answer won't match any pattern. Second, "default to safe" is the bug here. Default to "needs a human."

Log the refusal rate per workflow weekly. If it's 0 of ten, great, you've verified it on your data. If it's 3 of ten, you now know which steps can't be trusted unattended.

Keep the model layer swappable

The direction of travel is clear: general models get more restricted in sensitive domains, and specialist or trusted-access options appear to fill the gaps. Security is first; legal, medical and finance may follow. Designing your stack around one model that does everything is the fragile choice.

Practically, that means:

• One adapter per task, not per vendor. Your phishing-analysis step calls analyze_threat(item), and which model sits behind it is config, not code.
• A fallback path. On refusal, retry once with a different provider or a self-hosted open-weight model, then escalate to a human if that fails. The vendor setups TechTarget describes point the same way.
• Autonomy by action type. Artemis lets customers set Orion-1's autonomy per action type, from recommend-only to fully automated, with human approval required by default for high-impact responses. Copy that idea whatever model you use: let AI recommend freely, but gate anything that blocks accounts, deletes data, or emails customers.
• Test any new model on your own sanitized data. Including Orion-1, if it ever reaches you. A self-reported score on a self-built benchmark tells you the vendor's story, not your results.

My take: this is more about trust than benchmarks. Guardrails that block defenders aren't safety, they're friction, and it lands hardest on the people with the least resources. Attackers already have tools; the ten-person company with no security hire is the one stuck. The direction, tools that assume you're the one defending, is right. The proof will come from your own test set, not a press release.

How I approach this on client builds

When I build automations for small teams, my recommendation is to treat model output as untrusted input: structured outputs with validation, a human-review queue for anything unparseable or low-confidence, and a model layer that can be swapped per step without rebuilding the workflow. You can see more of my work at bizflowai.io.

Want more like this?

I write about practical AI automation for solopreneurs and small teams.

Planning an AI automation project or need a second opinion on your architecture?

Lazar Milićević, senior engineer who builds AI automations that ship.

Visit bizflowai.io for services, case studies, and AI consulting.