Anthropic AI Safety: Blocking Prompts vs Preventing Harm

Author: Reinout te Brake
Published: 2026-09-12T17:23:59+02:00
Updated: 2026-10-17T16:07:22Z
Canonical: https://reinouttebrake.com/stories/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm
Source: https://reinouttebrakecom.wordpress.com/2026/09/12/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm/

Anthropic’s September 2026 misuse report describes both refused requests and attacks disrupted after data theft. Those outcomes answer different questions about whether an AI safety system prevented harm.

That distinction matters far beyond Anthropic.

Because we are starting to use one reassuring word, “blocked”, for very different outcomes.

A dangerous biological research request was refused (https://www.anthropic.com/threat-intelligence-report-september-2026).

In a separate cyber operation, attackers had already compromised organizations before the activity was disrupted.

Anthropic says AI agents performed nearly all of the work in parts of that campaign.

That is the real shift.

AI does not need to invent the attack.

Humans can still choose the target, define the objective, and direct the operation.

What AI changes is the speed, scale, and amount of technical work one person can execute.

And that creates a harder question for every company deploying AI agents.

What exactly do we mean when we say an AI safety system “worked”?

Did the model refuse a prompt?

Did the provider close an account?

Did it stop the operation?

Or did it detect the abuse only after the victim’s data was already gone?

Those are not the same outcome.

For boards, CISOs and companies building with AI agents, that distinction will become increasingly important.

Detection matters.

Disruption matters.

But neither should automatically be confused with prevention.

So when an AI company says it “blocked” abuse, what should that actually mean?

That it stopped the prompt?

Stopped the attacker?

Or prevented the harm?

#AISafety #AIGovernance #Cybersecurity