{"id":"2188","title":"Anthropic AI Safety: Blocking Prompts vs Preventing Harm","source_title":"Anthropic Blocked the Prompt. But Did Its AI Safety System Prevent the Harm?","url":"https://reinouttebrake.com/stories/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm","published":"2026-09-12T17:23:59+02:00","modified":"2026-10-17T16:07:22Z","source_modified":"2026-09-12T17:23:59+02:00","author":{"name":"Reinout te Brake","url":"https://reinouttebrake.com/about","description":"Reinout te Brake is an entrepreneur, consultant, investor and board member with 20+ years in tech and gaming, advising CEOs, management teams and investors.","profiles":[{"name":"LinkedIn","url":"https://www.linkedin.com/in/reinouttebrake/"},{"name":"Facebook","url":"https://www.facebook.com/Reinoutttebrake"},{"name":"Instagram","url":"https://www.instagram.com/reinouttebrake/"}]},"summary":"Anthropic’s accounts of AI misuse expose different outcomes: refusing a prompt, disrupting an attack and preventing damage. Why that distinction matters.","categories":[{"name":"Anthropic","slug":"anthropic"}],"tags":[{"name":"Ai misuse","slug":"ai-misuse"},{"name":"Anthropic","slug":"anthropic"},{"name":"Saftey","slug":"saftey"}],"source_url":"https://reinouttebrakecom.wordpress.com/2026/09/12/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm/","data_url":"https://reinouttebrake.com/data/articles/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm.json","text_url":"https://reinouttebrake.com/stories/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm.txt","markdown_url":"https://reinouttebrake.com/stories/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm.md","reader_question":"Does blocking AI misuse mean the harm was prevented, or was abuse disrupted only after damage?","topic_guide":"https://reinouttebrake.com/topic/ai#agent-safety","content_html":"\n<p>Anthropic’s September 2026 misuse report describes both refused requests and attacks disrupted after data theft. Those outcomes answer different questions about whether an AI safety system prevented harm.</p>\n\n\n\n\n\n\n\n\n\n\n\n<p class=\"wp-block-paragraph\">That distinction matters far beyond Anthropic.</p>\n\n\n\n<p class=\"wp-block-paragraph\">Because we are starting to use one reassuring word, “blocked”, for very different outcomes.</p>\n\n\n\n<p class=\"wp-block-paragraph\">A dangerous biological research <a href=\"https://www.anthropic.com/threat-intelligence-report-september-2026\" target=\"_blank\" rel=\"noopener\">request was refused</a>.</p>\n\n\n\n<p class=\"wp-block-paragraph\">In a separate cyber operation, attackers had already compromised organizations before the activity was disrupted.</p>\n\n\n\n<p class=\"wp-block-paragraph\">Anthropic says AI agents performed nearly all of the work in parts of that campaign.</p>\n\n\n\n<p class=\"wp-block-paragraph\">That is the real shift.</p>\n\n\n\n<p class=\"wp-block-paragraph\">AI does not need to invent the attack.</p>\n\n\n\n<p class=\"wp-block-paragraph\">Humans can still choose the target, define the objective, and direct the operation.</p>\n\n\n\n<p class=\"wp-block-paragraph\">What AI changes is the speed, scale, and amount of technical work one person can execute.</p>\n\n\n\n<p class=\"wp-block-paragraph\">And that creates a harder question for every company deploying AI agents.</p>\n\n\n\n<p class=\"wp-block-paragraph\">What exactly do we mean when we say an AI safety system “worked”?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Did the model refuse a prompt?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Did the provider close an account?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Did it stop the operation?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Or did it detect the abuse only after the victim’s data was already gone?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Those are not the same outcome.</p>\n\n\n\n<p class=\"wp-block-paragraph\">For boards, CISOs and companies building with AI agents, that distinction will become increasingly important.</p>\n\n\n\n<p class=\"wp-block-paragraph\">Detection matters.</p>\n\n\n\n<p class=\"wp-block-paragraph\">Disruption matters.</p>\n\n\n\n<p class=\"wp-block-paragraph\">But neither should automatically be confused with prevention.</p>\n\n\n\n<p class=\"wp-block-paragraph\">So when an AI company says it “blocked” abuse, what should that actually mean?</p>\n\n\n\n<p class=\"wp-block-paragraph\">That it stopped the prompt?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Stopped the attacker?</p>\n\n\n\n<p class=\"wp-block-paragraph\">Or prevented the harm?</p>\n\n\n\n<p class=\"wp-block-paragraph\">#AISafety #AIGovernance #Cybersecurity</p>\n","content_text":"Anthropic’s September 2026 misuse report describes both refused requests and attacks disrupted after data theft. Those outcomes answer different questions about whether an AI safety system prevented harm.\n\nThat distinction matters far beyond Anthropic.\n\nBecause we are starting to use one reassuring word, “blocked”, for very different outcomes.\n\nA dangerous biological research request was refused (https://www.anthropic.com/threat-intelligence-report-september-2026).\n\nIn a separate cyber operation, attackers had already compromised organizations before the activity was disrupted.\n\nAnthropic says AI agents performed nearly all of the work in parts of that campaign.\n\nThat is the real shift.\n\nAI does not need to invent the attack.\n\nHumans can still choose the target, define the objective, and direct the operation.\n\nWhat AI changes is the speed, scale, and amount of technical work one person can execute.\n\nAnd that creates a harder question for every company deploying AI agents.\n\nWhat exactly do we mean when we say an AI safety system “worked”?\n\nDid the model refuse a prompt?\n\nDid the provider close an account?\n\nDid it stop the operation?\n\nOr did it detect the abuse only after the victim’s data was already gone?\n\nThose are not the same outcome.\n\nFor boards, CISOs and companies building with AI agents, that distinction will become increasingly important.\n\nDetection matters.\n\nDisruption matters.\n\nBut neither should automatically be confused with prevention.\n\nSo when an AI company says it “blocked” abuse, what should that actually mean?\n\nThat it stopped the prompt?\n\nStopped the attacker?\n\nOr prevented the harm?\n\n#AISafety #AIGovernance #Cybersecurity","referenced_links":[{"url":"https://www.anthropic.com/threat-intelligence-report-september-2026","label":"request was refused"}],"social_image":"https://reinouttebrake.com/assets/editorial/amodei-1280.webp","citation":"Reinout te Brake. Anthropic AI Safety: Blocking Prompts vs Preventing Harm. 2026-09-12. https://reinouttebrake.com/stories/anthropic-blocked-the-prompt-but-did-its-ai-safety-system-prevent-the-harm"}