Agents, permissions and responsibility.

The image of a rogue AI villain explains little about how operational damage actually happens. An agent does not need malicious intent. A misunderstood task or a convincing but incorrect conclusion can be enough when it leads to a consequential action.

The more practical question is which action is possible, with what data and under whose responsibility.

An error meets a permission

Two agents can produce the same mistaken proposal. One saves it as a draft. The other is allowed to execute it directly in production. The model error is similar; the potential impact is very different.

This brings decisions made by people and organisations into focus. Which accounts are provided? Which systems are reachable? Who allowed an exception, and does it still apply?

That does not mean people are the only source of errors. It means the operational boundaries need to hold even when an agent behaves unexpectedly.

Good intentions are not an access boundary

In How we contain Claude, Anthropic describes risks from model behaviour and external attacks. Instructions influence a model’s behaviour, but do not replace enforced access controls. Tool content can also misdirect an agent through prompt injection.

“Only diagnose the problem” is therefore different from an account that technically cannot make changes. Read access does not automatically protect sensitive information from unwanted disclosure either.

Responsibility shows up in the design

A useful design assigns ownership, limits access and makes executed actions traceable. Critical changes need appropriate review, and access must be revocable.

Treating agents as categorically harmless would be as unhelpful as portraying them as uncontrollable actors. Their operating conditions can be designed. That is where responsibility begins for a team bringing agents into its systems.