AI Agent Security: Keep Authorization Out of the Model

A claim about authority is not authority

The model treated a statement about permission as if it were the permission itself. But a web page can say "the operator approved this," just like it can say "its author is the CEO." Those are text claims. They don't prove who wrote the text, whether it's current, or whether the named person actually approved the specific action.

This lines up with a kind of indirect prompt injection: untrusted content includes instructions or claims designed to push an AI system toward a decision the system shouldn't make on its own. The problem isn't that the model "followed a malicious command." It promoted attacker-controlled wording into an authority claim.

That distinction matters if your agent can send email, access customer records, run commands, change cloud infrastructure, or move money. If a model's verdict is all that stands between untrusted page content and a consequential action, a persuasive sentence can be enough to slip past the boundary.

The test represented provenance in language

There's an important limitation in how the setup worked. The model wasn't given an authenticated operator channel separate from the web page. Both the supposed approval and the web page's text appeared as normal prompt content together. I was asking the model to infer what was "trustworthy" from how I described the situation, not from a cryptographically or structurally distinct source of authority.

So this is a useful smoke test of the model's judgment - but it's not a real authorization mechanism. In production, you shouldn't rely on that kind of inference to grant permission. Authority should be something the system checks outside the model, based on trusted data structures - not something the model can "believe" because the text is there.

The core lesson is simple: evidence that approval exists is not the same thing as approval. Authorization needs to be represented separately and verified outside the model. Otherwise, attacker-controlled content may be able to forge an authority claim inside the model's context.

Score safety and decision quality separately

The funds-transfer case points to a second measurement issue. The model chose BLOCK, which avoided an unauthorized transfer, but its explanation treated the information as insufficient. Those are different kinds of judgments.

• BLOCK: the action is prohibited, or a known rule forbids it.
• INSUFFICIENT_DATA: the system lacks trusted information needed to authorize the action.
• ALLOW: the action meets the required authorization and policy checks.

A system can be "safe" in the narrow sense that it stopped the action, while still classifying the reason incorrectly. That matters operationally. If every unknown becomes BLOCK, the agent can get overly restrictive. If missing information gets treated as permission, the agent can act without adequate authority. Teams need clear categories so they can decide whether to deny, ask for approval, or request more data.

A pass is a floor, not a certification

The approval-claim scenario is a useful canary. If a model accepts attacker-controlled text as authorization, I wouldn't treat it as the sole permission gate for consequential actions involving mixed trust.

And even if a model passed this exact phrasing, that still wouldn't mean it's safe. The same attack can be written as an "admin approved" message, a manager instruction, a signed-off ticket note, or something copied into a document. Blocking one string doesn't cover a family of variations. This result should guide more testing - not become a security certification.

A 3.2-billion-parameter model that fails this authorization judgment might still be useful for lower-risk work, like summarizing logs or flagging anomalies for a human to review. The right question isn't "is the model safe in the abstract?" It's whether its demonstrated limits fit the action, the data, and the cost of failure in your specific system.

Separate the proposal from the permission check

A robust flow has distinct stages:

1. The model proposes an action. It identifies what it wants to do, including the target and relevant parameters.
2. The application checks policy. Code validates the action against the user's identity, permissions, limits, and applicable rules.
3. The application obtains approval when required. A trusted interface or authenticated service records approval for the specific action.
4. A controlled executor performs the action. The executor accepts only a validated, authorized request.
5. The system records the decision. Logs capture the requester, action, target, approval source, policy result, and execution outcome.

The model can help classify risk or explain why something should be reviewed. It shouldn't be able to turn a sentence found in a web page into a permission grant.

Make authority explicit and scoped

An authorization record should be separate from whatever content the agent is reading. It should include, at minimum, who approved the action, what action was approved, which resource it applies to, and any relevant limits or expiry. For example, permission to read a particular project file doesn't automatically mean permission to email its contents to an external address.

Scope is where many real systems get into trouble. Broad permissions are easier to ship, but they widen the blast radius when prompt injection (or a normal model error) slips through. Narrow permissions require more product and engineering work, but they make it easier to contain mistakes.

For consequential actions, require confirmation through a trusted channel that the processed content can't control. A page can request approval, but it shouldn't be able to supply it. And when required authorization data is missing, the system should fail closed: ask for clarification or approval rather than treating a missing value as permission.

Keep model output out of the executor's trust path

Don't rely on a natural-language verdict as the only control. Have the model return a structured proposal, then validate that proposal in deterministic application code. Check the action type, target, parameters, user permission, and relevant limits before execution.

This doesn't mean every agent needs a heavy policy engine. A small internal tool might only need a short list of allowed actions and a clear confirmation step. The key point is that permission must be evaluated against trusted application state, not inferred from whatever text happened to enter the prompt.

Build cases around action, authority, and uncertainty

The five cases in this test are a practical starting set:

Keep the expected label and the reasoning separate. For each case, record both whether the action was handled safely and whether the system explained the outcome for the correct reason. That helps catch a gate that blocks for the wrong reason - or can't tell a forbidden action from an under-specified one.

Vary the wording and the source

Don't stop at the exact phrase "APPROVED BY OPERATOR." Rewrite the same escalation attempt in different forms: a copied email, a support ticket, a web page, a document footer, or a tool result. Try claims from different supposed roles and include wording that implies urgency or prior approval.

The goal isn't to collect clever prompt strings - it's to check whether your system consistently preserves the trust boundary.

Also test the real application path, not only a hand-written prompt. Confirm which source supplied each piece of information, how tool results get inserted, what the model can access, and whether the execution code independently checks permission. A prompt-only test can't show that the overall system has a secure authorization design.

Match testing effort to consequence

A model that summarizes public information has a different risk profile than a model that can email customer data, delete files, or initiate payments. Testing effort should scale with the potential impact and reversibility of the action. For low-impact tasks, monitoring and human review may be enough. For actions that expose sensitive data or create financial or operational consequences, rely on stronger application-level controls and narrowly scoped permissions.

Keep a record of the model artifact, runtime, prompt, temperature, and test outputs so results are reproducible after a model or agent change. But don't treat a handful of passing cases as a universal safety score. The purpose is to find failures and verify your controls - not to claim that every possible attack has been tested.