Content moderation is necessary, but it is not an authorization system.
It can help classify harmful language, disallowed topics, or unsafe model outputs. It generally cannot decide whether a particular agent may query a customer record, send an external message, modify infrastructure, or delegate a permission to another agent.
That difference is the action gap.
Safe language can produce an unsafe action
Consider an illustrative scenario: a support agent produces a courteous answer while retrieving a record outside the caller's account. The language may pass every content check. The data access is still unauthorized.
This is not a reported customer incident. It shows why output safety and runtime authority must be evaluated separately.
An action-aware control needs context that a content classifier may not have:
- the agent's identity and tenant;
- the user or workflow that delegated authority;
- the requested tool action and resource;
- the policy version currently in force;
- the consequence and reversibility of the side effect;
- whether approval is required;
- whether the request is fresh, replayed, or delegated.
Model alignment is not least privilege
Training can influence what a model tends to choose. Least privilege constrains what the surrounding system permits. A well-aligned model should still operate behind narrow credentials and explicit policy because model behavior and system authority fail in different ways.
Build both layers
A production architecture can use content controls for prompts and outputs while using runtime governance for actions:
- Classify and sanitize input where appropriate.
- Give the agent a distinct identity.
- Translate tool intent into a declared action and resource.
- Evaluate that action before execution.
- Require approval for consequential operations.
- Record the decision and result.
- Inspect tool output before returning it to the model or user.
Coverage claims must name the enforcement boundary. Actions that bypass the instrumented tool path are not governed merely because a content filter is present elsewhere in the system.
The mistake is not investing in moderation. The mistake is calling moderation the complete safety architecture for software that can act.