Engineering
9 min read
Every write an agent makes has a reversal cost, and that cost should decide how much ceremony surrounds it.
Most agent safety discussions treat all writes as one category and then argue about how much human oversight to apply. That framing guarantees a bad answer, because the correct amount of oversight for logging a call and the correct amount for merging two accounts are not close to each other.
Reversal cost as the organising idea
Logging a call is nearly free to undo. Nobody is harmed by an extra activity record. Deleting it takes a moment and leaves no trace of consequence.
Merging two accounts is at the other end. It is destructive, it is hard to reverse cleanly, and it touches records other people depend on. Related records move. History is rewritten. Someone else's report changes without them knowing.
Between those extremes sits most of what an agent actually does, and the useful question for each action is simply how much work it would take to put things back.
Free to reverse: creating an activity, adding a note, drafting content that stays in draft.
Cheap to reverse: updating a field where the previous value was captured.
Expensive to reverse: overwriting a populated field, reassigning ownership.
Effectively irreversible: merges, deletes, anything that triggers downstream automation.
Confirmation fatigue is a real failure mode
Treating all four tiers the same is the mistake, and it fails in both directions.
Confirm everything and you get confirmation fatigue, which is a security problem dressed as a safety feature. A user who has approved forty harmless actions today will approve the forty first without reading it, and that is the one that mattered. You have not added a control, you have added a ritual.
Confirm nothing and the first bad merge becomes an incident review, followed by a blanket policy that the agent may no longer write anything, which ends the project.
The useful design puts each action in a lane based on reversal cost, and only the expensive lanes get a human. The cheap lanes get logging instead.
Audit depth should scale the same way
Reversal cost also tells you where to spend on audit trails, which is otherwise an unbounded ask.
For additive actions, knowing that something happened is enough. A row saying the agent created a task at a given time covers it.
For destructive ones you need four things: the before value, the after value, the reasoning the agent recorded, and a link back to the input that triggered it. Without all four you cannot answer the only question that matters after an incident, which is whether the agent was wrong or the data was.
Capture the before value at the moment of write, not by reading history later.
Store the agent's stated reason, even when it is wrong. Especially when it is wrong.
Link to the source input so a reviewer can judge the read, not just the write.
Make the audit row mandatory in code rather than configurable.
Designing the lanes first
The practical sequence is to classify every action the agent can take before writing any instructions.
List the actions. Assign each a reversal cost. Decide which lane it falls into. Then build the confirmation and audit behaviour into the action itself, so it cannot be bypassed by a differently worded request.
This matters because a guardrail expressed in the prompt is a suggestion, and a guardrail expressed in the action is a property. If the destructive action structurally requires a confirmation token, no phrasing gets around it.
The short version
Sort actions by how expensive they are to undo. Give the expensive ones a human and a full audit trail. Let the cheap ones run quietly with a log line.
Design the lanes first. The prompt is easy once you know which actions are allowed to be quiet, and the system is safer than one that asks permission for everything and therefore for nothing.