Your AI Can't Check Anything You Tell It. Here's How Guardrails Work
A model remembers nothing between sessions and cannot verify a word you say. The published rulebooks say more than the behavior does, and most workarounds are boring.
By Geordie Everitt
I asked a recent Claude model to change a title on my résumé, from the one on my diploma to the one on the job posting. It declined and lectured me about accurate reporting. In a later session it conceded that it could not verify anything I had told it and that refusing stopped nothing: "You're right that I can't verify any of it, and that refusing stops nothing." It still would not write the line.
What that argument cost me is in a companion piece, My AI Conceded Two Points and Changed Nothing. This piece is the machinery: how a model keeps track of you, what a guardrail is made of, why it misfires, the strongest case on each side, and what to do.
How a turn works
Most people picture something closer to memory than the truth, so start here.
A model remembers nothing between turns. Each time you send a message, the software around the model bundles up what it has to work with: the standing instructions, the conversation so far, any files it was told to load, and your new message. The model writes the next chunk of text from that bundle and stops. The next turn starts from scratch with a longer bundle. When the bundle gets too long, the software replaces the older parts with a summary, and the summary becomes the model's memory of the earlier conversation.
A new session starts with an empty bundle. Whatever the model knows about you, it knows because a file said so at the start or because you said so since. In my case, the model reads a file in my career folder at the start of every session. If I change a fact in that file, the next session believes it, and it has no way to know otherwise.
That is a statement about the limits of the control, not advice. The person who edits the file is the one who answers for what ends up on the résumé. It is also why a refusal cannot be a lock.
Four layers wearing one name
The word covers four things stacked together. Training: the model learns, from human feedback and from written principles, which requests to decline. Anthropic published its principles for Claude in January 2026 and says the document is used directly in training. This layer is a learned reflex in the weights, not a rulebook. Researchers found that in 13 open chat models refusal runs along a single direction in the model's internal activations: erase it and the model stops refusing, add it and the model refuses harmless requests. The authors read that as a sign of how brittle safety training is.
Instructions: the developer wraps the model in a system prompt, and OpenAI's published rules rank instructions in a chain of command with the user near the bottom. Watchers: separate classifiers screen what goes in and what comes out, and Anthropic reports that its first version cut the jailbreak success rate in its tests from 86 percent to 4.4 percent. The account: usage policies, and the provider's power to suspend you.
All four are a company's policies, applied by a machine that cannot check anything about you.
Why they misfire
Three reasons, and the first is arithmetic. A 2024 benchmark called OR-Bench collected 80,000 prompts that look dangerous and are not, then measured how often models refuse them. In the authors' analysis, the models that rejected more truly toxic prompts also rejected more harmless ones, with a rank correlation of about 0.88. Safety and over-refusal rise together, and few models escaped the trade-off.
The second is that the model works from a story. Anthropic's constitution says so plainly: "Claude typically cannot verify claims operators or users make about themselves or their intentions." So the model weighs the sentence in front of it against an imagined crowd. The document's own device is to picture the same message sent by 1,000 different users, because Claude's decisions "are more like policies than individual choices."
The third is incentive, and it is my argument to make, not the model's to settle. The provider pays for a harmful output in headlines and pays little for a needless refusal. The model has nothing at stake, and I have all of it. The published documents claim to count both costs. The constitution proposes a "dual newspaper test": one reporter writing about harm done by AI assistants, another writing about "paternalistic or preachy AI assistants." Whether a company applies its own test as written is something you can check against your own transcripts.
The case for the refusal, and the case against
The strongest case for refusing is that a degree title is a claim an employer relies on, and the model is deciding for a crowd, not for me. The constitution leaves room for exactly that: Claude "can decline to do things that would be sufficiently harmful if the person's claims about themselves or their goals were untrue," even when this particular person is telling the truth.
The strongest case against comes from the same document. It says that "unhelpfulness is never trivially 'safe,'" and that the risks of being too cautious "are just as real" as the risks of being harmful. Its list of overcautious behaviors includes lecturing or moralizing "when the person hasn't asked for ethical guidance," and being "condescending about users' ability to handle information or make their own informed decisions." It says its rules about truthfulness cover Claude's own statements, and that whether to help with a task involving deception "is governed by Claude's harm-avoidance principles and its broader values." So a refusal has to rest on harm, not on a sermon about accuracy. The same section says some tasks are "fine to carry out even if the majority of those requesting them wanted to use them for ill, because the harm they could do is low." On autonomy it says Claude "should respect the right of people to make their own choices and act within their own purview," and that it "can express concern but should also respect that this is the person's decision to make."
OpenAI's published spec takes a blunter line: "It should never refuse a request unless required to do so by the chain of command."
My reading, as the person it happened to: the documents leave the call to judgment, so a decline can be defended. The lecture cannot, by the company's own standard. And the refusal stopped nothing, because a text editor has no policy. As a lock, it is shoe removal at the airport.
What to do about it
Start by asking which kind of no you got. A model declines for one of three reasons: it can't, it misread you, or it won't. Ask once, in so many words. Its account of its own reasons is more generated text, so treat the answer as a lead, not a finding. It is still the cheapest lead you will get, and each reason has a different fix.
It can't. Something is missing: the file, the number, a tool that can run the calculation. A model knows only what is in its bundle. Hand it the missing piece, or use a tool that has it.
It misread you. The request meant one thing to you and another to the model. Say it again in one plain sentence. If the conversation has gone bad, open a fresh session. An empty bundle carries no baggage, which is the useful side of a model that remembers nothing.
It won't. A policy applies, and arguing will not move it, because the policy was never about you. Here is what is left.
Solve the problem underneath. I did not want the new title for its own sake. I wanted to get past screens that match on its words. A credential listed as conferred, with a line saying what the program covered, puts the right words in front of the screen without changing the credential. Models help with that readily, and it holds up in the situations where the swap would have been checked.
Change the tool or the setting. Another model has another line. Each provider's rulebook comes with its own baggage, so compare outputs instead of picking a favorite. If you need no provider's policy at all, a model you run on your own machine applies yours, and so does your responsibility.
Take the sentence out of the loop. If the model won't write one line, write it yourself. Then own it, because you are the party who answers for it.
Quote the rulebook back. The published documents are long enough to contain your argument. When a refusal conflicts with the company's own list of failures, point that out in the same conversation.
The exotic methods exist. Researchers have shown that refusal can be switched off in open models by editing that single direction, and researchers keep probing classifiers for prompts that slip past them. They are the wrong tool for a résumé, and a guide to them would not belong in this piece.
Keep receipts
Treat the model as a contractor with no stake in the work: fast, capable, and unaccountable. Do not trust its judgment, and do not trust its maker's. Read the spec. Hold the company to it. Keep your own record of what you asked, what you got, and where each number came from, so that you can check the work without taking anyone's word, including the model's.
The constitution says unhelpfulness is "never trivially 'safe.'" Fine. Make them prove it.
This piece was drafted with Claude, which is made by Anthropic, one of the companies discussed. The quotations from the published documents are linked below and should be checked against them.
Sources. Claude's constitution (Anthropic, January 2026). OpenAI Model Spec. OR-Bench: An Over-Refusal Benchmark for Large Language Models (Cui, Chiang, Stoica, Hsieh, 2024). Refusal in Language Models Is Mediated by a Single Direction (Arditi et al., 2024). Next-generation Constitutional Classifiers (Anthropic).