Guardrails

Not a prompt asking the model to be careful. Guardrails are the controls around the model that limit what it can do when it gets something wrong — and it will get things wrong, so the design has to assume it.

The ones that carry the weight#

Confidence thresholds. Below an agreed threshold, the case goes to a human queue rather than through. This single control is what makes the automated majority trustworthy.

Scoped tool authority. An agent acts with the signed-in user's real entitlements, enforced at query time — not with a service account that can see everything. Irreversible actions above a threshold route to a person regardless of confidence.

Input handling. Untrusted content is never treated as instruction. This is the defence against prompt injection, and it is architectural rather than a matter of wording.

PII minimisation. Redact or tokenise before anything leaves your environment, and scrub before anything reaches a log line.

Cost and rate limits. So a runaway loop produces an alert, not an invoice.

Audit logging. What was asked, what context was retrieved, what was answered, by which model version. This is also what your risk function will eventually require.

What does not work#

Instructions alone. "Do not hallucinate", "never give financial advice", "always be accurate" have close to no reliable effect, because the model has no separate signal telling it which of its outputs are grounded. Prompt wording helps at the margin; controls are what hold.

The design question worth asking#

Not "how do we stop it being wrong" — you cannot. Ask "what is the worst thing this can do when it is wrong, and what stands between it and that outcome". If the answer is a sentence in the prompt, there are no guardrails.