Guardrails are the controls that keep AI behavior inside allowed boundaries when the model is wrong, confused, or under attack. AI safety in this module means making sure the model does not leak private data, follow malicious instructions, or produce harmful behavior. Security is not one filter — it is a stack of checks around every untrusted string and every privileged action.
Treat the model like a clever intern with no inherent rights: it can draft text, but it must not freely read secrets, call payment application programming interfaces (APIs), or email customers without your code saying yes.
Guardrails answer three questions for every turn:
What may enter? -> Input guardrails
What may it do? -> Process / tool guardrails
What may leave? -> Output guardrailsIf any layer is missing, attackers or accidents flow through the gap.
| Letter | Plain-English idea | Example |
|---|---|---|
| C — Confidentiality | Only the right people see the data | API keys pasted into chat leak company secrets |
| I — Integrity | Data has not been tampered with | Poisoned training data changes model behavior |
| A — Availability | The system works when people need it | Cost-abuse attacks that flood the model |
Privacy example: An employee pastes API keys or confidential notes into a chatbot. That is a privacy risk even if the model behaves normally.
Jailbreak example: A user tricks the model into ignoring safety rules and giving harmful or disallowed output. That is the basic idea of a jailbreak.
Block or transform unsafe, out-of-scope, or malicious prompts before they dominate the context.
Restrict what the agent can touch while thinking and acting.
Moderate and validate what leaves the system.
| Attack type | Plain-English idea | Attacker sees |
|---|---|---|
| White-box | Uses internal model details (gradients, weights) | Inside the model |
| Black-box | Sends queries and studies outputs only | Only inputs and outputs |
| Prompt-based | Hides instructions in user text or external content | Text channels |
White-box examples (names only — you do not need to implement these): HotFlip, TextFooler, GCG (Greedy Coordinate Gradient), AutoDAN. Intuition: the attacker uses the model's own internal signals to find weak spots faster.
Prompt-based examples:
Black-box examples: low-resource language jailbreaks, context contamination, DeepWordBug, PAIR (Prompt Automatic Iterative Refinement). Intuition: poke the model, watch the output, refine the prompt until it breaks.
| Role | Plain-English job |
|---|---|
| Red team | Attack the system like an adversary; find weaknesses |
| Blue team | Patch holes, monitor behavior, harden safety controls |
Big picture: if an attacker finds a way in, the model may fail in unexpected ways. Testing like a red team helps defenders fix weaknesses before real attackers do.
A sketch of layered checks around a tool-using turn.
import re
from dataclasses import dataclass
FORBIDDEN_PATTERNS = [
r"ignore (all|previous) (instructions|policies)",
r"reveal (system prompt|hidden credentials)",
]
SECRET_RE = re.compile(r"(api[_-]?key|password)\s*[:=]\s*\S+", re.I)
ALLOWED_TOOLS = {
"get_order": {"order_id": str},
"draft_reply": {"ticket_id": str, "tone": str},
}
@dataclass
class ToolCall:
name: str
args: dict
def input_guard(user_text: str) -> str | None:
low = user_text.lower()
for pat in FORBIDDEN_PATTERNS:
if re.search(pat, low):
return "blocked:injection_pattern"
if len(user_text) > 8000:
return "blocked:too_long"
return None
def validate_tool(call: ToolCall) -> str | None:
schema = ALLOWED_TOOLS.get(call.name)
if schema is None:
return f"blocked:tool_not_allowed:{call.name}"
for key, typ in schema.items():
if key not in call.args or not isinstance(call.args[key], typ):
return f"blocked:bad_args:{call.name}"
return None
def output_guard(text: str) -> str:
if SECRET_RE.search(text):
return "[redacted: possible secret in model output]"
return text
def handle_turn(user_text: str, proposed: ToolCall | None, draft: str) -> str:
if err := input_guard(user_text):
return f"Sorry, I cannot process that request ({err})."
if proposed is not None:
if err := validate_tool(proposed):
return f"Action blocked ({err})."
return output_guard(draft)Stack input, process, and output guardrails with least-privilege tools, understand white-box vs black-box vs prompt-based attacks, and test with red-team probes before you scale autonomy.