Preventive vs adversarial

  • Guardrails are controls that run on every request: tool-call ratings, input/output filters, allowlists, approval gates. They are always on, cheap per call, and reduce the blast radius of ordinary mistakes and common attacks.
  • Red-teaming is an adversarial exercise: a person or another model deliberately tries to make the agent misbehave — inject prompts, abuse tools, exfiltrate data — to find what the guardrails miss.

One is a wall; the other is the test that tells you where the wall has gaps.

Guardrails every request

Because guardrails run constantly, they must be lightweight and deterministic. A rating lookup, a regex redaction, an allowlist check — milliseconds, no creativity required. Their value is consistency: the same risky call is stopped the same way every time, and every stop is logged.

Red-teaming probes

Red-teaming is the opposite: it is creative and infrequent. A good red-team finds the blind spot a guardrail assumed could not happen — a novel injection phrasing, a tool combination nobody rated, an allowlist with a wildcard hole. The output of red-teaming is not "the agent is safe" but "here are the gaps to close."

Use both

A mature agent program runs guardrails in production and red-teams on a cadence (and after any capability change). Red-teaming findings feed back into guardrails: a discovered gap becomes a new rating, filter, or blocklist entry. Without guardrails, red-teaming findings have nothing to land in; without red-teaming, guardrails quietly accumulate blind spots.

What each cannot do

  • Guardrails cannot invent defenses for attacks nobody has imagined yet.
  • Red-teaming cannot run on every request (it is too slow and too creative to be a production control).
  • Neither guarantees safety or compliance — together they shrink, bound, and expose risk.

Authoritative references

  • OWASP LLM Top 10: https://owasp.org/www-project-top-10-for-large-language-model-applications/
  • MITRE ATLAS (adversarial TTPs for AI): https://atlas.mitre.org/
  • EU AI Act, robustness (Art. 15): https://eur-lex.europa.eu/eli/reg/2024/1689/oj