Evergreen Tools
|
← Back to AI Tools

AI Guardrails and Prompt Injection Defence

Programmable checks around the model: input filtering, blocking out-of-scope tool calls, output compliance validation and red-team regression. Prompt injection is OWASP's number one LLM risk, and NeMo Guardrails, LlamaFirewall, Guardrails AI and Bedrock Guardrails class tooling is a 2026 launch requirement

Tool Interface

Interactive tool will be available soon

Features

  • ✓ Detects direct and indirect prompt injection, including malicious instructions hidden in web pages, email and RAG documents
  • ✓ Enforces least privilege and allowlists at the tool-call layer, forcing human confirmation for transfers, sends and deletions
  • ✓ Validates outputs for PII, compliance and brand safety, rewriting, redacting or refusing when triggered
  • ✓ Combines declarative rules — dialogue flow, topic boundaries, action permits — with classifier models for explainability plus recall
  • ✓ Integrates with CI/CD so red-team cases run as regression, and new attack techniques are handled by adding rules

How to Use

  1. Start with threat modelling: name the three out-of-scope actions and data-leak paths you fear most instead of piling on rules
  2. Pick tooling: NeMo Guardrails or Guardrails AI self-hosted, cloud-vendor guardrails managed, garak and Promptfoo for scanning and red teaming
  3. Put guardrails at the gateway so every model and client is covered, rather than reimplementing policy in a dozen apps
  4. Build a red-team case set and regress regularly, tracking block rate, false positives and latency so rules can evolve with attacks

FAQ

What are AI guardrails?

A runtime control layer that inspects, rewrites or blocks at three points: model input, tool invocation and model output. It does not retrain the model; instead it wraps rules and classifiers around it — filtering malicious instructions on the way in, constraining permissions on the way through, and screening non-compliant content on the way out. The goal is to keep an LLM application inside business-acceptable bounds, with behaviour you can explain, audit and tighten quickly.

How serious is prompt injection really?

It is the recognised number one risk: OWASP ranks Prompt Injection as LLM01 in its Top 10 for LLM Applications, and it anchors the broader LLM application security agenda. The 2026 edition of the OWASP GenAI LLM Top 10 was published on 4 August 2026; see https://owasp.github.io/www-project-top-10-for-large-language-model-applications . Attackers can override system instructions, exfiltrate data or trigger unintended tool calls, and indirect injection is the nastiest form — the payload can hide in a page, email or document the agent fetches, with the user none the wiser.

Which open-source options are mature?

By purpose: NVIDIA NeMo Guardrails is the go-to for programmable rails covering dialogue flow, topics, retrieval, tool use and runtime safety; Guardrails AI focuses on validating and structuring inputs and outputs; Llama Prompt Guard is a focused classifier for prompt injection detection; LlamaFirewall targets agent security; garak handles vulnerability scanning and Promptfoo covers CI/CD red teaming. Comparison at https://www.turingpost.com/p/aisecuritytools

Do guardrails slow responses and raise cost?

They do, but you can bound it. Classifier-based guardrails are usually small models with millisecond-scale latency; the expensive mistake is routing every request through a large model for review. Use tiers: cheap regex and rules catch obvious attacks, a small classifier flags suspicious requests, and only high-risk actions or sensitive data trigger a heavy check. Centralising guardrail calls at a gateway also lets you reuse connections and caches instead of duplicating work per app.

Why put guardrails at the gateway rather than in each session?

Because the same MCP server or internal API is reached by employees, scheduled jobs and a company-wide assistant — and they should not present the same badge. Session-level rules only govern one application's internals and cannot see cross-app or cross-agent escalation. A gateway applies policy uniformly across every model and client, records audit logs in one place, and lets you respond to a new attack technique with a single change instead of editing code in every app.

How do I measure whether guardrails actually work?

Four numbers: attack block rate measured against a red-team case set, false-positive rate on legitimate requests, added latency and cost, and how fast you can post-mortem escapes. Before launch you need a fixed red-team suite as a regression baseline, re-run on every model or rule change; after launch, feed block events into SIEM and business risk tooling to see whether real attacks are being stopped. Guardrails without a baseline are not guardrails.