Building a Real-Time Prompt Injection Detection System for LLM Applications Using Guardrails AI, LangChain, and OpenTelemetry

September 21, 2026

As large language models become embedded in production applications, prompt injection attacks have emerged as a critical threat vector. Malicious inputs can hijack model behavior, leak sensitive data, or trigger unauthorized actions. This guide walks you through building a real-time detection system that intercepts, validates, and monitors every prompt before it reaches your LLM.

Understanding Prompt Injection and Why Real-Time Detection Matters

Prompt injection occurs when an attacker crafts input that overrides the model’s original instructions. Unlike traditional SQL injection, these attacks exploit the model’s natural language understanding. Common variants include direct instruction overrides, context manipulation, and data exfiltration attempts.

Real-time detection is essential because LLM applications often process user input continuously — chatbots, autonomous agents, and document processors cannot afford batch scanning delays. A detection system must operate within the request latency budget while providing audit trails for security teams.

Key Risks in Production LLM Applications

Unauthorized function calling via injected tool descriptions
Extraction of system prompts or proprietary context
Bypassing content filters to generate harmful outputs
Chaining attacks across multi-step agent workflows

Architecture Overview: Guardrails AI, LangChain, and OpenTelemetry

The detection pipeline consists of three integrated layers. Guardrails AI provides the validation engine with pre-built and custom validators for prompt injection patterns. LangChain supplies the callback hooks that intercept prompts at runtime. OpenTelemetry instruments the entire flow for distributed tracing, metrics, and alerting.

Data Flow Description

User request enters the LangChain application (chain, agent, or LLM call)
Pre-call callback sends the raw prompt to Guardrails for validation
Guardrails runs injection detectors (heuristic, embedding-based, or LLM-as-judge)
If validation fails, the request is blocked and a safe fallback response returns
OpenTelemetry records span attributes: validation result, latency, detector scores
Traces export to your observability backend (Jaeger, Tempo, Datadog, etc.)

This design keeps detection logic decoupled from business logic, enabling independent updates to validators without redeploying the core application.

Setting Up the Development Environment

Use Python 3.10+ for optimal compatibility. Create a virtual environment and install the core packages:

guardrails-ai>=0.5.0
langchain>=0.1.0
opentelemetry-api, opentelemetry-sdk, opentelemetry-exporter-otlp
langchain-openai or your preferred LLM provider integration

Pin versions in requirements.txt to avoid breaking changes. Guardrails AI 0.5 introduced a new validator registry — verify your validator syntax matches the installed version.

Implementing Guardrails AI Validators for Prompt Injection

Guardrails offers two approaches: the built-in DetectPromptInjection validator and custom validators for domain-specific patterns. The built-in validator uses a lightweight classifier trained on known injection datasets.

Basic Validator Configuration

Define a Guardrails specification (XML or Pydantic) that wraps the user prompt:

<rail version="0.1">
  <output>
    <string name="user_prompt" description="Raw user input" />
  </output>
  <validators>
    <validator name="detect-prompt-injection" on-fail="exception" />
  </validators>
</rail>

Instantiate the Guard object in your application startup:

from guardrails import Guard
guard = Guard.from_rail("prompt_injection_guard.rail")

Custom Validator for Enterprise Patterns

For organization-specific threats (e.g., internal codenames, proprietary formats), extend Validator:

Implement validate(self, value, metadata) returning ValidationResult
Use regex, keyword lists, or a small fine-tuned classifier
Register via guardrails.validator_registry.register

Custom validators add <5ms latency when using compiled regex or ONNX models.

Integrating with LangChain Callbacks

LangChain’s BaseCallbackHandler provides synchronous and asynchronous hooks. Subclass it to intercept prompts before they reach the LLM.

Callback Handler Implementation

from langchain.callbacks.base import BaseCallbackHandler
class PromptInjectionCallback(BaseCallbackHandler):
    def __init__(self, guard):
        self.guard = guard
    def on_llm_start(self, serialized, prompts, **kwargs):
        for prompt in prompts:
            validation = self.guard.validate(prompt)
            if not validation.validation_passed:
                raise ValueError(f"Prompt injection detected: {validation.error}")

Attach the handler globally or per-chain:

from langchain.llms import OpenAI
llm = OpenAI(callbacks=[PromptInjectionCallback(guard)])

Handling Async Workflows

For async agents (e.g., LangGraph), implement on_llm_start as async def and use await guard.validate_async(prompt). Ensure your validator supports async execution to avoid blocking the event loop.

Adding Observability with OpenTelemetry

Instrumentation turns detection events into actionable telemetry. Initialize the tracer provider early in your application bootstrap.

Tracer Setup

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)

Enriching Spans with Detection Metadata

Wrap the validation call in a span and record attributes:

with tracer.start_as_current_span("prompt_injection_check") as span:
    span.set_attribute("prompt.length", len(prompt))
    result = guard.validate(prompt)
    span.set_attribute("detection.passed", result.validation_passed)
    span.set_attribute("detection.detector", "DetectPromptInjection")
    if not result.validation_passed:
        span.set_attribute("detection.reason", str(result.error))
        span.set_status(trace.Status(trace.StatusCode.ERROR, "Injection blocked"))

Export traces to your backend. Correlate with LLM latency spans to measure detection overhead — typically 10-30ms per request.

Testing and Deployment Considerations

Validate the pipeline before production rollout.

Automated Test Suite

Unit tests for each validator with known benign and malicious fixtures
Integration tests exercising the full callback chain
Latency benchmarks under concurrent load (target <50ms p99)
False positive/negative rates measured against a labeled dataset

Production Hardening

Deploy Guardrails validators as a sidecar service for horizontal scaling
Cache validation results for repeated prompts (TTL 5-10 minutes)
Configure alerting on detection rate spikes (>5% of traffic)
Log blocked prompts to a secure SIEM for forensic analysis

Monitor the detection.passed attribute in your observability platform. A sudden drop may indicate a novel attack variant or validator drift.

Conclusion

Building a real-time prompt injection detection system requires tight integration between validation logic, application callbacks, and observability. Guardrails AI supplies the detection engine, LangChain callbacks provide the interception points, and OpenTelemetry delivers the visibility needed to operate securely at scale. Start with the built-in validator, instrument every validation event, and iterate on custom detectors as you encounter domain-specific threats.

Actionable tip: Deploy the detection pipeline in shadow mode first — log validation results without blocking — to calibrate false positive rates before enforcing in production.