Designing Guardrails & Constraints

Guardrails are the mechanisms that keep an LLM product operating within the boundaries it was designed for. They are what stands between a capable, powerful model and a product that causes harm, embarrassment, or legal liability.

Guardrails are designed to align LLMs with desired behaviors and mitigate potential harm. They address risks including hallucination, bias, toxicity, privacy, and out-of-distribution behavior. Effective guardrail design requires a deep understanding of the intended use case, relevant regulations, and ethical considerations.

Guardrails are not a single mechanism. They operate at multiple layers of the system, and the most reliable products have them at every layer.

1. Input guardrails operate before the model sees the user's message. They detect and handle inputs that should not be processed: personally identifiable information that should be filtered, content that falls outside the product's intended scope, adversarial inputs designed to manipulate the model.

1. System instruction guardrails define the model's behavioral boundaries through the system prompt itself. A well-designed system instruction explicitly tells the model what it should never do, how to handle requests that fall outside its scope, and what to say when asked to behave in ways that conflict with the product's intent.

1. Output guardrails operate after the model generates a response but before the user sees it. They check for content policy violations, validate that structured outputs are correctly formatted, flag low-confidence responses for human review, and catch anything the input guardrails and system instruction missed.

1. Human oversight is the final guardrail and the most important one for high-stakes use cases. For any decision that materially affects a user's life, an AI output that has not been reviewed by a qualified human is not a guardrail. It is an assumption.