EvalLayer vs NeMo Guardrails: dialog rails vs authorization
NVIDIA NeMo Guardrails is an open-source toolkit for adding programmable rails to LLM conversations: what topics are in bounds, how dialogs should flow, what gets checked on the way in and out. EvalLayer is a hosted authorization layer for commercial agent actions: whether this quote, refund, or commitment is inside the business's policy. One shapes conversations, the other gates consequences, and a serious customer-facing agent frequently needs both. This comparison aims to be accurate about each.
What NeMo Guardrails is
NeMo Guardrails is an open-source toolkit from NVIDIA, Apache 2.0 licensed, for adding programmable guardrails to LLM-based conversational systems. Its distinctive idea is Colang, a purpose-built modeling language with a Python-like feel for describing conversational flows: which user intents exist, how the bot should respond to each, and which paths a dialog may take. Around that core, the runtime supports five rail types that fire at different stages: input rails that can reject or alter what the user sent, dialog rails that enforce conversation paths, retrieval rails that screen RAG chunks, execution rails around tool calls, and output rails that can change or withhold the bot's reply. It integrates with common orchestration stacks and sits within NVIDIA's broader NeMo ecosystem.
What NeMo Guardrails is designed for
It is a toolkit for making conversational systems behave predictably. The natural fit:
- Topic control: a banking assistant that will not discuss investments, a support bot that stays on product.
- Programmed dialog flows: greeting patterns, escalation paths, refusal styles that are consistent instead of improvised.
- Layered screening: jailbreak and moderation checks on input, screening of retrieved context, checks on drafted output.
- Teams that want conversation policy as reviewable code in the repo, expressed in Colang.
What NeMo Guardrails does not try to do
NeMo Guardrails governs conversational behavior; it does not model commercial authority. There is no native vocabulary for discount caps, dollar spend authority, forbidden commitment types, or approved claims registers, no severity ladder that ends in a human approval queue, and no independent audit record, since the toolkit runs inside the application it protects. Execution rails can wrap tool calls, but deciding whether a specific refund is inside a specific agent's authority is logic you would write yourself. Keeping a conversation on the rails and deciding whether a commitment may ship are different jobs, a distinction covered in guardrails vs authorization.
What EvalLayer does
EvalLayer is an independent assurance API. POST /evaluate scores completed agent work. POST /authorize checks a proposed action against a commercial policy: max_discount_pct, commitment_authority_usd, forbidden_commitments, approved_claims, required_disclaimers, blocked_topics. Countable rules run deterministically, semantic rules get one AI pass, and the decision returns as allow, rewrite, require_approval, or block with violations listed and an approved rewrite when available. Every decision is stored as an audit record. Pricing starts at $0.01 per check.
Comparison table
| Dimension | NeMo Guardrails | EvalLayer |
|---|---|---|
| Primary job | Programmable conversational rails for LLM apps | Authorization of commercial agent actions |
| Checking layer | In-app runtime around the model: input, dialog, retrieval, execution, output rails | Independent hosted API at the action boundary |
| Policy model | Colang flows and rail configurations written by your team | Commercial policy object: caps, authority, commitments, claims, disclaimers, topics |
| Decisions returned | Allow, refuse, or alter within the conversation flow | allow, rewrite, require_approval, or block, plus violations and rewrite |
| Audit records | Application-level logs and tracing you operate | Every decision stored as an audit record with policy and violations |
| Pricing model | Open source, Apache 2.0; integration and inference costs are yours | Hosted, from $0.01 per check |
When to use which
Use NeMo Guardrails when
- The problem is conversational shape: off-topic drift, inconsistent handling, unsafe inputs.
- You want dialog policy as code your team writes and version-controls.
- The assistant informs and routes but does not commit the business to anything.
Use EvalLayer when
- The agent's outputs carry authority: prices, refunds, delivery promises, public claims.
- You need a decision trail an auditor or counterparty can inspect, independent of the app that produced the action.
- You want approval routing for over-authority actions without building the workflow.
When to use both
They compose in sequence, and the seam between them is clean because they never ask the same question. Rails run through the conversation: the input is screened, the dialog follows your flows, retrieved chunks are checked, the draft output is safe and on-topic. None of that inspects whether the number inside a compliant-sounding sentence is a number the business allows. Then the consequential moment arrives, the assistant is about to send a quote, and that action goes to authorization: 25% off against a 10% cap comes back blocked with a compliant rewrite. Every incident in the incident database involved an output that a well-behaved conversation could have produced; the failure was authority, not tone. Rails keep the conversation in bounds. Authorization keeps the business in bounds. The second half takes about 90 seconds to see in the live demo.