Learn how to measure and enforce groundedness in RAG systems using RAGAS, NLI, LLM judges, and production guardrails. Includes practical code, evaluation frameworks, and refusal logic.
Groundedness in RAG: The Trust Layer That Prevents Hallucinations
What Exactly Is Groundedness?
Let’s cut through the jargon.
Groundedness checks whether every claim in an LLM response is supported by the retrieved context provided to the model. If any claim adds information the context doesn’t contain, the response is ungrounded.
Think of it this way:
Grounded = The answer stays inside the lines of what the retrieved documents actually say.
Ungrounded = The answer colors outside those lines, adding stuff the documents never mentioned.
The whole idea behind RAG is to ground an LLM’s answers not in its own learned knowledge, but in the database you provide. If your system generates answers that aren’t grounded in that database, you’ve just built a very expensive hallucination engine.
Why groundedness matters ?
A RAG system can sound polished and still be wrong. Groundedness is the check that asks whether the answer is actually supported by the retrieved evidence, not just whether it sounds plausible. When we talk about groundedness in RAG, we mean one simple thing: the AI is actually using the information you gave it, rather than just freestyling from its training data.
A grounded system does something valuable for the business: it preserves friction. When the evidence is missing, it refuses, qualifies, or asks for clarification instead of inventing a confident answer.
The RAG Triad:
Groundedness is usually evaluated as one part of the RAG Triad.
Context Relevance: Did we fetch the right evidence?
Groundedness: Did the model stick to that evidence?
Answer Relevance: Did the final answer actually address the user’s intent?
If any one of these fails, the system becomes less trustworthy.
Groundedness workflow:
User Query
↓
Retriever
↓
Retrieved Context
↓
Generator
↓
Groundedness Checker
↓
Answer / Refusal
The core idea is simple: retrieval finds evidence, generation drafts a response, and a verifier decides whether the response is allowed to pass.
NLI verification code
NLI (Natural Language Inference) Verification in Groundedness is a technique used in Agentic RAG to mathematically verify that every claim generated by an LLM is directly supported by the retrieved context..An NLI-style verifier checks whether the answer is entailed by the retrieved context. By classifying each statement as entailment (supported), contradiction (conflicting), or neutral (unsupported), NLI ensures every answer stays strictly faithful to source context. This automated validation process eliminates AI hallucinations, guarantees data accuracy, and builds user trust in enterprise RAG systems.
from typing import List, Dict
class NLIVerifier:
def __init__(self, threshold: float = 0.8):
self.threshold = threshold
def verify(self, context: str, answer: str) -> Dict:
ctx = context.lower()
ans = answer.lower()
if “not” in ans and “not” not in ctx:
return {“verdict”: “contradiction”, “score”: 0.2}
if any(term in ctx for term in ans.split()[:8]):
return {“verdict”: “supported”, “score”: 0.9}
return {“verdict”: “uncertain”, “score”: 0.5}
This is a simplified stand-in for a real NLI model, but it shows the production shape: score, verdict, and a decision boundary.
LLM judge code
An LLM as a judge for groundedness acts like an automated fact-checker that verifies if an AI’s answer is 100% backed by your retrieved source documents. Instead of relying on slow human reviews, this evaluation method scans generated text line-by-line to catch AI hallucinations before users see them. In modern Agentic RAG pipelines, an LLM judge scores factual accuracy, context relevance, and response faithfulness in real time. Simply put, it guarantees your AI gives trustworthy, context-backed answers instead of making things up.
import os
import asyncio
from openai import AsyncOpenAI
class GroundednessJudge:
def __init__(self, model: str = “gpt-4o-mini”):
self.client = AsyncOpenAI(api_key=os.environ.get(“OPENAI_API_KEY”))
self.model = model
async def score(self, context: str, answer: str) -> float:
prompt = “””
You are a strict groundedness evaluator.
Score how well the answer is supported by the context.
Return only a float from 0.0 to 1.0:
– 1.0 = fully supported
– 0.5 = partially supported
– 0.0 = unsupported or contradictory
“””
response = await self.client.chat.completions.create(
model=self.model,
messages=[
{“role”: “system”, “content”: prompt},
{“role”: “user”, “content”: f”CONTEXT:\n{context}\n\nANSWER:\n{answer}”}
],
temperature=0.0
)
return float(response.choices[0].message.content.strip())
Judge calibration code
LLM judges should be calibrated against human labels before production use.
from typing import List, Tuple
def calibrate_judge(judge_scores: List[float], human_scores: List[float]) -> Dict:
if len(judge_scores) != len(human_scores) or not judge_scores:
return {“error”: “invalid calibration data”}
diffs = [abs(j – h) for j, h in zip(judge_scores, human_scores)]
avg_error = sum(diffs) / len(diffs)
return {
“avg_abs_error”: round(avg_error, 3),
“calibrated”: avg_error <= 0.15,
“recommendation”: “use in production” if avg_error <= 0.15 else “recalibrate or retrain judge”
}
A good practice is to test the judge on 50 to 100 examples first.
Refusal logic code
If evidence is weak, the system should refuse rather than guess.
def should_refuse(groundedness_score: float, answer_relevance: float, threshold: float = 0.8) -> bool:
return groundedness_score < threshold or answer_relevance < threshold
def grounded_response(context: str, answer: str, groundedness_score: float, answer_relevance: float) -> str:
if should_refuse(groundedness_score, answer_relevance):
return “Based on the provided documentation, the text does not support a reliable answer.”
return answer
That refusal behavior is often more valuable than a fluent guess.
Context compression code
Compression helps reduce noise and token bloat while preserving the most relevant evidence.
from typing import List
def compress_context(chunks: List[str], max_words: int = 80) -> List[str]:
compressed = []
for chunk in chunks:
words = chunk.split()
if len(words) <= max_words:
compressed.append(chunk)
else:
compressed.append(” “.join(words[:max_words]) + “…”)
return compressed
In production, this can be upgraded to summarization or extraction-based compression.
RAGAS integration
RAGAS is a practical framework for evaluating groundedness, faithfulness, answer relevance, and retrieval quality.
# Example structure only
# from ragas import evaluate
# from ragas.metrics import faithfulness, answer_relevancy, context_precision, context_recall
#
# result = evaluate(dataset, metrics=[
# faithfulness,
# answer_relevancy,
# context_precision,
# context_recall
# ])
# print(result)
RAGAS is useful because it gives teams a ready-made evaluation surface instead of forcing them to build every metric from scratch.
Production groundedness pipeline:
class GroundedRAGPipeline:
def __init__(self, verifier, judge):
self.verifier = verifier
self.judge = judge
async def answer(self, context: str, user_answer: str) -> str:
nli_result = self.verifier.verify(context, user_answer)
judge_score = await self.judge.score(context, user_answer)
if nli_result[“score”] < 0.8 or judge_score < 0.8:
return “Based on the provided documentation, the text does not support a reliable answer.”
return user_answer
This pattern is the practical enterprise version: verify first, release second.
High groundedness looks like this:
- High Groundedness: “According to Section 4.2 of the 2026 Manual, your policy covers water damage up to ₹50,000 [Source: PDF Page 12].”
- Low Groundedness: “Your policy generally covers water damage, usually up to standard limits.” (Vague, unverified, and lacks proof).
- High Groundedness: “The provided documents do not contain information about international shipping fees for electronics.”
- Low Groundedness: “International shipping fees for electronics typically range from ₹1,500 to ₹3,000.” (The AI hallucinated a plausible guess because the answer wasn’t in the data).
- High Groundedness: “Based on your uploaded inventory sheet, the company owns 14 laptops.”
- Low Groundedness: “Laptops are essential for modern business productivity.” (The AI drifted into generic training data instead of sticking to the specific query).
- High Groundedness: “Your Q3 revenue was ₹4.2 Lakhs, matching the balance sheet exactly.”
- Low Groundedness: “Your business showed strong financial growth in Q3.” (Using generic adjectives instead of precise, grounded numbers).
Troubleshooting guide
Problem: The model is fluent but wrong
Likely cause: weak grounding checks or weak retrieval.
Fix: tighten evidence selection, add refusal logic, and use judge verification.
Problem: The judge overestimates support
Likely cause: uncalibrated judge.
Fix: compare against human labels and recalibrate.
Problem: The system refuses too often
Likely cause: threshold too strict or retrieval too narrow.
Fix: improve recall, relax threshold slightly, or retrieve more context.
Problem: Answers are grounded but incomplete
Likely cause: context is insufficient.
Fix: improve retrieval depth or chunk coverage.
Problem: Token cost is too high
Likely cause: too much context passed into generation.
Fix: compress context and remove redundant chunks.
Quick reference card:
Groundedness = supported by evidence.
Context relevance = right evidence retrieved.
Answer relevance = answer matches the question.
Use NLI for contradiction and support checks.
Use an LLM judge for semantic scoring.
Calibrate judges before trusting them.
Refuse when evidence is weak.
Compress context to control cost.
This is especially important in enterprise RAG systems where policies, local rules, and region-specific exceptions can be mixed across English and code-switched content. Groundedness prevents the system from inventing local policy details when the source text does not fully support them. It is the trust layer of RAG. It is what turns a fluent assistant into a reliable one. If the system cannot support the answer with evidence, it should say so clearly. That discipline is not a weakness — it is the signal that the system knows the difference between retrieval and invention.