Inside grounding: how retrieval turns 'I don't know' into a feature

The reason most chatbots guess is architectural: they have no mechanism to check whether a question is answerable. Grounding changes that by anchoring the model to a defined knowledge source.

What grounding actually does

Grounding routes every user question through a retrieval step before generation. The model is given a packet of relevant passages from an approved corpus and instructed to answer only from that packet. If retrieval returns nothing relevant, the model is told to say so.

The three signals that enable honest refusal

1. Retrieval confidence scores. When the top retrieved chunks score below a threshold, the system can trigger a refusal path instead of generation.
2. Coverage checks. The retriever can report which parts of the question it found evidence for. Partial coverage becomes a hand-off, not a guess.
3. Citation requirements. When the system must attach a source to every claim, uncited claims can be stripped automatically.

Where grounding breaks down

Poor chunking fragments context across passages, and the retriever misses them. See Inside document chunking. Keyword-only or embedding-only search also misses matches that hybrid retrieval would catch, covered in Inside hybrid search.

Designing the refusal path

A good refusal is specific: 'I don't have current pricing in my sources' beats 'I don't know.' Pair it with a hand-off button, a suggested rephrase, or a logged request so the knowledge base can be improved.

See the full playbook in How to set up RAG for enterprise knowledge bases.