Retrieval, reranking, citations, and no-answer
Design retrieval and answer contracts that expose support, uncertainty, and safe refusal behavior.
Retrieval contract
Define how a query is transformed, filtered, searched, fused, reranked, limited, and assembled into context. Record which index, embedding, reranker, policy, and configuration versions were used.
Tune retrieval for the task rather than maximizing generic similarity. Test exact identifiers, terminology, multi-part questions, recent changes, negative cases, ambiguous requests, and content near permission boundaries.
Citation contract
A citation should identify a source the user is permitted to access and support the associated claim. It should resolve to the exact version or snapshot used where practical. Test citation correctness, coverage, broken references, conflicting sources, and cases where the answer adds unsupported detail.
No-answer behavior
Define when the system must say it lacks sufficient permitted evidence, ask for clarification, return retrieval-only results, or escalate. Do not reward plausible completion when evidence is absent.
Context controls
Label retrieved material as data, isolate it from governing instructions, limit size and source count, preserve source boundaries, and handle conflicts deliberately. Reranking and context compression can remove qualifiers or change visibility, so they require their own evaluation.
The answer should expose uncertainty and support without revealing denied source existence or content.