The cancelled-flight-versus-involuntary-travel-disruption bit is the whole damn game. Keyword search reads those as two unrelated things, and the employee gets a smug 'I couldn't find anything about that' while the answer sits in paragraph six. The expired policy is the nastier one though — that document retrieves beautifully. Perfectly relevant, confidently wrong, and nobody notices until someone files the claim.
This hit close to home. I had a RAG assistant keep citing an old expense limit because the updated policy chunk scored a hair lower than the stale one. The fix wasn't a fancier embedding model — it was filtering by recency before the vector search even ran.
The cancelled-flight-versus-involuntary-travel-disruption bit is the whole damn game. Keyword search reads those as two unrelated things, and the employee gets a smug 'I couldn't find anything about that' while the answer sits in paragraph six. The expired policy is the nastier one though — that document retrieves beautifully. Perfectly relevant, confidently wrong, and nobody notices until someone files the claim.
This hit close to home. I had a RAG assistant keep citing an old expense limit because the updated policy chunk scored a hair lower than the stale one. The fix wasn't a fancier embedding model — it was filtering by recency before the vector search even ran.