note · growing
Where naive RAG breaks
Low precision, low recall, stale data, one shot. All of it looks like a model bug from outside.
When naive RAG breaks it looks like a model bug from the outside. It is almost always a retrieval bug.
Low precisionnot every retrieved chunk is relevant
Low recallnot every relevant chunk is retrieved
Stale datathe index drifted from the source
One shotno chance to correct a bad retrieval
Two different diseases
- Low precision fills the context with noise. The model may see the right chunk and still lose it in the middle — lost in the middle. The answer comes out close but wrong.
- Low recall means the model never sees the material to synthesise from. The answer is either incomplete or invented.
More retrieved tokens does not mean better performance. Raising top-k usually costs precision and makes the answer worse.