experiment · evergreen

Route by query shape

Symbol-shaped queries go to BM25, sentences to dense; always-hybrid measured worse than either.

Two retrievers, one regex. A query that looks like a symbol goes to BM25; a sentence goes to the dense index. Hybrid fusion and reranking stay behind flags — both were measured and both lost.

SettingRecall@8MRRTR prose R@8p50
BM25 only0.4050.2400.2632 ms
hybrid (dense + BM25 → RRF)0.7860.6040.68440 ms
dense only0.7860.6780.68432 ms
auto: symbol → BM25, prose → dense ✓0.7860.6900.68434 ms

Why always-hybrid loses

RRF adds up ranks. When one channel has nothing useful — dense on QUEUE_NAMES, BM25 on "how does auth work" — its best guess is still promoted at full strength and pollutes the top of the list. Routing removes the channel that would have guessed.

What counts as a symbol

One token with an internal boundary: camelCase, snake_case, kebab-case, a.b.c, a/b, A::B, or ALLCAPS. Deliberately narrow — a false positive sends a real question to BM25, which is measurably bad; a false negative only forgoes a small win. Symbols score 1.0 / 1.0 on BM25 and 1.0 / 0.875 on dense.

An LLM router was on the table. A regex got the same MRR gain for free and adds no latency.

#routing #bm25 #dense #rrf

See this note on the whiteboard →