Vector search is the easy part of a codebase RAG. The four things that are actually hard decided every part of this system.

exact symbolshandleAuthCallback

Turkish → Englishprose vs. code

stalenessrepo moved on

saying "no"kNN never says no

Problem	What we did	Measured
Symbols	route symbol-shaped queries to BM25	1.0 / 1.0 recall@8 / MRR
Turkish questions	multilingual embedder, optional LLM descriptions	0.04 → 0.684 → 1.0 (subset)
Staleness	sha256 manifest diff on push	38 changed files in 23 s
No answer	three score bands + negative golden cases	abstain 0.846, false alarm 0.048

Before the real system there was a throwaway one: MiniLM-L6-v2, a small Milvus, 30 questions. It was wrong about almost everything and that was the point.

recall@5	symbol query	plain sentence
dense (MiniLM)	0.60	0.62
BM25	0.80	0.36
RRF of both	0.60	0.52

What survived into the real build

• Symbols want BM25. An embedding has nothing to say about QUEUE_NAMES; a lexical index has everything.
• Fusing always is not free. RRF promoted the losing channel's best guess at full strength and came out below either channel alone.
• Turkish over English code scored 0.04. Not a retrieval bug — there was simply no Turkish text to match. Two fixes were carried forward: a multilingual embedder, and LLM-written descriptions as an option.
• Local models drift. qwen2.5 slid into Chinese mid-sentence under load and did not say so. Every generated description is script-checked since.

One process, one collection, one database file. Each piece was picked against the four hard problems, not against a feature list.

Piece	Why	Rejected
Milvus 2.6	BM25 is a built-in Function → no second index; partition key makes repo filters cheap	pgvector (no BM25), Qdrant (BM25 client-side), Elasticsearch (a second world)
BGE-M3, local, 1024d	multilingual: Turkish prose 0.684 where MiniLM had 0.04; code never leaves the machine	OpenAI / Voyage embeddings (code leaves; kept as a flag)
tree-sitter	chunk = code unit, citation = file:line — symbol	fixed windows, LLM chunking
sha256 manifest	rename, mode, submodule edge cases vanish; local dirs take the same path	git diff
webhook + poller	push-time freshness with a safety net	cron only, webhook only
SQLite + one worker	one file, no Redis, one pending job per repo	Postgres + Celery

A push should cost seconds, not a full re-index. Change detection is by content hash, not by git diff.

pushwebhook or poll

→

jobdeduped per repo

→

fetch + reset

→

sha256 diffvs manifest

→

delete + rewritechanged paths only

→

manifestupdated

Why a hash and not a diff

• Renames, mode changes, submodules, force-pushes: none of them are special cases when you compare file contents.
• A local directory that is not a git repo goes through the exact same path.
• The manifest is the single source of truth for read_code too — the agent can only read files that are actually indexed.

Crash safety by ordering

A file's manifest row is removed before its chunks are deleted and put back after the new chunks are written. A job killed halfway leaves the file looking "unindexed", so the next sync simply redoes it. Nothing is ever half there.

Two retrievers, one regex. A query that looks like a symbol goes to BM25; a sentence goes to the dense index. Hybrid fusion and reranking stay behind flags — both were measured and both lost.

Setting	Recall@8	MRR	TR prose R@8	p50
BM25 only	0.405	0.240	0.263	2 ms
hybrid (dense + BM25 → RRF)	0.786	0.604	0.684	40 ms
dense only	0.786	0.678	0.684	32 ms
auto: symbol → BM25, prose → dense ✓	0.786	0.690	0.684	34 ms

Why always-hybrid loses

RRF adds up ranks. When one channel has nothing useful — dense on QUEUE_NAMES, BM25 on "how does auth work" — its best guess is still promoted at full strength and pollutes the top of the list. Routing removes the channel that would have guessed.

What counts as a symbol

One token with an internal boundary: camelCase, snake_case, kebab-case, a.b.c, a/b, A::B, or ALLCAPS. Deliberately narrow — a false positive sends a real question to BM25, which is measurably bad; a false negative only forgoes a small win. Symbols score 1.0 / 1.0 on BM25 and 1.0 / 0.875 on dense.

The textbook says: retrieve 40, rerank with a cross-encoder, hand over 8. On this corpus the textbook was wrong, and the numbers said so.

Setting	Recall@8	MRR	TR prose	p50
auto + dense (no rerank)	0.786	0.690	0.684	34 ms
hybrid, k=40 candidates	0.952	—	0.895	39 ms
hybrid + bge-reranker-v2-m3	0.762	0.508	0.579	4389 ms
auto + bge-reranker-v2-m3	0.762	0.514	0.579	2050 ms

• The headroom is real. Recall@40 is 0.95 — the right chunk is almost always in the candidate pool. A good reranker has 17 points to win.
• This reranker spent them. MRR dropped from 0.690 to 0.514 and the Turkish slice fell furthest. It also cost two to four seconds per query.
• As a "no answer" gate it was worse. It caught 12/12 negatives — and flagged 12/42 real answers as junk. A 29% false alarm rate.

The users ask in Turkish; the code, comments and commit messages are English. Nothing in a bi-encoder bridges that gap unless the text does.

Step one: a multilingual embedder

BGE-M3 alone took Turkish prose from 0.04 to 0.684. English prose sits at 0.842 on the same corpus, so a gap of ~16 points remained.

Step two: write the missing text

Contextual retrieval, applied to code: one LLM call per file returns a 2–3 sentence Turkish description of each chunk. The description goes only into the embedded/BM25 text — the chunk shown to the model stays the real source. Cached by chunk hash + model, so a re-index never pays twice.

Same 46 files / 423 chunks	Recall@8	MRR	TR prose MRR	false weak-match
no enrichment	0.929	0.839	0.778	0.119
enriched (local qwen3.5:9b)	1.000	0.912	0.932	0.048

Negatives were unchanged (abstain 0.923 both ways) — the descriptions did not manufacture confidence on unrelated questions — and the score calibration did not move.

A chunk is a function, class, method, interface, type or exported constant — with boundaries from a real parser, not from counting braces.

parsetree-sitter

→

units≤ 2000 B

→

splitbig class → members

→

mergesmall ones ≥ 200 B

→

headeronly for the index

The four rules

• Big containers split into members. A 5 KB class becomes a header chunk plus one chunk per method, each carrying parent = class.
• Small things merge with a neighbour. One-line types, short consts, import blocks and doc comments stick to the next unit.
• Nothing exceeds ~2000 bytes (≈ 500 tokens). The model accepts 8192, but a long chunk's embedding averages toward nothing. A single oversized function is windowed by lines and keeps its symbol.
• Text stays clean. The "what am I, where do I live" header — file, symbol, parent, imports — is prepended only to the text that gets embedded and BM25-indexed. Citations show real source.

Identifiers for BM25

The standard analyser keeps handleAuthCallback as one token, so "auth callback" never touches it. Every chunk's identifiers are split into lowercase sub-words and appended to the indexed text.

What does syntax-aware chunking actually buy? We indexed the same repo twice — once with tree-sitter units, once with blank-line windows of the same size — and ran the same 42 questions.

Chunker	Recall@8	MRR	EN prose	TR prose	symbol R / MRR
tree-sitter units	0.786	0.690	0.842 / 0.744	0.684 / 0.570	1.0 / 1.0
blank-line windows	0.762	0.598	0.895 / 0.737	0.632 / 0.518	0.75 / 0.32

Reading it honestly

• Recall moved by one question. MRR moved by 0.09 — and almost all of it is symbol queries (0.32 → 1.0) plus the Turkish slice.
• On English prose the plain windows were equal or better.
• So AST chunking does not find more. It puts the right piece on top and knows its name. That is worth having; it is not a 20-point lever.

The literature agrees

The cAST paper reports +1 to +4 points over same-size line windows. An independent controlled replication found ≈ 0, and a third found the fixed windows slightly ahead. Naive function-per-chunk is reliably worse (−4 to −6 pp) — the value sits in merging small siblings and capping size, not in the syntactic boundary itself. Reranking and LLM-written context move retrieval 3–10× more.

The tempting design was a table: extension → language, folder → role (controllers/ = controller), a regex per language for symbols. We surveyed what production systems do instead, and measured what a table would buy.

What everyone else does

GitHub code search, Sourcegraph, Cursor, aider, Continue, Sweep, Qodo, Greptile — all of them outsource language coverage to a parser ecosystem (tree-sitter grammars with upstream tags.scm, or universal-ctags), chunk with a grammar-generic size cap, keep a universal line-window fallback, and carry path + symbol as metadata. None of them has framework-specific logic. None of them keeps a folder→role taxonomy.

Proposed	Verdict	Why
extension → language table	replaced by a rule	371 grammars ship in the pack; if the extension name is a grammar name (.lua .vue .razor .zig) it just works; a small alias table (.ts .cs .kt) is validated against the pack at import
regex symbol extractors	no	tree-sitter already yields symbol, kind, parent for 20+ languages
folder → role tags	no	a role is a deterministic function of the path, and the path is already in the embedded text; it fixed none of the 11 misses in the golden set
vendor-name skip lists (jquery, bootstrap…)	no	a per-repo ignore file generalises; a name list encodes one company
per-framework rules (Next.js, Vue, Razor)	no	route files already carry the route in their path; Razor blocks fall out of the grammar

What we kept hand-written, and why

• Deliberately plain: json, yaml, css, html — grammars exist, but windows measured equal and a parser is only crash surface.
• Never indexed: csv, tsv, diff, po, pem — grammars exist for those too.
• Grammar exists, parser refused: sql (the generated grammar segfaults on migration files) and cobol (hangs past 300 s on unbalanced brackets). Both were found by a smoke test that runs every grammar in its own subprocess on degenerate input — a crash kills the child, not the indexer.

Nearest-neighbour search has no concept of "nothing is close". Ask a codebase whether you have been to a five-star resort and it returns eight chunks (best score 0.366). Hallucination starts right there.

below 0.45dropped, counted

→

0.45 – 0.55returned with a weak-match note

→

0.55 and upnormal

Why not one threshold

Cosine does not separate the grey zone on this model and corpus: real answers have a top-1 dense median of 0.636 and a minimum of 0.526; irrelevant questions have a median of 0.524 and a maximum of 0.587. They overlap. A single cut either drops real answers or lets junk through.

Gate	abstain on 13 negatives	false alarm on 42 positives	extra latency
note below 0.55	0.833	0.048	0
floor 0.45 + note 0.55 ✓	0.846	0.048	0
reranker score below 0.05	1.000	0.286	+550 ms

The floor removed nothing from the golden set and turned the resort question into an empty result. The two negatives that still slip through are questions with genuinely similar code in the repo — there the call belongs to the reader.

The consumer is an agent — Claude Code, Cursor — so the retriever is exposed as three MCP tools mounted at /mcp inside the same FastAPI process.

search_codequery, repo, path prefix, category

→

read_codeindexed file, line range

→

list_reposfreshness

Design choices

• Same process, same retriever. No stdio sidecar loading the embedding model twice; blocking work (embedding, Milvus, disk) runs in threads so the MCP session's event loop keeps flowing.
• Output is data, not instructions. Code returned to the agent is wrapped as untrusted content.
• Signals travel with the hits. Weak-match band, DOC label, "N candidates dropped", last index time per repo.
• read_code reads only indexed files. The path must be in the manifest — so the agent cannot read node_modules or .env, and cannot be told "that file exists" when it does not. If the file on disk no longer matches the indexed hash, the slice comes back flagged stale.

Gotcha	Fix
mounted sub-app lifespans never run	session_manager.run() wraps the parent app's lifespan
transport rejects foreign Host headers with 421 (DNS rebinding)	extra hosts via config
/mcp → /mcp/ is a 307; redirect-averse clients lose the POST body	a pure-ASGI middleware rewrites the path instead

The rule of the repo: no retrieval flag changes its default without a JSON in evals/results/. "It seems better" is not a result.

golden42 positives + 13 negatives

→

runauto · k=8

→

scoreRecall@8 · MRR · abstain · false-weak

→

ledgerREADME table

The golden set is 19 English prose questions, 19 Turkish, 4 symbols, and 13 questions whose answer is not in the repo. Every run also prints p50/p95 and a calibration block. Repo-specific sets stay out of git — they contain internal paths.

What the numbers overturned

Belief	Measured	Decision
always fuse dense + BM25	MRR 0.604 vs 0.690 routed	route by query shape
a cross-encoder reranker helps	MRR 0.690 → 0.514, +2–4 s	flag, default off
a reranker can gate 'no answer'	29% false alarm	three cosine bands instead
folder → role tags make embeddings smarter	fixed 0 of 11 misses	not built
more languages = big win	AST vs windows: 1 question, MRR +0.09	generic rule, zero per-language code
Turkish needs a better matcher	text was missing, not matching: 0.04 → 0.684 → 0.932	multilingual embedder + optional enrichment

Things that cost an afternoon each and are now one line in a config or a test.