Answer engine over 6.9M tax passages

01 — The problem

Tax and accounting questions need exact answers from a very large body of guidance and study material. Keyword search misses reworded questions, and a plain chat model can produce an answer that sounds right and isn't.

02 — What we built

We built the whole pipeline: crawling and uploads, chunking and indexing into Qdrant, a self-hosted embedding model and reranker on a GPU server, and keyword and vector search run in parallel and merged by rank. The best passages go to the chat model, which streams the answer back.

03 — What changed

It runs in production today. Warm searches took about 46–49 ms when measured on 30 July 2026. It has real limits too: the server is short of memory, and cold searches can take several seconds.

• 01Keyword and vector search run side by side and are merged by rank fusion, because tax questions depend on exact terms as well as meaning. Common abbreviations are expanded first, so “NOL” also finds “net operating loss”.
• 02A cross-encoder reranker rescores the top 20 passages before the model sees any of them. Search finds candidates; the reranker decides what is actually relevant.
• 03We found a silent bug: after a collection was renamed, document deletes did nothing and reported no error, so deleted content stayed searchable. We fixed it and removed 746,056 stale vectors, taking the index from 7.64M to 6.89M, then checked the result by search.
• 04On a memory-limited server we moved the search graph to disk. Swap use fell from about 11.7 GB to about 5.3 GB and warm search speed held, at the known cost of slower cold searches.
• 05Before deleting a wider set of about 916,000 suspect vectors, we tested the list and found it was mixed with live content. We left them in place: the space saved was not worth losing searchable knowledge.
• 06Full snapshots of the index go offsite every week, and the recovery copy was integrity-checked before the older local snapshot was retired.

Technology