AI & automationJune 20266 min read
Enterprise RAG: where to start
RAG fails to deliver on its promise when you rush to the model. The real work is upstream, on your data.
RAG, for Retrieval-Augmented Generation: you retrieve the right passages from your documents, then ask the model to answer based on them. On paper, it is simple. In practice, the quality of the answer depends 90% on what you retrieve, not on the model that writes it.
Clean before you index
Poorly scanned PDFs, duplicates, outdated versions: a dirty corpus produces wrong answers, and states them with confidence. Before anything else, we sort, deduplicate and date. It is thankless, and it is decisive.
Chunk in the right place
A document split any which way loses its meaning. We chunk by logical unit, section, paragraph or procedure, not every 500 characters. Good chunking is the difference between a useful passage and a mangled one.
An LLM does not fix bad retrieval. It dresses it up and makes it credible.
Once these foundations are in place, plugging in the model becomes the easy part. We then evaluate on real business questions, and iterate on the indexing, not on the prompt.