Everyone is arguing about which AI model is best. What actually decided whether ours worked was something most people cannot define: the embedding. In plain terms, an embedding turns text into numbers that capture meaning, so software can tell which documents are genuinely related. It sounds like plumbing. It is the whole job.
The layer that decides what the model is allowed to see
In any AI that answers from your own documents — which is the entire category of useful business AI — the embedding decides what the model is allowed to read before it writes a single word. Get it wrong and your brilliant, expensive model answers with total confidence from the wrong material. Nobody notices until a customer does.
That is the quiet danger of retrieval-based AI. The failure is invisible. The output is fluent, well structured and wrong, because the right document never made it in front of the model.
The part that humbled me
Choosing the language model, the decision everyone obsesses over, took about a day. Run the benchmark, read the numbers, done.
Getting the embeddings right took three weeks. Three weeks of promising ideas that scored worse than where we started. Of the most-recommended option dragging us backwards. Of dead ends that looked like progress until we measured them properly. The flashy layer was the easy one. The unglamorous layer underneath, the one nobody posts about, was the difference between a demo that charms in a meeting and a system that holds up in front of a customer.
A leaderboard names the candidates. Selection happens on the pitch.
A public benchmark tells you which embedding models are worth trying. Selection happens with your own documents, every single time. The model that tops a generic leaderboard can be the one that drags your specific corpus backwards, and you only find that out by measuring on your data.
We build this self-hosted, on the client’s own hardware, which means there is no vendor to blame when retrieval goes wrong. You measure, or you ship something that sounds confident and gets it wrong. It is the same discipline behind Cortex, where every answer has to trace back to a real source document, and behind the stack we run on a single GPU.
So while everyone debates which model is best, the real question is simpler, and almost nobody asks it: what is your AI retrieving before it answers?