Somewhere in your retrieval system there is a number — 0.7, 0.75, 0.8 — that decides which
evidence reaches the model and which is thrown away. Ask where it came from. In most systems we
review, the honest answer is that somebody copied it from a tutorial, a vendor’s default, or a previous
project on a different corpus with a different embedding model. A cosine score of 0.75 has no
universal meaning. A threshold that was not derived from your own data is not a relevance
floor. It is decoration.
Why the number does not travel
Cosine similarity compares the angle between two vectors. Mathematically it runs from −1 to 1,
although real embedding models often occupy a much narrower band than that. Zero means orthogonal in the
vector space — not “unrelated” in any way a business user would recognise.
More to the point, the distribution of those scores shifts with the embedding model, the
domain and the chunk length. Change any one of the three and yesterday’s sensible floor becomes today’s
silent filter. This is why a threshold carried across a model upgrade is one of the most common causes
of a retrieval system that “suddenly got worse” without anyone deploying a change to it.
The vector search chapter of our 2026 guide puts the rule simply: build
positive and negative examples from your labelled set, plot the two distributions, and choose the
threshold where they separate — at the balance you want between abstention and recall.
The two failure modes, and why only one of them is visible
Set the threshold too high and good passages are discarded before the model ever sees them. The system
answers “I don’t have information on that” while the answer sits in your document store. This failure is
quiet. Nobody files a ticket saying the AI was insufficiently helpful about something they assumed it
did not know.
Set it too low and near-misses reach the context window. The model, being a language model, writes
fluently around them. This failure is loud but misattributed: it looks like hallucination, so teams go
hunting for a better model when the actual defect is upstream in retrieval.
Both are threshold problems. Only one of them gets investigated.
The twenty-minute version
You do not need an evaluation platform to fix this. You need a set of real questions with known good
answers — fifty to a hundred is enough to start.
- Run each question through retrieval and record the similarity score of every candidate returned.
- Label each candidate as relevant or not relevant. Tedious, but it is an afternoon, once.
- Plot the two score distributions. You are looking for where they separate, and how much they
overlap. - Put the floor where the separation is cleanest, then move it deliberately in whichever direction your
use case prefers. A system that must not miss anything wants recall. A system that must not confabulate
wants abstention. - Keep the labelled set. Rerun it whenever the corpus, the chunking or the embedding model changes.
The same reasoning applies to the other numbers
Once you see the pattern, you find it elsewhere in the same pipeline. Hybrid retrieval systems combine
a semantic score and a lexical one, and the weight between them — the α in
α × semantic + (1 − α) × lexical — is exactly the same kind of decision. Our
chapter on hybrid retrieval and fusion gives starting values by traffic shape:
around 0.75 where users ask natural-language questions over prose, around 0.25 where they search part
numbers, case references and technical identifiers. Those are starting points to test, not settings to
adopt.
Reciprocal Rank Fusion sidesteps the calibration problem by using rank positions instead of raw
scores, which is why it is such a good default before you have enough labels. Even there, the constant
k = 60 that everyone inherits is a convention from information retrieval practice, not a law.
On short candidate lists a smaller value creates more separation.
The exercise most teams skip
Before tuning any of it, take a few hundred real searches and classify them: conceptual, exact
identifier, or mixed. Support and legal systems routinely contain far more exact lookup than the team
expects. That single exercise explains most cases where a dense-only system demos brilliantly and
frustrates in production — and it usually changes what you should be tuning in the first place.
One more trap worth naming: if your production queries carry access controls, measure retrieval
with the real filters applied. A graph index built over the whole corpus can behave very
differently when only a small permitted subset is eligible. Unfiltered benchmark quality tells you very
little about a system where every query arrives with an ACL attached.
The full treatment — embeddings, dimensions, index choice, thresholds and the metrics that tell you
which part is broken — is in Understanding AI in 2026.