What does a search score mean?

A search score is a number that puts results in order for one query: a higher score ranks first, and that is all it promises. It is not a grade, a percentage or the probability that a result is right.

Three words are often used as one. A rank is a place in the list: first, second, third. Relevance is whether a result answers what you need, and only a reader can judge it. A score sits between the two. It is the engine’s evidence for the rank, on a scale that belongs to one search.

Most scores are a sum of parts, each measured on its own scale. Our paper on search scores names five: search terms, text similarity, graph links, recency and boosts. Other engines use a similar set under other names.

Search terms. Keyword engines score the words a record shares with the query, commonly with BM25. A rare term counts for more than a common one, a repeated term adds less each time, and a long record is damped. Elasticsearch uses BM25 by default, with two settings: k1 at 1.2 controls how fast repeats stop counting, and b at 0.75 how much length matters. A BM25 score has no fixed top. Our article on hybrid search walks through the formula.

Text similarity. Meaning-based search compares vectors, the lists of numbers a language model produces for a text. Cosine similarity runs from minus 1 to 1. Elasticsearch turns it into a score as (1 + cosine) / 2, so a cosine of 0, no measurable likeness at all, still scores 0.5.

Graph links, boosts and recency. A graph can add a signal for the links two records share, such as the same part or supplier. A boost makes chosen fields or sources count more. Curiosity’s ranking tuning guide notes that boosts multiply: a title boost of 2.0 makes a title match count twice as much as a body match at 1.0. Recency lowers the score of older records. In Curiosity Studio the last 14 days keep their full score by default.

A percentage is a share of a known whole. A score has no whole, and no value means a perfect match. The Apache Lucene project’s wiki puts it plainly: “score values are meaningful only for purposes of comparison between other documents for the exact same query and the exact same index.”

The same page shows why dividing by the best score does not help. Three documents score 2.345, 16.415 and 2.345. As shares of the best, they read 14%, 100% and 14%. Delete the best one, and the other two jump to 100%. Neither of them changed, and neither became more relevant.

Nor is a score a probability. Robertson and Zaragoza’s 2009 review of BM25 is explicit about it. It derives the formula on the basis that “the only property we wish to preserve is the rank order of documents.” Parts that would not change the order are dropped along the way. The review notes that a good estimate of the probability of relevance would need them.

A score depends on the query and on everything else in the index. BM25 weighs a term by how rare it is in the collection, so adding records changes the score of records that did not change. Microsoft’s Azure AI Search documentation lists this among the reasons scores shift from one query to the next. It describes scores as “reflecting the strength of match relative to other documents in the same result set.”

So a 14.2 for a query with a part number and a 3.1 for a vague question say nothing about each other. Rare terms score high and common words score low, whatever the results are worth. Keyword scores and vector scores also sit on different scales. That is why hybrid search has to merge its two lists by rank, or after rescaling the scores.

• A fixed cutoff on a keyword score, such as hiding everything under 5. It drops good results for queries in common words and keeps weak ones for rare words.
• A bar or a percentage read as a grade. The top bar is full in every search, including one where nothing matched well.
• Two searches, or two versions of an index, compared by their top scores instead of by judged results.
• An assistant that treats the first result as the answer because its score is high. A high score says the record is the best match found, not that it answers the question.

Three questions cover most decisions about a ranked list:

1. Am I comparing results from one search only? Order and gaps inside one list carry meaning. Numbers across lists do not.
2. Where does the list drop? A large gap between two neighbors often marks the end of the strong matches. A flat list means nothing stood out.
3. Have I read the top results? Whether a result fits is in its content, not in its number.

When a system needs a threshold, set it on a bounded scale and measure it. The tuning guide sets a vector similarity cutoff at the lowest value that still keeps the right results on queries people have judged. Our article on hybrid search describes how to judge a ranking.

In Curiosity Studio, keyword search, vector similarity and graph links run over the same graph. A result’s score combines the parts its search uses. Which parts count depends on the search: a plain keyword search mainly uses search terms and recency, and a similarity view can add meaning and links.

A similarity view draws a bar beside each result, as long as its score compared with the best one in the list. So the top bar is always full, and the bars show the order and the gaps, not a grade. Developers can see why a result scored where it did: see search for developers.

The boosts, the recency window and the vector cutoffs are settings you tune against your own judged queries. Our paper What a search score means explains all of this in ten pages, for anyone who reads ranked results.