Chapter 3. Relevancy and Scoring

As we are in the era of big data, it's very important that only the most relevant results are shown first. This means that even when there are millions of items that match our requirement, we only usually have the patience to go through the top limited number of items. For example, when we search for something in Google, it throws something like 234,324,000,000 results out of which, most of us would not even go to the second page. We usually constrain ourselves with the first page or the first 10 results. If the first 10 results are irrelevant, we assume that the next or remaining documents are more irrelevant than the current one and skip them too. This nature of Internet browsing has opened up huge scope for improvement in relevancy and scoring algorithms especially in the big data domain, as we have to deal with huge volumes of documents where mostly, only a few matter in their respective contexts.

The value scoring and relevancy give to...