Different search paradigms – sparse, dense, and hybrid
There are different types of vectors, and this difference is important to this discussion because you need to use different types of vector searches depending on the type of vector you are searching. Let’s talk in depth about the differences between these types of vectors.
Dense search
Dense search (semantic search) uses vector embedding representation of data to perform search. As we have talked about previously, this type of search allows you to capture and return semantically similar objects. It relies on the meaning of the data in order to perform that query. This sounds great in theory, but there are some limitations. If the model we are using was trained on a completely different domain, the accuracy of our queries would be poor. It is very dependent on the data it was trained on.
Searching for data that is a reference to something (such as serial numbers, codes, IDs, and even people’s names...