Semantic similarity experiment with FLAIR
In this experiment, we will qualitatively evaluate the sentence representation models thanks to the flair library, which really simplifies obtaining the document embeddings for us.
We will perform experiments while taking on the following approaches:
- Document average pool embeddings
- RNN-based embeddings
- BERT embeddings
- SBERT embeddings
We need to install these libraries before we can start the experiments:
!pip install sentence-transformers !pip install dataset !pip install flair
For qualitative evaluation, we define a list of similar sentence pairs and a list of dissimilar sentence pairs (five pairs for each). What we expect from the embeddings models is that they should measure a high score and a low score, respectively.
The sentence pairs are extracted from the SBS Benchmark dataset, which we are already familiar with from the sentence-pair regression part of Chapter 6, Fine-Tuning Language Models...
