Using BERT instead of word embeddings

A recent development in the embeddings world is BERT, also known as Bidirectional Encoder Representations from Transformers, which, like word embeddings, gives a vector representation, but it takes context into account and can represent a whole sentence. We can use the Hugging Face sentence_transformers package to represent sentences as vectors.

Getting ready

For this recipe, we need to install PyTorch with Torchvision, and then the transformers and sentence transformers from Hugging Face. Follow these installation steps in an Anaconda prompt. For Windows, use the following code:

conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
pip install transformers
pip install -U sentence-transformers

For macOS, use the following code:

conda install pytorch torchvision torchaudio -c pytorch
pip install transformers
pip install -U sentence-transformers

How to do it…

The Hugging Face code makes using BERT very easy. The...