NER with character and token embeddings

Nowadays, recurrent models used to solve the NER task are much more sophisticated than having just a single embedding layer and an RNN model. They involve using more advanced recurrent models like Long Short-Term Memory (LSTM), Gated Recurrent Units (GRUs), etc. We will set aside the discussion about these advanced models for several upcoming chapters. Here we will focus our discussion on a technique that provides the model embeddings at multiple scales, enabling it to understand language better. That is, instead of relying only on token embeddings, also use character embeddings. Then a token embedding is generated with the character embeddings by shifting a convolutional window over the characters in the token. Don’t worry if you don’t understand the details yet. The following sections will go into specific details of the solution. This exercise is available in ch06_rnns_for_named_entity_recognition.ipynb in the Ch06-Recurrent...