Introduction to token classification
The task of classifying each token in a token sequence is called token classification. This task says that a specific model must be able to classify each token into a class. POS and NER are two of the most well-known tasks in this criterion. However, QA is also another major NLP task that fits in this category. We will discuss the basics of these three tasks in the following sections.
Understanding NER
One of the well-known tasks in the category of token classification is NER – the recognition of each token as an entity or not and identifying the type of each detected entity. For example, a text can contain multiple entities at the same time – person names, locations, organizations, and other types of entities. The following text is a clear example of NER:
George Washington is one the presidents of the United States of America.
George Washington is a person name while the United States of America is a location name. A sequence...
