Sentence boundary detection
Sentence boundary detection is an important step in NLP and an essential problem to be solved before analyzing the text for further use in information extraction, word tokenization, part of speech tagging, and so on. A sentence is a basic unit of text. Tough SBD has been solved to a good extent, extracting sentences from a text document is not a simple process. Sentence boundary detection is language dependent since the sentence termination character in each language may be different. This can be done using a machine learning approach by training a model rule-based approach. If we consider the English language then the simple set of rules which can give us a fairly accurate results are:
Text is terminated by a period ( . )
Text is terminated by an exclamation mark ( ! )
Text is terminated by a question mark ( ? )
Consider the following example:
NLP is a vast topic. Lots of research has been done in this field.
When we apply the preceding set of rules, we can extract...
