In this chapter, you will need to install the following software if it has not already been installed:

• Eclipse Photon 4.8.0
• Java JDK 8 or later

We will be using the following APIs, which you will be instructed to add for each recipe as appropriate:

• OpenNLP 1.9.0
• Stanford NLP 3.8.0
• LingPipe 4.1.0

The code files for this chapter can be found at https://github.com/PacktPublishing/Natural-Language-Processing-with-Java-Cookbook/tree/master/Chapter02.

There are several approaches to performing SBD using the Java core JDK. While they will not always yield good results as do other more advanced SBD techniques, they may prove quite useful in some situations while not incurring the expense of specialized NLP models.

In this recipe, we will examine two approaches:

• The first approach will use the split method of the String class
• The second approach uses a simple regular expression

Create a new Java project. Since we will not be using any specialized libraries, the project does not need to be a Maven project.

There are several Java core techniques that can be used to perform sentence boundary detection. In this recipe, we will use the BreakIterator class. This class supports the identification of more than just sentences. It can also be used to isolate lines and words.

Create a new Java project. This does not need to be a Maven project since we will not be using a specialized NLP library.

The necessary steps include the following:

1. Add the following import statement to the project:

import java.text...

OpenNLP is a popular NLP library that supports the SBD process among other NLP tasks. As we will see, it is easy to use. We will use the SentenceDetectorME class to demonstrate this process. This is a maximum entropy model that is based on a statistical classification approach.

To prepare, we need to do the following:

1. Create a new Maven project.
2. Add the following POM dependency to your project:

<dependency>
    <groupId>org.apache.opennlp</groupId>
    <artifactId>opennlp-tools</artifactId>
    <version>1.9.0</version>
</dependency>

3. Download the en-sent.bin file from http://opennlp.sourceforge.net/models-1.5/. Save the file in your project...

The Stanford NLP API possesses several techniques for detecting SBD. We will use the WordToSentenceProcessor class to illustrate how this can be performed. This provides an alternate approach to using OpenNLP.

To prepare, we need to do the following:

1. Create a new Maven project
2. Add the following dependency to the project's POM file:

<dependency>
    <groupId>edu.stanford.nlp</groupId>
    <artifactId>stanford-corenlp</artifactId>
    <version>3.8.0</version>
</dependency>

Chunking is a technique that breaks up text into units referred to as chunks. The LingPipe libraries contain classes that support SBD. In this recipe, we will demonstrate how to perform SBD using chunking.

To prepare, we need to do the following:

1. Create a new Maven project
2. Add the following dependency to the project's POM file:

<dependency>
    <groupId>de.julielab</groupId>
    <artifactId>aliasi-lingpipe</artifactId>
    <version>4.1.0</version>
</dependency>

Unique types of text, such as medical or unusual languages, pose challenges when performing SBD. The frequent heavy use of specialized words and numeric values will not always yield good result with a model trained on a normal text. As a result, there are numerous models that have been trained on specialized datasets. In this recipe, we will demonstrate the use of a LingPipe model that has been trained to handle medical text.

The model will be demonstrated against an actual paragraph from a medical research article found in the Journal of Biomedical Science in 2018, Association between heavy metal levels and acute ischemic stroke, by Ching-Huang Lin, Yi-Ting Hsu, Cheng-Chung Yen, Hsin-Hung Chen, Ching-Jiunn Tseng, Yuk-Keung Lo, and Julie Y. H. Chan at https://jbiomedsci.biomedcentral.com/articles/10.1186/s12929-018-0446-0.

Specifically, we will...

When there are no specialized models for SBD, it becomes necessary to train a new model. In this recipe, we will illustrate how this can be performed using the OpenNLP API. We will create a set of training data and use it to train a neural network model. The model will then be tested using the OpenNLP technique illustrated in the Using OpenNLP to perform SBD recipe.

To prepare, we need to do the following:

1. Create new Maven project
2. Add the following dependency to the POM file:

<dependency>
    <groupId>org.apache.opennlp</groupId>
    <artifactId>opennlp-tools</artifactId>
    <version>1.9.0</version>
</dependency...