Using NLP, from project setup to report building

In this section, you will implement a practical example that performs sentiment analysis of some text extracted from Twitter. This example compares a pretrained model and a custom model, showing the results in Comet.

The full code of the example described in this section is available at the following link: https://github.com/PacktPublishing/Comet-for-Data-Science/tree/main/09.

We will focus on the following aspects:

• Configuring the environment
• Loading the dataset
• Implementing a pretrained pipeline
• Logging results in Comet
• Using a custom pipeline
• Building the final report

Let’s start from the first point, configuring the environment.

Configuring the environment

Environment configuration involves the following two steps:

1. We initialize Spark NLP. We import and start the Spark NLP library as follows:

   import sparknlp
   spark = sparknlp.start()

Note

If you are writing you...