Exploring the Spark NLP package

Spark NLP is an open source library for NLP released by John Snow Labs. It supports different programming languages, including Python, Java, and Scala. Spark NLP is widely used in production, since it is natively integrated with Apache Spark, a multi-language engine for large-scale analytics.

Spark NLP provides more than 50 features, including tokenization, NER, and sentiment analysis.

In this section, you will investigate the following aspects:

• Introducing the Spark NLP package
• Integrating Spark NLP with Comet

Let’s start from the first point, introducing the Spark NLP package.

Introducing the Spark NLP package

Spark NLP is an open source library built on top of Apache Spark and Spark ML (a machine learning library implemented on top of Apache Spark). The Spark NLP library provides almost all the NLP tasks, including tokenization, stemming, lemmatization, PoS tagging, sentiment analysis, spellchecking, and NER...