The code of this chapter can be found at https://github.com/PacktPublishing/Mastering-spaCy-Second-Edition.

NLP is a subfield of AI that analyzes text, speech, and other forms of human-generated language data. Human language is complicated – even a short paragraph contains references to the previous words, pointers to real-world objects, cultural references, and the writer’s or speaker’s personal experiences. Figure 1.1 shows such an example sentence, which includes a reference to a relative date (recently), phrases that can be resolved only by another person who knows the speaker (regarding the city that the speaker’s parents live in), and who has general knowledge about the world (a city is a place where human beings live together):

Figure 1.1 – An example of human language, containing many cognitive and cultural aspects

How do we process such a complicated structure using computers? With spaCy, we can easily model natural language with statistical models, and process linguistic features to turn the text...

Let’s get started by installing and setting up spaCy. spaCy is compatible with 64-bit CPython 3.7+ and can run on Unix/Linux, macOS/OS X, and Windows. CPython is a reference implementation of Python in C. If you already have Python running on your system, most probably your CPython modules are fine too – hence, you don’t need to worry about this detail. The newest spaCy releases are always downloadable via pip (https://pypi.org/) and conda (https://conda.io/en/latest/). pip and conda are two of the most popular distribution packages.

It’s always a good idea to create a virtual environment to isolate the independent set of Python packages for each project. On Windows, we can create a virtual environment and install spacy with pip using these commands:

python -m venv .env
.env\Scripts\activate
pip install -U pip setuptools wheel
pip install -U spacy

If your machine has a GPU available, you can install spaCy with GPU support with this...

The spaCy installation doesn’t come with the statistical language models needed for the spaCy pipeline tasks. spaCy language models contain knowledge about a specific language collected from a set of resources. Language models let us perform a variety of NLP tasks, including parts of speech tagging popularly called as POS tagging and named entity recognition (NER).

Different languages have different models that are language-specific. There are also different models available for the same language. The naming convention of the models is [lang]_[name]. The [name] part usually contains information about the model capabilities, the genre, and the size. For example, the pt_core_web_sm model is a small Portuguese pipeline trained on web text. Large models can require a lot of disk space, for example, en_core_web_lg takes up 382 MB, while en_core_web_md needs 31 and en_core_web_sm takes only 12 MB.

It is a good practice to match the model...

Visualization is the easiest way to explain some concepts to your colleagues, your boss, and any technical or non-technical audience. Visualization of language data is specifically useful and allows you to identify patterns in your data at a glance.

There are many Python libraries and plugins such as matplotlib, seaborn, tensorboard, and so on. spaCy also comes with its own visualizer – displaCy. In this subsection, you’ll learn how to spin up a displaCy server on your machine, in a Jupyter notebook, and in a web application. We’ll start by exploring the easiest way – using displaCy’s interactive demo.

Getting started with displaCy

Go ahead and navigate to https://demos.explosion.ai/displacy to use the interactive demo. Enter your text in the Text to parse box and then click the search icon on the right to generate the visualization. The result might look like Figure 1.3.

Figure 1.3 ...

This chapter gave you an introduction to NLP with Python and spaCy. You now have a brief idea about why to use Python for language processing and the reasons to prefer spaCy for creating your NLP applications. We also got started on our spaCy journey by installing spaCy and downloading language models. This chapter also introduced us to the visualization tool, displaCy.

In the next chapter, we will continue our exciting spaCy journey with spaCy core operations such as tokenization and lemmatization. It’ll be our first encounter with spaCy features in detail. See you there!