Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Hands-On Python Natural Language Processing

You're reading from   Hands-On Python Natural Language Processing Explore tools and techniques to analyze and process text with a view to building real-world NLP applications

Arrow left icon
Product type Paperback
Published in Jun 2020
Publisher Packt
ISBN-13 9781838989590
Length 316 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Authors (2):
Arrow left icon
Mayank Rasu Mayank Rasu
Author Profile Icon Mayank Rasu
Mayank Rasu
Aman Kedia Aman Kedia
Author Profile Icon Aman Kedia
Aman Kedia
Arrow right icon
View More author details
Toc

Table of Contents (16) Chapters Close

Preface 1. Section 1: Introduction
2. Understanding the Basics of NLP FREE CHAPTER 3. NLP Using Python 4. Section 2: Natural Language Representation and Mathematics
5. Building Your NLP Vocabulary 6. Transforming Text into Data Structures 7. Word Embeddings and Distance Measurements for Text 8. Exploring Sentence-, Document-, and Character-Level Embeddings 9. Section 3: NLP and Learning
10. Identifying Patterns in Text Using Machine Learning 11. From Human Neurons to Artificial Neurons for Understanding Text 12. Applying Convolutions to Text 13. Capturing Temporal Relationships in Text 14. State of the Art in NLP 15. Other Books You May Enjoy

Distance/similarity calculation between document vectors

We have seen two methods of building vectors to represent text documents. The next question that comes up is:

How can you measure how similar or dissimilar text documents are and how can the vectors built so far be leveraged to have a solution to this problem?

If the words being used in two documents are similar, it indicates that the documents are similar as well. In this section, we will look into cosine similarity and use it to find how similar documents are based on the term vectors.

Cosine similarity

Cosine similarity provides insights into the angle between two vectors. Two vectors would be similar if they are pretty close in terms of both direction and magnitude. We will use techniques developed in the previous sections to build these vectors, and then figure out how close or far they are from each other using cosine similarity.

Cosine similarity helps in measuring the cosine of the angles between two vectors. The value...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime