Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Natural Language Understanding with Python

You're reading from   Natural Language Understanding with Python Combine natural language technology, deep learning, and large language models to create human-like language comprehension in computer systems

Arrow left icon
Product type Paperback
Published in Jun 2023
Publisher Packt
ISBN-13 9781804613429
Length 326 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Author (1):
Arrow left icon
Deborah A. Dahl Deborah A. Dahl
Author Profile Icon Deborah A. Dahl
Deborah A. Dahl
Arrow right icon
View More author details
Toc

Table of Contents (21) Chapters Close

Preface 1. Part 1: Getting Started with Natural Language Understanding Technology
2. Chapter 1: Natural Language Understanding, Related Technologies, and Natural Language Applications FREE CHAPTER 3. Chapter 2: Identifying Practical Natural Language Understanding Problems 4. Part 2:Developing and Testing Natural Language Understanding Systems
5. Chapter 3: Approaches to Natural Language Understanding – Rule-Based Systems, Machine Learning, and Deep Learning 6. Chapter 4: Selecting Libraries and Tools for Natural Language Understanding 7. Chapter 5: Natural Language Data – Finding and Preparing Data 8. Chapter 6: Exploring and Visualizing Data 9. Chapter 7: Selecting Approaches and Representing Data 10. Chapter 8: Rule-Based Techniques 11. Chapter 9: Machine Learning Part 1 – Statistical Machine Learning 12. Chapter 10: Machine Learning Part 2 – Neural Networks and Deep Learning Techniques 13. Chapter 11: Machine Learning Part 3 – Transformers and Large Language Models 14. Chapter 12: Applying Unsupervised Learning Approaches 15. Chapter 13: How Well Does It Work? – Evaluation 16. Part 3: Systems in Action – Applying Natural Language Understanding at Scale
17. Chapter 14: What to Do If the System Isn’t Working 18. Chapter 15: Summary and Looking to the Future 19. Index 20. Other Books You May Enjoy

Evaluation metrics

There are two important concepts that we should keep in mind when selecting an evaluation metric for NLP systems or, more generally, any system that we want to evaluate:

  • Validity: The first is validity, which means that the metric corresponds to what we think of intuitively as the actual property we want to know about. For example, we wouldn’t want to pick the length of a text as a measurement for its positive or negative sentiment because the length of a text would not be a valid measure of its sentiment.
  • Reliability: The other important concept is reliability, which means that if we measure the same thing repeatedly, we always get the same result.

In the next sections, we will look at some of the most commonly used metrics in NLU that are considered to be both valid and reliable.

Accuracy and error rate

In Chapter 9, we defined accuracy as the number of correct system responses divided by the overall number of inputs. Similarly,...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at £16.99/month. Cancel anytime