Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
R Programming By Example

You're reading from   R Programming By Example Practical, hands-on projects to help you get started with R

Arrow left icon
Product type Paperback
Published in Dec 2017
Publisher Packt
ISBN-13 9781788292542
Length 470 pages
Edition 1st Edition
Languages
Arrow right icon
Authors (2):
Arrow left icon
Omar Trejo Navarro Omar Trejo Navarro
Author Profile Icon Omar Trejo Navarro
Omar Trejo Navarro
Omar Trejo Navarro Omar Trejo Navarro
Author Profile Icon Omar Trejo Navarro
Omar Trejo Navarro
Arrow right icon
View More author details
Toc

Table of Contents (12) Chapters Close

Preface 1. Introduction to R 2. Understanding Votes with Descriptive Statistics FREE CHAPTER 3. Predicting Votes with Linear Models 4. Simulating Sales Data and Working with Databases 5. Communicating Sales with Visualizations 6. Understanding Reviews with Text Analysis 7. Developing Automatic Presentations 8. Object-Oriented System to Track Cryptocurrencies 9. Implementing an Efficient Simple Moving Average 10. Adding Interactivity with Dashboards 11. Required Packages

Improving our results with TF-IDF

In general in text analysis, a high raw count for a term inside a text does not necessarily mean that the term is more important for the text. One of the most important ways to normalize the term frequencies is to weigh a term by how often it appears not only in a text, but also in the entire corpus.

The more a word appears inside a given text and doesn't appear too much across the whole corpus, it means that it's probably important for that specific text. However, if the term appears a lot inside a text, but also appears a lot in other texts in the corpus, it's probably not important for the specific text, but for the entire corpus, and this dilutes it's predictive power.

In IR, TF-IDF is one of the most popular term-weighting schemes and it's the mathematical implementation of the idea expressed in the preceding paragraph...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime