Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Practical Data Wrangling

You're reading from   Practical Data Wrangling Expert techniques for transforming your raw data into a valuable source for analytics

Arrow left icon
Product type Paperback
Published in Nov 2017
Publisher Packt
ISBN-13 9781787286139
Length 204 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Author (1):
Arrow left icon
Allan Visochek Allan Visochek
Author Profile Icon Allan Visochek
Allan Visochek
Arrow right icon
View More author details
Toc

Table of Contents (10) Chapters Close

Preface 1. Programming with Data FREE CHAPTER 2. Introduction to Programming in Python 3. Reading, Exploring, and Modifying Data - Part I 4. Reading, Exploring, and Modifying Data - Part II 5. Manipulating Text Data - An Introduction to Regular Expressions 6. Cleaning Numerical Data - An Introduction to R and RStudio 7. Simplifying Data Manipulation with dplyr 8. Getting Data from the Web 9. Working with Large Datasets

Understanding the need for pattern recognition


The simplest way to process the values of text fields to treat them as categorical variables. In a categorical variable, the data entries take on a fixed number of values. To illustrate working with categorical variables, consider a categorical field, such as the US states. If the state of Connecticut, for instance, were to appear in a large enough number of data entries, you might expect to see certain characteristic misspellings, such as the following:

  • Conecticut
  • Conneticut
  • Connetict

An easy way to fix all of the misspellings might be to iterate through each of the data entries and check against a list of common misspellings as is done in the following demonstration. Note that the following code sample is just for demonstration purposes and doesn't belong to a particular file:

misspellings = ["Conecticut", "Conneticut", "Connectict"]
for ind in range(len(data)): 
    if data[ind]["state"] in misspellings:
        data[ind]["state"] = "Connecticut...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image