Python Data Cleaning Cookbook: Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI , Second Edition

Michael Walker

£37.99

4.9 (24 Ratings)

Paperback May 2024 486 pages 2nd Edition

Michael Walker

£37.99

4.9 (24 Ratings)

Paperback May 2024 486 pages 2nd Edition

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Preview Book

Python Data Cleaning Cookbook

Anticipating Data Cleaning Issues When Working with HTML, JSON, and Spark Data

This chapter continues our work on importing data from a variety of sources and the initial checks we should do on the data after importing it. Over the last 25 years, data analysts have found that they increasingly need to work with data in non-tabular, semi-structured forms. Sometimes, they even create and persist data in those forms. We will work with a common alternative to traditional tabular datasets in this chapter, JSON, but the general concepts can be extended to XML and NoSQL data stores such as MongoDB. We will also go over common issues that occur when scraping data from websites.

Data analysts have also been finding that increases in the volume of data to be analyzed have been even greater than improvements in machine processing power, at least those computing resources that are available locally. Working with big data sometimes requires us to rely on technology like Apache Spark, which...

Versioning data

There may be times when we want to persist data without overwriting a prior version of the data file. This can be accomplished by appending a time stamp to a filename or a unique identifier. However, there are more elegant solutions available. One such solution is the Delta Lake library, which we will explore in this recipe.

We will work with the land temperature data again in this recipe. We will load the data, save it to a data lake, and then save an altered version to the same data lake.

Getting ready

We will be using the Delta Lake library in this recipe, which can be installed with pip install deltalake. We will also need the os library so that we can make a directory for the data lake.

How to do it...

You can get started with the data and version it as follows:

We start by importing the Delta Lake library. We also create a folder called temps_lake for our data versions:
```
import pandas as pd
from deltalake.writer import write_deltalake...
```

Download Code

Key benefits

Get to grips with new techniques for data preprocessing and cleaning for machine learning and NLP models
Use new and updated AI tools and techniques for data cleaning tasks
Clean, monitor, and validate large data volumes to diagnose problems using cutting-edge methodologies including Machine learning and AI

Description

Jumping into data analysis without proper data cleaning will certainly lead to incorrect results. The Python Data Cleaning Cookbook - Second Edition will show you tools and techniques for cleaning and handling data with Python for better outcomes. Fully updated to the latest version of Python and all relevant tools, this book will teach you how to manipulate and clean data to get it into a useful form. he current edition focuses on advanced techniques like machine learning and AI-specific approaches and tools for data cleaning along with the conventional ones. The book also delves into tips and techniques to process and clean data for ML, AI, and NLP models. You will learn how to filter and summarize data to gain insights and better understand what makes sense and what does not, along with discovering how to operate on data to address the issues you've identified. Next, you’ll cover recipes for using supervised learning and Naive Bayes analysis to identify unexpected values and classification errors and generate visualizations for exploratory data analysis (EDA) to identify unexpected values. Finally, you’ll build functions and classes that you can reuse without modification when you have new data. By the end of this Data Cleaning book, you'll know how to clean data and diagnose problems within it.

Who is this book for?

This book is for anyone looking for ways to handle messy, duplicate, and poor data using different Python tools and techniques. The book takes a recipe-based approach to help you to learn how to clean and manage data with practical examples. Working knowledge of Python programming is all you need to get the most out of the book.

What you will learn

Using OpenAI tools for various data cleaning tasks
Producing summaries of the attributes of datasets, columns, and rows
Anticipating data-cleaning issues when importing tabular data into pandas
Applying validation techniques for imported tabular data
Improving your productivity in pandas by using method chaining
Recognizing and resolving common issues like dates and IDs
Setting up indexes to streamline data issue identification
Using data cleaning to prepare your data for ML and AI models

Estimated delivery fee Deliver to Great Britain

Standard delivery 1 - 4 business days

£4.95

Premium delivery 1 - 4 business days

£7.95

(Includes tracking information)

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

£37.99

The Machine Learning Solutions Architect Handbook

£37.99

Total £ 113.97

FAQs

What is the delivery time and cost of print book?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

What is custom duty/charge?

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order?

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges?

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.

How can I cancel my order?

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy?

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged?

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use?

You can pay with the following card types:

Visa Debit
Visa Credit
MasterCard
PayPal

What is the delivery time and cost of print books?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

Filter reviews by

All

Packt verified reviews

Feefo verified reviews

Amazon verified reviews

Absar Jan 30, 2024

good content..waiting for the rest of the chapters

Subscriber review

Bryan Edwards Jul 29, 2024

Great book - the author does a great job explaining the various concepts, and the examples are very helpful

Feefo Verified review

N/A Jul 27, 2024

Airton Leal Jun 10, 2024

This book is a goldmine of practical techniques for wrangling your data into shape using powerful Python libraries like pandas, NumPy, Matplotlib, scikit-learn, and the exciting newcomer - OpenAI tools.The content and inclusion of OpenAI tools, reflecting the latest advancements in the field. The hands-on approach ensures that readers not only understand the theoretical aspects of data cleaning but also acquire practical skills by working through real datasets. The clear, concise explanations and step-by-step instructions make it easy to follow along, while the numerous code snippets and illustrations help solidify understanding. Whether you're a data analyst, scientist, or engineer, this cookbook is an invaluable tool for enhancing your data cleaning capabilities and ensuring your analyses are built on a solid foundation of well-prepared data.

Amazon Verified review

James W Jun 01, 2024

Discover how to describe your data in detail, identify data issues, and find out how to solve them using commonly used techniques and tips and tricks.Key Features I picked out from the bookVarious data cleaning techniques to reveal key insights to manipulate data of different complexities to shape them into the right form.Clean, monitor, and validate large data volumes to diagnose problems before moving on to data analysis.Book DescriptionGetting clean data to reveal insights is essential, as directly jumping into data analysis without proper data cleaning may lead to incorrect results.This book shows you tools and techniques that you can apply to clean and handle data with Python.You'll begin by getting familiar with the shape of data by using practices that can be deployed routinely with most data sources. Then, the book teaches you how to manipulate data to get it into a useful form.You'll also learn how to filter and summarise data to gain insights and better understand what makes sense and what does not, along with discovering how to operate on data to address the issues you've identified.Moving on, you'll perform key tasks, such as handling missing values, validating errors, removing duplicate data, monitoring high volumes of data, and handling outliers and invalid dates.Next, you'll cover recipes on using supervised learning and Naive Bayes analysis to identify unexpected values and classification errors, and generate visualisations for exploratory data analysis (EDA) to visualise unexpected values.Finally, you'll build functions and classes that you can reuse without modification when you have new data.By the end of this Python book, you'll be equipped with all the key skills that you need to clean data and diagnose problems within it.

Python Data Cleaning Cookbook: Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI , Second Edition

What do you get with Print?

Python Data Cleaning Cookbook

Anticipating Data Cleaning Issues When Working with HTML, JSON, and Spark Data

Technical requirements

Importing simple JSON data

Importing more complicated JSON data from an API

Importing data from web pages

Working with Spark data

Getting ready

Persisting JSON data

Versioning data

Getting ready

How to do it...

Summary

Join our community on Discord

Page 1 of 9

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs

Python Data Cleaning Cookbook: Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI , Second Edition

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs