This chapter continues our work on importing data from a variety of sources and the initial checks we should do on the data after importing it. Over the last 25 years, data analysts have found that they increasingly need to work with data in non-tabular, semi-structured forms. Sometimes, they even create and persist data in those forms. We will work with a common alternative to traditional tabular datasets in this chapter, JSON, but the general concepts can be extended to XML and NoSQL data stores such as MongoDB. We will also go over common issues that occur when scraping data from websites.

Data analysts have also been finding that increases in the volume of data to be analyzed have been even greater than improvements in machine processing power, at least those computing resources that are available locally. Working with big data sometimes requires us to rely on technology like Apache Spark, which...

Versioning data

There may be times when we want to persist data without overwriting a prior version of the data file. This can be accomplished by appending a time stamp to a filename or a unique identifier. However, there are more elegant solutions available. One such solution is the Delta Lake library, which we will explore in this recipe.

We will work with the land temperature data again in this recipe. We will load the data, save it to a data lake, and then save an altered version to the same data lake.

Getting ready

We will be using the Delta Lake library in this recipe, which can be installed with pip install deltalake. We will also need the os library so that we can make a directory for the data lake.

How to do it...

You can get started with the data and version it as follows:

We start by importing the Delta Lake library. We also create a folder called temps_lake for our data versions:
```
import pandas as pd
from deltalake.writer import write_deltalake...
```

Download Code

Key benefits

Get to grips with new techniques for data preprocessing and cleaning for machine learning and NLP models
Use new and updated AI tools and techniques for data cleaning tasks
Clean, monitor, and validate large data volumes to diagnose problems using cutting-edge methodologies including Machine learning and AI

Description

Jumping into data analysis without proper data cleaning will certainly lead to incorrect results. The Python Data Cleaning Cookbook - Second Edition will show you tools and techniques for cleaning and handling data with Python for better outcomes. Fully updated to the latest version of Python and all relevant tools, this book will teach you how to manipulate and clean data to get it into a useful form. he current edition focuses on advanced techniques like machine learning and AI-specific approaches and tools for data cleaning along with the conventional ones. The book also delves into tips and techniques to process and clean data for ML, AI, and NLP models. You will learn how to filter and summarize data to gain insights and better understand what makes sense and what does not, along with discovering how to operate on data to address the issues you've identified. Next, you’ll cover recipes for using supervised learning and Naive Bayes analysis to identify unexpected values and classification errors and generate visualizations for exploratory data analysis (EDA) to identify unexpected values. Finally, you’ll build functions and classes that you can reuse without modification when you have new data. By the end of this Data Cleaning book, you'll know how to clean data and diagnose problems within it.

Who is this book for?

This book is for anyone looking for ways to handle messy, duplicate, and poor data using different Python tools and techniques. The book takes a recipe-based approach to help you to learn how to clean and manage data with practical examples. Working knowledge of Python programming is all you need to get the most out of the book.

What you will learn

Using OpenAI tools for various data cleaning tasks
Producing summaries of the attributes of datasets, columns, and rows
Anticipating data-cleaning issues when importing tabular data into pandas
Applying validation techniques for imported tabular data
Improving your productivity in pandas by using method chaining
Recognizing and resolving common issues like dates and IDs
Setting up indexes to streamline data issue identification
Using data cleaning to prepare your data for ML and AI models

Estimated delivery fee Deliver to Russia

Economy delivery 10 - 13 business days

$6.95

Premium delivery 6 - 9 business days

$21.95

(Includes tracking information)

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$34.98 ~~$49.99~~

$39.98 ~~$49.99~~

The Machine Learning Solutions Architect Handbook

$34.98 ~~$49.99~~

Total $ 109.94 149.97 40.03 saved

FAQs

What is the delivery time and cost of print book?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

What is custom duty/charge?

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order?

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges?

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.

How can I cancel my order?

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy?

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged?

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use?

You can pay with the following card types:

Visa Debit
Visa Credit
MasterCard
PayPal

What is the delivery time and cost of print books?

Shipping Details

USA:

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Unfortunately, due to several restrictions, we are unable to ship to the following countries:

Afghanistan
American Samoa
Belarus
Brunei Darussalam
Central African Republic
The Democratic Republic of Congo
Eritrea
Guinea-bissau
Iran
Lebanon
Libiya Arab Jamahriya
Somalia
Sudan
Russian Federation
Syrian Arab Republic
Ukraine
Venezuela

Filter reviews by

All

Packt verified reviews

Feefo verified reviews

Amazon verified reviews

Petar Dimov Sep 13, 2024

I recently finished reading the second edition of Mike Walker’s 𝗣𝘆𝘁𝗵𝗼𝗻 𝗗𝗮𝘁𝗮 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸.💡 𝗜𝘁’𝘀 𝗼𝗳𝘁𝗲𝗻 𝘀𝗮𝗶𝗱 𝘁𝗵𝗮𝘁 𝗱𝗮𝘁𝗮 𝘀𝗰𝗶𝗲𝗻𝗰𝗲 𝗶𝘀 𝗺𝗼𝘀𝘁𝗹𝘆 𝗮𝗯𝗼𝘂𝘁 𝗰𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗱𝗮𝘁𝗮, 𝗮𝘀 𝗶𝗳 𝗶𝘁’𝘀 𝗮𝗻 𝘂𝗻𝗱𝗲𝘀𝗶𝗿𝗮𝗯𝗹𝗲 𝘁𝗮𝘀𝗸. However, this perspective overlooks its significance. Just as prepping ingredients is crucial in cooking or collecting materials is necessary for writing, 𝗱𝗮𝘁𝗮 𝗰𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗶𝘀 𝗮 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗮𝘀𝗽𝗲𝗰𝘁 𝗼𝗳 𝗱𝗮𝘁𝗮 𝘀𝗰𝗶𝗲𝗻𝗰𝗲. 𝗧𝗵𝗲 𝗯𝗲𝘁𝘁𝗲𝗿 𝘆𝗼𝘂 𝗮𝗿𝗲 𝗮𝘁 𝗶𝘁, 𝘁𝗵𝗲 𝗺𝗼𝗿𝗲 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗮𝗻𝗱 𝘃𝗮𝗹𝘂𝗮𝗯𝗹𝗲 𝘆𝗼𝘂𝗿 𝘄𝗼𝗿𝗸 𝗯𝗲𝗰𝗼𝗺𝗲𝘀.✍ 𝗜 𝗹𝗼𝘃𝗲𝗱 𝗵𝗼𝘄 𝘁𝗵𝗲 𝗯𝗼𝗼𝗸 𝗷𝘂𝗺𝗽𝘀 𝗿𝗶𝗴𝗵𝘁 𝗶𝗻, offering practical tasks for immediate results. The cookbook style makes it simple and actionable.🔧 𝗧𝗵𝗲 𝗯𝗼𝗼𝗸 𝗱𝗲𝗹𝘃𝗲𝗱 𝗶𝗻𝘁𝗼 𝘂𝘁𝗶𝗹𝗶𝘇𝗶𝗻𝗴 𝗢𝗽𝗲𝗻𝗔𝗜 𝘁𝗼𝗼𝗹𝘀 𝗳𝗼𝗿 𝗱𝗶𝘃𝗲𝗿𝘀𝗲 𝗱𝗮𝘁𝗮 𝗰𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝘁𝗮𝘀𝗸𝘀, showcasing how AI can enhance the efficiency and effectiveness of data preprocessing. It also emphasized summarizing dataset attributes, providing a comprehensive overview of data at both column and row levels.🚨 𝗪𝗮𝗹𝗸𝗲𝗿 𝗮𝗱𝗱𝗿𝗲𝘀𝘀𝗲𝗱 𝗮𝗻𝘁𝗶𝗰𝗶𝗽𝗮𝘁𝗶𝗻𝗴 𝗶𝘀𝘀𝘂𝗲𝘀 𝘄𝗶𝘁𝗵 𝘁𝗮𝗯𝘂𝗹𝗮𝗿 𝗱𝗮𝘁𝗮 𝗶𝗻 𝗽𝗮𝗻𝗱𝗮𝘀, helping readers proactively tackle common data import problems. 𝗘𝗻𝗵𝗮𝗻𝗰𝗶𝗻𝗴 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝘃𝗶𝘁𝘆 𝘄𝗶𝘁𝗵 𝗺𝗲𝘁𝗵𝗼𝗱 𝗰𝗵𝗮𝗶𝗻𝗶𝗻𝗴 𝗶𝗻 𝗽𝗮𝗻𝗱𝗮𝘀 was another key highlight, streamlining workflows and improving code readability.⚙️ 𝗧𝗵𝗲 𝗯𝗼𝗼𝗸 𝗰𝗼𝘃𝗲𝗿𝗲𝗱 𝗵𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗰𝗼𝗺𝗺𝗼𝗻 𝗱𝗮𝘁𝗮 𝗶𝘀𝘀𝘂𝗲𝘀, including problematic dates and IDs, and setting up indexes to quickly identify and assess data problems. 𝗜𝘁 𝗮𝗹𝘀𝗼 𝗲𝗺𝗽𝗵𝗮𝘀𝗶𝘇𝗲𝗱 𝗽𝗿𝗲𝗽𝗮𝗿𝗶𝗻𝗴 𝗱𝗮𝘁𝗮 𝗳𝗼𝗿 𝗠𝗟 𝗮𝗻𝗱 𝗔𝗜 𝗺𝗼𝗱𝗲𝗹𝘀, ensuring that the data is clean and reliable for advanced analytics.✨ 𝗢𝘃𝗲𝗿𝗮𝗹𝗹, 𝗜 𝗳𝗼𝘂𝗻𝗱 𝘁𝗵𝗶𝘀 𝗯𝗼𝗼𝗸 𝗯𝗲𝗻𝗲𝗳𝗶𝗰𝗶𝗮𝗹 𝗮𝗻𝗱 𝗲𝗻𝗷𝗼𝘆𝗮𝗯𝗹𝗲. 𝗜𝘁 𝗿𝗲𝗹𝗶𝗲𝘃𝗲𝗱 𝘁𝗵𝗲 𝗯𝘂𝗿𝗱𝗲𝗻 𝗼𝗳 𝗱𝗮𝘁𝗮 𝗰𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗺𝗮𝗱𝗲 𝗶𝘁 𝗳𝗲𝗲𝗹 𝗹𝗶𝗸𝗲 𝗮 𝗳𝗹𝘂𝗶𝗱 𝗽𝗮𝗿𝘁 𝗼𝗳 𝘁𝗵𝗲 𝗽𝗿𝗼𝗰𝗲𝘀𝘀.

Amazon Verified review

Vishi kaushal Sep 09, 2024

This book cover all the concepts in details with proper exercise. You should go for this if you are starting your data journey.

Bryan Edwards Jul 29, 2024

Great book - the author does a great job explaining the various concepts, and the examples are very helpful

Feefo Verified review

Matvey Osipov Jul 28, 2024

Great book if you want to know how to read the different formats of data into a pandas dataframe and perform exploratory analysis and many different types of transformations.The structure of the book is great for beginners and experts alike, it breaks topics down into: setup, how to do it, AND how it works - making it perfect as a reference if you need something quick from your library or repo. The book itself covers the basics you need (get data, look for issues using visualizations, fix issues) to more complex operations like modeling/joining different data together. Highly recommend overall!

N/A Jul 27, 2024

Python Data Cleaning Cookbook: Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI , Second Edition

What do you get with Print?

Python Data Cleaning Cookbook

Anticipating Data Cleaning Issues When Working with HTML, JSON, and Spark Data

Technical requirements

Importing simple JSON data

Importing more complicated JSON data from an API

Importing data from web pages

Working with Spark data

Getting ready

Persisting JSON data

Versioning data

Getting ready

How to do it...

Summary

Join our community on Discord

Page 1 of 9

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs

Python Data Cleaning Cookbook: Prepare your data for analysis with pandas, NumPy, Matplotlib, scikit-learn, and OpenAI , Second Edition

What do you get with Print?

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs