Understanding the role of data manipulation skills

In practical situations, we rarely have our data in the format that we want; we usually have different datasets that we want to merge, and often, we need to normalize and clean up the data. For these reasons, data manipulation and preparation will always play a big part in any data visualization process. So, we will be focusing on this in this chapter and throughout the book.

The plan for preparing our dataset is roughly the following:

1. Explore the different files one by one.
2. Check the available data and data types and explore how each can help us categorize and analyze the data.
3. Reshape the data where required.
4. Combine different DataFrames to add more ways to describe our data.

Let's go through these steps right away.

Exploring the data files

We start by reading in the files in the data folder:

import os
import pandas as pd
pd.options.display.max_columns = None
os.listdir('data&apos...