Exploratory analysis techniques are one part of the larger process of collecting data, learning from data, acting on data, and exploring data to uncover a meaningful pattern. The Exploratory Data Analysis (EDA) is a crucial step to take before diving into advanced analytics and machine learning, as it provides the context needed to develop an appropriate model for the problem at hand and to correctly interpret its results through visualization techniques to tease apart hidden patterns. In this chapter, we will discuss some of EDA's most common and essential practices, in order to summarize and visualize data so that the task of finding trends and patterns becomes causally easier.

In this recipe, we summarize the data using summary functions.

If you have not already downloaded the files for this chapter, do it now and ensure that the auto-mpg.csv file is in your R working directory.

Read the data from auto-mpg.csv, which includes a header row and columns separated by the default "," symbol.

1. Read the data from auto-mpg.csv and convert cylinders to factor:

> auto  <- read.csv("auto-mpg.csv", header = TRUE,     stringsAsFactors = FALSE) 
> # Convert cylinders...

In this recipe, we discuss two ways to subset data. The first approach uses the row and column indices/names and the other uses the subset() function.

Download the files for this chapter and store the auto-mpg.csv file in your R working directory. Read the data using the following command:

> auto <- read.csv("auto-mpg.csv", stringsAsFactors=FALSE) 

The same subsetting principles apply for vectors, lists, arrays, matrices, and data frames. We illustrate with data frames.

The following steps extract...

When we have categorical variables, we often want to create groups corresponding to each level and to analyze each group separately to reveal some significant similarities and differences between them.

The split function divides data into groups based on a factor or vector. The unsplit() function reverses the effect of split.

Download the files for this chapter and store the auto-mpg.csv file in your R working directory. Read the file using the read.csv command and save in the auto variable:

> auto <- read.csv("auto-mpg.csv", stringsAsFactors=FALSE) 

Analysts need an unbiased evaluation of the quality of their machine learning models. To get this, they partition the available data into two parts. They use one part to build the machine learning model and retain the remaining data as hold out data. After building the model, they evaluate the model's performance on the hold out data. This recipe shows you how to partition data. It separately addresses the situations when the target variable is numeric and when it is categorical. It also covers the process of creating two partitions or three.

If you have not already done so, make sure that the BostonHousing.csv and boston-housing-classification.csv files from the code files...

Before even embarking on any numerical analyses, you may want to get a good idea about the data through a few quick plots. The base R system supports basic graphics, so for more advanced plots requirement, we generally use lattice and ggplot packages. In this recipe we will cover the simplest form of basic graphs.

If you have not already done so, download the data files for this chapter and ensure that they are available in your R environment's working directory, and run the following commands:

> auto <- read.csv("auto-mpg.csv") 
> 
> auto$cylinders <- factor(auto$cylinders, levels = c(3,4,5,6,8),     labels...

We often want to see plots side by side for comparisons. This recipe shows how we can achieve this.

If you have not already done so, download the data files for this chapter and ensure that they are available in your R environment's working directory. Once this is done, run the following commands:

> auto <- read.csv("auto-mpg.csv") 
> cylinders <- factor(cylinders,     levels = c(3,4,5,6,8),     labels = c("3cyl", "4cyl", "5cyl", "6cyl", "8cyl")) 
> attach(auto) 

The lattice package produces Trellis plots to capture multivariate relationships in the data. Lattice plots are useful for looking at complex relationships between the variables in a dataset. For example, we may want to see how y changes with x across various levels of z. Using the lattice package, we can draw histograms, boxplots, scatterplots, dot plots, and so on. Both plotting and annotation are done in one single call.

Download the files for this chapter and store the auto-mpg.csv file in your R working directory. Read the file using the read.csv function and save in the auto variable. Convert cylinders into a factor variable:

> auto <- read.csv("auto-mpg...

In large datasets, we often gain good insights by examining how different segments behave. The similarities and differences can reveal interesting patterns. This recipe shows how to create graphs that enable such comparisons with different variable types.

If you have not already done so, download the book's files for this chapter and save the daily-bike-rentals.csv file in your R working directory. Read the data into R using
the following command and check the packages needed too:

> library(dplyr)

> bike <- read.csv("daily-bike-rentals.csv")
> bike$season <- factor(bike$season, levels = c(1,2,3,4),labels = c("Spring", "Summer...

When presenting data, rather than merely presenting information, we usually want to present an explanation of some phenomenon. Visualizing hypothesized causality helps to communicate our ideas clearly.

If you have not already done so, download the book's files for this chapter and save the hourly-bike-rentals.csv file in your R working directory. Read the data into R as follows:

> library(lattice)
> bike <- read.csv("daily-bike-rentals.csv")
> bike$season <- factor(bike$season, levels = c(1,2,3,4), 
    labels = c("Spring", "Summer", "Fall", "Winter"))
> bike$weathersit <- factor(bike...