During this chapter we will make use of the following R packages. If you don't already have them installed, you can look into Appendix, Required Packages for instructions on how do so.

In June 2016, a referendum was held in the United Kingdom (UK) to decide whether or not to remain part of the European Union (EU). 72% of registered voters took part, and of those, 51.2% voted to leave the EU. In February 2017, Martin Rosenbaum, freedom of information specialist at BBC News, published the article, Local Voting Figures Shed New Light on EU Referendum (http://www.bbc.co.uk/news/uk-politics-38762034). He obtained data from 1,070 electoral wards (the smallest administrative division for electoral purposes in the UK), with numbers for Leave and Remain votes in each ward.

Martin Rosenbaum calculated some statistical associations between the proportion of Leave votes in a ward and some of its social, economic, and demographic characteristics by making use of the most recent UK census, which was conducted in 2011. He used his data for a university...

Setting up the data for this example is straightforward. We will load the data, correctly label missing values, and create some new variables for our analysis. Before we start, make sure the data.csv file is in the same directory as the code you're working with, and that your working directory is properly setup. If you don't know how to do so, setting up your working directory is quite easy, you simply call the setwd() function passing the directory you want to use as such. For example, setwd(/home/user/examples/) would use the /home/user/examples directory to look for files, and save files to.

If you don’t know how to do so, setting up your working directory is quite easy, you simply call the setwd() function passing the directory you want to use as such. For example, setwd(/home/user/examples/) would use the /home/user/examples...

To get a summary of the data, we may execute summary(data) and see the relevant summaries for each type of variable. The summary is tailored for each column's data type. As you can see, numerical variables such as ID and NVotes get a quantile summary, while factor (categorical) variables get a count for each different category, such as AreaType and RegionName. If there are many categories, the summary will show the categories that appear the most and group the rest into a (Other) group, as we can see at the bottom of RegionName.

summary(data)
#>       ID        RegionName              NVotes          Leave
#> Min.   :  1    Length: 1070        Min.   : 1039    Min.   : 287
#> 1st Qu.: 268   Class : character   1st Qu.: 4252    1st Qu.: 1698
#> Median : 536   Mode  : character   Median : 5746    Median : 2874
#> Mean   : 536...

Now that we have some clean data to work with, we will create lots of plots to build intuition about the data. In this chapter, we will work with plots that are easy to create and are used for exploratory purposes. In Chapter 4, Simulating Sales Data and Working with Databases, we will look into publication ready plots that are a little more verbose to create.

Our first plot is a simple one and shows the proportion of votes by each RegionName. As you can see in the plot shown below, the London, North West, and West Midlands regions account for around 55 percent of the observations in the data.

To create the plot, we...

What we have learned so far in this chapter is that age, education, and ethnicity are important factors in understanding the way people voted in the Brexit Referendum. Younger people with higher education levels are related with votes in favor of remaining in the EU. Older white people are related with votes in favor of leaving the EU. We can now use this knowledge to make a more succinct data set that incorporates this knowledge. First we add relevant variables, and then we remove non-relevant variables.

Our new relevant variables are two groups of age (adults below and above 45), two groups of ethnicity (whites and non-whites), and two groups of education (high and low education levels):

data$Age_18to44 <- (
    data$Age_18to19 +
    data$Age_20to24 +
    data$Age_25to29 +
    data$Age_30to44
)
data$Age_45plus <- (
 ...

Principal Component Analysis (PCA) is a dimensionality reduction technique that is widely used in data analysis when there are many numerical variables, some of which may be correlated, and we would like to reduce the number of dimensions required to understand the data.

It can be useful to help us understand the data, since thinking in more than three dimensions can be problematic, and to accelerate algorithms that are computationally intensive, especially with large numbers of variables. With PCA, we can extract most of the information into only one or two variables constructed in a very specific way, such that they capture the most variance while having the added benefit of being uncorrelated among them by construction.

The first principal component is a linear combination of the original variables which captures the maximum...

Now that we have the fundamentals about analyzing data with descriptive statistics, we're going to improve our code's structure and flexibility by breaking it up into functions. Even though this is common knowledge among efficient programmers, it's not a common practice among data analysts. Many data analysts would simply paste the code we have developed all together, as-is, into a single file, and run it every time they wanted to perform the analysis. We won't be adding new features to the analysis. All we'll do is reorder code into functions to encapsulate their inner-workings and communicate intention with function names (this substantially reduces the need for comments).

We'll focus on producing high-quality code that is easy to read, reuse, modify, and fix (in case of bugs). The way we actually do...

This chapter showed how to perform a qualitative analysis that is useful as a first step when doing data analysis. We showed some descriptive statistics techniques and how to implement them programmatically. With these skills, we are able to perform simple yet powerful analyses and save the results for later use. Specifically, we showed how to do basic data cleaning, how to create graphs programmatically, how to create matrix scatter plots and matrix correlations, how to perform Principal Component Analysis, and how to combine these tools to understand the data at hand. Finally, we touched on the basics of high-quality code and showed how to transform your initial data analysis code into programs that are modular, flexible, and easy to work with.

In Chapter 3, Predicting Votes with Linear Models, we'll show how to extend the current analysis with qualitative tools...