Alright, if you want to be a data scientist, we need to talk about the types of data that you might encounter, how to categorize them, and how you might treat them differently. Let's dive into the different flavors of data you might encounter:

This will seem pretty basic, but we've got to start with the simple stuff and we'll work our way up to the more complicated data mining and machine learning things. It is important to know what kind of data you're dealing with because different techniques might have different nuances depending on what kind of data you're handling. So, there are several flavors of data, if you will, and there are three specific types of data that we will primarily focus on. They are:

• Numerical data
• Categorical data
• Ordinal data

Again, there are different variations of techniques that you might use for different types...

Let's do a little refresher of statistics 101. This is like elementary school stuff, but good to go through it again and see how these different techniques are used: Mean, median, and mode. I'm sure you've heard those terms before, but it's good to see how they're used differently, so let's dive in.

This should be a review for most of you, a quick refresher, now that we're starting to actually dive into some real statistics. Let's look at some actual data and figure out how to measure these things.

The mean, as you probably know, is just another name for the average. To calculate the mean of a dataset, all you have to do is sum up all the values and divide...

Let's start doing some real coding in Python and see how you compute the mean, median, and mode using Python in an IPython Notebook file.

So go ahead and open up the MeanMedianMode.ipynb file from the data files for this section if you'd like to follow along, which I definitely encourage you to do. If you need to go back to that earlier section on where to download these materials from, please go do that, because you will need these files for the section. Let's dive in!

What we're going to do is create some fake income data, getting back to our example from the previous section. We're going to create some fake data where...

Let's talk about standard deviation and variance. The concepts and terms you've probably heard before, but let's go into a little bit more depth about what they really mean and how you compute them. It's a measure of the spread of a data distribution, and that will make a little bit more sense in a few minutes.

Standard deviation and variance are two fundamental quantities for a data distribution that you'll see over and over again in this book. So, let's see what they are, if you need a refresher.

Let's look at a histogram, because variance and standard deviation are all about the spread of the data, the shape of the distribution of a dataset. Take...

So we've already seen some examples of a normal distribution function for some of the examples in this book. That's an example of a probability density function, and there are other types of probability density functions out there. So let's dive in and see what it really means and what some other examples of them are.

We've already seen some examples of a normal distribution function for some of the code we've looked at in this book. That's an example of a probability density function, and there are other types of probability density functions out there. Let's dive in...

Let's look at some real examples of probability distribution functions and data distributions in general and wrap your head a little bit more around data distributions and how to visualize them and use them in Python.

Go ahead and open up the Distributions.ipynb from the book materials, and you can follow along with me here if you'd like.

Let's start off with a really simple example: uniform distribution. A uniform distribution just means there's a flat constant probability of a value occurring within a given range.

import numpy as np 
Import matplotlib.pyplot as plt 
 
values = np.random.uniform(-10.0, 10.0, 100000) 
plt.hist(values, 50) 
plt.show() 

So...

Next, we'll talk about percentiles and moments. You hear about percentiles in the news all the time. People that are in the top 1% of income: that's an example of percentile. We'll explain that and have some examples. Then, we'll talk about moments, a very fancy mathematical concept, but it turns out it's very simple to understand conceptually. Let's dive in and talk about percentiles and moments, a couple of a pretty basic concepts in statistics, but again, we're working our way up to the hard stuff, so bear with me as we go through some of this review.

Let's see what percentiles mean. Basically, if you were to sort all of the data in a dataset, a...

In this chapter, we saw the types of data (numeric, categorical, and ordinal data) that you might encounter and how to categorize them and how you treat them differently depending on what kind of data you're dealing with. We also walked through the statistical concepts of mean, median and mode, and we also saw the importance of choosing between median and mean, and that often the median is a better choice than the mean because of outliers.

Next, we analyzed how to compute mean, median, and mode using Python in an IPython Notebook file. We learned the concepts of standard deviation and variance in depth and how to compute them in Python. We saw that they’re a measure of the spread of a data distribution. We also saw a way to visualize and measure the actual chance of a given range of values occurring in a dataset using probability density functions and probability...