Data Science Projects with Python: A case study approach to successful data science projects using Python, pandas, and scikit-learn

The first chapter got you started with some basic Python, and then progressed to equipping you with tools for data exploration. Specifically, we performed operations such as loading the dataset and verifying data integrity, and we performed our first exploratory analysis on our case study dataset.

In this chapter, we finish our exploration of the data by examining the response variable. After we've concluded that the data is of high quality and makes sense, we will be ready to move forward with the practical concerns of developing machine learning models. We will take our first steps with scikit-learn, one of the most popular machine learning packages available in the Python language. Before learning the details of how mathematical models work in the next chapter, here we'll start to get comfortable with the syntax for using them in scikit-learn.

We will also learn some common techniques for how to answer the question, "Is this model good or not?" There are many possible ways...

We have now looked through all the features to see whether any data is missing, as well as to generally examine them. The features are important because they constitute the inputs to our machine learning algorithm. On the other side of the model lies the output, which is a prediction of the response variable. For our problem, this is a binary flag indicating whether or not an account will default the next month, which would have been October for our historical dataset.

Our overall job on this project is to come up with a predictive model for this target. Since the response variable is a yes/no flag, this problem is called a binary classification task. In our labeled data, the samples (accounts) that defaulted (that is, 'default payment next month' = 1) are said to belong to the positive class, while those that didn't belong to the negative class. The key piece of information to examine regarding the response of a binary...

While pandas will save you a lot of time in loading, examining, and cleaning data, the machine learning algorithms that will enable you to do predictive modeling are located in other packages. We consider scikit-learn to be the premier machine learning package for Python, outside of deep learning. While it's impossible for any one package to offer "everything," scikit-learn comes pretty close in terms of accommodating a wide range of approaches for classification and regression, and unsupervised learning. That being said, a few other packages you should also be aware of are as follows:

SciPy:

StatsModels:

Before we start building predictive models in earnest, we would like to know how we can determine, once we've created a model, whether it is "good" in some sense of the word. As you may imagine, this question has received a lot of attention from researchers and practitioners. Consequently, there is a wide variety of model performance metrics to choose from.

When selecting a model performance metric to assess the predictive quality of a model, it's important to keep two things in mind.

Appropriateness of the metric for the problem

Metrics are typically only defined for a specific class of problems, such as classification or regression. For a binary classification problem, several metrics characterize the correctness of the yes or no question that the model answers. An...

In this chapter, we finished the initial exploration of the case study data by examining the response variable. Once we became confident in the completeness and correctness of the data set, we became prepared to explore the relation between features and response and build models.

We spent much of this chapter getting used to model fitting in scikit-learn at a technical, coding level, and learning about metrics we could use with the binary classification problem of the case study. When trying different feature sets and different kinds of models, you will need some way to tell if one approach is working better than another. Consequently, you'll need to use model performance metrics.

While accuracy is a familiar and intuitive metric, we learned why it may not give a useful assessment of the performance of a classifier. We learned how to use a majority class null model to tell whether an accuracy rate is truly good, or no better than what would result from prediction of simply the most...