Linear regression models are the most basic type of regression models and are widely used in predictive data analysis. The overall idea of regression models is to examine two things:

1. Does a set of explanatory features / input variables do a good job at predicting an output variable? Is the model using features that account for the variability in changes to the dependent variable (output variable)?
2. Which features in particular are significant ones of the dependent variable? And in what way do they impact the dependent variable (indicated by the magnitude and sign of the parameters)? These regression parameters are used to explain the relationship between one output variable (dependent variable) and one or more input features (independent variables).

A regression equation will formulate the impact of the input variables (independent variables) on the...

In this section, we are going to go through logistic regression, which is one of the widely used algorithms for classification.

What's logistic regression? The simple definition of logistic regression is that it's a type of classification algorithm involving a linear discriminant.

We are going to clarify this definition in two points:

1. Unlike linear regression, logistic regression doesn't try to estimate/predict the value of the numeric variable given a set of features or input variables. Instead, the output of the logistic regression algorithm is the probability that the given sample/observation belongs to a specific class. In simpler words, let's assume that we have a binary classification problem. In this type of problem, we have only two classes in the output variable, for example, diseased or not diseased. So, the probability...

The sinking of the ship, Titanic, is one of the most infamous events in history. This incident led to the deaths of 1,502 passengers and crew out of 2,224. In this problem, we will use data science to predict whether the passenger will survive this tragedy or not and then test the performance of our model based on the actual statistics of the tragedy.

To follow up with the Titanic example, you need to do the following:

1. Download this repository in a ZIP file by clicking on https://github.com/ahmed-menshawy/ML_Titanic/archive/master.zip or execute from the terminal:
2. Git clone: https://github.com/ahmed-menshawy/ML_Titanic.git
3. Install [virtualenv]: (http://virtualenv.readthedocs.org/en/latest/installation.html)
4. Navigate to the directory where you unzipped or cloned the repo and create a virtual environment with virtualenv ml_titanic...

In machine learning, there are two types of errors, and as a newcomer to data science, you need to understand the crucial difference between both of them. If you end up minimizing the wrong type of error, the whole learning system will be useless and you won’t be able to use it in practice over unseen data. To minimize this kind of misunderstanding between practitioners about these two types of errors, we are going to explain them in the following two sections.

This the first type of error that you don't have to care about minimizing. Getting a small value for this type of error doesn't mean that your model will work well over the unseen data (generalize). To better understand this type of error, we'll give a trivial example of a class scenario. The purpose of solving problems in the classroom is not to be able to solve the same problem again in the exam, but to be able to solve other problems that won’t necessarily be similar to the ones you practiced in the classroom. The exam problems could be from the same family of the classroom problems, but not necessarily identical.

Apparent error is the ability of the trained model to perform on the training set for which we already know the true outcome/output. If you manage to get 0 error over the training set, then it is a good indicator...

This is the second and more important type of error in data science. The whole purpose of building learning systems is the ability to get a smaller generalization error on the test set; in other words, to get the model to work well on a set of observation/samples that haven't been used in the training phase. If you still consider the class scenario from the previous section, you can think of generalization error as the ability to solve exam problems that weren’t necessarily similar to the problems you solved in the classroom to learn and get familiar with the subject. So, generalization performance is the model's ability to use the skills (parameters) that it learned in the training phase in order to correctly predict the outcome/output of unseen data.

In Figure 13, the light blue line represents the generalization error. You can see...

A linear model is a very powerful tool that you can use as an initial learning algorithm if your data matches its assumptions. Understanding linear models will help you to understand more sophisticated models that use linear models as building blocks.

Next up, we will continue using the Titanic example by addressing model complexity and assessment in more detail. Model complexity is a very powerful tool and you need to use it carefully in order to enhance the generalization error. Misunderstanding it will lead to overfitting problems.