You're reading from Python Machine Learning Learn how to build powerful Python machine learning algorithms to generate useful data insights with this data analysis tutorial

Product type Paperback

Published in Sep 2015

Publisher Packt

ISBN-13 9781783555130

Length 454 pages

Edition 1st Edition

Languages

Python

Tools

SciPy

Concepts

Machine Learning

Author (1):

Sebastian Raschka

View More author details

Table of Contents (15) Chapters

Preface

1. Giving Computers the Ability to Learn from Data

2. Training Machine Learning Algorithms for Classification FREE CHAPTER

3. A Tour of Machine Learning Classifiers Using Scikit-learn

4. Building Good Training Sets – Data Preprocessing

5. Compressing Data via Dimensionality Reduction

6. Learning Best Practices for Model Evaluation and Hyperparameter Tuning

7. Combining Different Models for Ensemble Learning

8. Applying Machine Learning to Sentiment Analysis

9. Embedding a Machine Learning Model into a Web Application

10. Predicting Continuous Target Variables with Regression Analysis

11. Working with Unlabeled Data – Clustering Analysis

12. Training Artificial Neural Networks for Image Recognition

13. Parallelizing Neural Network Training with Theano

Index

Using regularized methods for regression

As we discussed in Chapter 3, A Tour of Machine Learning Classifiers Using Scikit-learn, regularization is one approach to tackle the problem of overfitting by adding additional information, and thereby shrinking the parameter values of the model to induce a penalty against complexity. The most popular approaches to regularized linear regression are the so-called Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO) and Elastic Net method.

Ridge regression is an L2 penalized model where we simply add the squared sum of the weights to our least-squares cost function:

Using regularized methods for regression

Here:

By increasing the value of the hyperparameter Using regularized methods for regression , we increase the regularization strength and shrink the weights of our model. Please note that we don't regularize the intercept term .

An alternative approach that can lead to sparse models is the LASSO. Depending on the regularization strength, certain weights can become zero, which makes the LASSO also useful...