Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Applied Deep Learning with Keras

You're reading from   Applied Deep Learning with Keras Solve complex real-life problems with the simplicity of Keras

Arrow left icon
Product type Paperback
Published in Apr 2019
Publisher
ISBN-13 9781838555078
Length 412 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Authors (3):
Arrow left icon
Matthew Moocarme Matthew Moocarme
Author Profile Icon Matthew Moocarme
Matthew Moocarme
Mahla Abdolahnejad Mahla Abdolahnejad
Author Profile Icon Mahla Abdolahnejad
Mahla Abdolahnejad
Ritesh Bhagwat Ritesh Bhagwat
Author Profile Icon Ritesh Bhagwat
Ritesh Bhagwat
Arrow right icon
View More author details
Toc

L1 and L2 Regularization


The most common type of regularization for deep learning models is the one that keeps the weights of the network small. This type of regularization is called weight regularization and has two different variations: L2 regularization and L1 regularization. In this section, you will learn about these regularization methods in detail, along with how to implement them in Keras. Additionally, you will practice applying them to real-life problems and observe how they can improve the performance of a model.

L1 and L2 Regularization Formulation

In weight regularization, a penalizing term is added to the loss function. This term is either L2 norm (the sum of the squared values) of the weights, or L1 norm (the sum of the absolute values) of the weights. If L1 norm is used, then it will be called L1 regularization. If L2 norm is used, then it will be called L2 regularization. In each case, the sum is multiplied by a hyperparameter called a regularization parameter (lambda).

Therefore...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at €18.99/month. Cancel anytime