Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Neural Networks with Keras Cookbook
Neural Networks with Keras Cookbook

Neural Networks with Keras Cookbook: Over 70 recipes leveraging deep learning techniques across image, text, audio, and game bots

Arrow left icon
Profile Icon V Kishore Ayyadevara
Arrow right icon
€32.99
Full star icon Full star icon Full star icon Half star icon Empty star icon 3.3 (8 Ratings)
Paperback Feb 2019 568 pages 1st Edition
eBook
€25.99
Paperback
€32.99
Subscription
Free Trial
Renews at €18.99p/m
Arrow left icon
Profile Icon V Kishore Ayyadevara
Arrow right icon
€32.99
Full star icon Full star icon Full star icon Half star icon Empty star icon 3.3 (8 Ratings)
Paperback Feb 2019 568 pages 1st Edition
eBook
€25.99
Paperback
€32.99
Subscription
Free Trial
Renews at €18.99p/m
eBook
€25.99
Paperback
€32.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Table of content icon View table of contents Preview book icon Preview Book

Neural Networks with Keras Cookbook

Building a Deep Feedforward Neural Network

In this chapter, we will cover the following recipes:

  • Training a vanilla neural network
  • Scaling the input dataset
  • Impact of training when the majority of inputs are greater than zero
  • Impact of batch size on model accuracy
  • Building a deep neural network to improve network accuracy
  • Varying the learning rate to improve network accuracy
  • Varying the loss optimizer to improve network accuracy
  • Understanding the scenario of overfitting
  • Speeding up the training process using batch normalization

In the previous chapter, we looked at the basics of the function of a neural network. We also learned that there are various hyperparameters that impact the accuracy of a neural network. In this chapter, we will get into the details of the functions of the various hyperparameters within a neural network.

All the codes for this chapter are available at...

Training a vanilla neural network

To understand how to train a vanilla neural network, we will go through the task of predicting the label of a digit in the MNIST dataset, which is a popular dataset of images of digits (one digit per image) and the corresponding label of the digit that is contained in the image.

Getting ready

Training a neural network is done in the following steps:

  1. Import the relevant packages and datasets
  2. Preprocess the targets (convert them into one-hot encoded vectors) so that we can perform optimization on top of them:
    • We shall be minimizing categorical cross entropy loss
  3. Create train and test datasets:
    • We have the train dataset so that we create a model based on it
    • The test dataset is not seen...

Scaling the input dataset

Scaling a dataset is a process where we limit the variables within a dataset to ensure they do not have a very wide range of different values. One way to achieve this is to divide each variable in the dataset by the maximum value of the variable. Typically, neural networks perform well when we scale the input datasets.

In this section, let's understand the reason neural networks perform better when the dataset is scaled.

Getting ready

To understand the impact of the scaling input on the output, let's contrast the scenario where we check the output when the input dataset is not scaled, with the output when the input dataset is scaled.

Input data is not scaled:

In the preceding table, note...

Impact on training when the majority of inputs are greater than zero

So far, in the dataset that we have considered, we have not looked at the distribution of values in the input dataset. Certain values of the input result in faster training. In this section, we will understand a scenario where weights are trained faster when the training time depends on the input values.

Getting ready

In this section, we will follow the model-building process in exactly the same way as we did in the previous section.

However, we will adopt a small change to our strategy:

  • We will invert the background color, and also the foreground color. Essentially, the background will be colored white in this scenario, and the label will be written in...

Impact of batch size on model accuracy

In the previous sections, for all the models that we have built, we considered a batch size of 32. In this section, we will try to understand the impact of varying the batch size on accuracy.

Getting ready

To understand the reason batch size has an impact on model accuracy, let's contrast two scenarios where the total dataset size is 60,000:

  • Batch size is 30,000
  • Batch size is 32

When the batch size is large, the number of times of weight update per epoch is small, when compared to the scenario when the batch size is small.

The reason for a high number of weight updates per epoch when the batch size is small is that less data points are considered to calculate the loss value. This...

Building a deep neural network to improve network accuracy

Until now, we have looked at model architectures where the neural network has only one hidden layer between the input and the output layers. In this section, we will look at the neural network where there are multiple hidden layers (and hence a deep neural network), while reusing the same MNIST training and test dataset that were scaled.

Getting ready

A deep neural network means that there are multiple hidden layers connecting the input to the output layer. Multiple hidden layers ensure that the neural network learns a complex non-linear relation between the input and output, which a simple neural network cannot learn (due to a limited number of hidden layers).

A typical...

Varying the learning rate to improve network accuracy

So far, in the previous recipes, we used the default learning rate of the Adam optimizer, which is 0.0001.

In this section, we will manually set the learning rate to a higher number and see the impact of changing the learning rate on model accuracy, while reusing the same MNIST training and test dataset that were scaled in the previous recipes.

Getting ready

In the previous chapter on building feedforward neural networks, we learned that the learning rate is used in updating weights and the change in weight is proportional to the amount of loss reduction.

Additionally, a change in a weight's value is equal to the decrease in loss multiplied by the learning rate. Hence...

Varying the loss optimizer to improve network accuracy

So far, in the previous recipes, we considered the loss optimizer to be the Adam optimizer. However, there are multiple other variants of optimizers, and a change in the optimizer is likely to impact the speed with which the model learns to fit the input and the output.

In this recipe, we will understand the impact of changing the optimizer on model accuracy.

Getting ready

To understand the impact of varying the optimizer on network accuracy, let's contrast the scenario laid out in previous sections (which was the Adam optimizer) with using a stochastic gradient descent optimizer in this section, while reusing the same MNIST training and test datasets that were scaled...

Understanding the scenario of overfitting

In some of the previous recipes, we have noticed that the training accuracy is ~100%, while test accuracy is ~98%, which is a case of overfitting on top of a training dataset. Let's gain an intuition of the delta between the training and the test accuracies.

To understand the phenomenon resulting in overfitting, let's contrast two scenarios where we compare the training and test accuracies along with a histogram of the weights:

  • Model is run for five epochs
  • Model is run for 100 epochs

The comparison-of-accuracy metric between training and test datasets between the two scenarios is as follows:

Scenario

Training dataset

Test dataset

5 epochs

97.59%

97.1%

100 epochs

100%

98.28%

Once we plot the histogram of weights that are connecting the hidden layer to the output layer, we will notice that the 100...

Speeding up the training process using batch normalization

In the previous section on the scaling dataset, we learned that optimization is slow when the input data is not scaled (that is, it is not between zero and one).

The hidden layer value could be high in the following scenarios:

  • Input data values are high
  • Weight values are high
  • The multiplication of weight and input are high

Any of these scenarios can result in a large output value on the hidden layer.

Note that the hidden layer is the input layer to output layer. Hence, the phenomenon of high input values resulting in a slow optimization holds true when hidden layer values are large as well.

Batch normalization comes to the rescue in this scenario. We have already learned that, when input values are high, we perform scaling to reduce the input values. Additionally, we have learned that scaling can also be performed using...

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • From scratch, build multiple neural network architectures such as CNN, RNN, LSTM in Keras
  • Discover tips and tricks for designing a robust neural network to solve real-world problems
  • Graduate from understanding the working details of neural networks and master the art of fine-tuning them

Description

This book will take you from the basics of neural networks to advanced implementations of architectures using a recipe-based approach. We will learn about how neural networks work and the impact of various hyper parameters on a network's accuracy along with leveraging neural networks for structured and unstructured data. Later, we will learn how to classify and detect objects in images. We will also learn to use transfer learning for multiple applications, including a self-driving car using Convolutional Neural Networks. We will generate images while leveraging GANs and also by performing image encoding. Additionally, we will perform text analysis using word vector based techniques. Later, we will use Recurrent Neural Networks and LSTM to implement chatbot and Machine Translation systems. Finally, you will learn about transcribing images, audio, and generating captions and also use Deep Q-learning to build an agent that plays Space Invaders game. By the end of this book, you will have developed the skills to choose and customize multiple neural network architectures for various deep learning problems you might encounter.

Who is this book for?

This intermediate-level book targets beginners and intermediate-level machine learning practitioners and data scientists who have just started their journey with neural networks. This book is for those who are looking for resources to help them navigate through the various neural network architectures; you'll build multiple architectures, with concomitant case studies ordered by the complexity of the problem. A basic understanding of Python programming and a familiarity with basic machine learning are all you need to get started with this book.

What you will learn

  • Build multiple advanced neural network architectures from scratch
  • Explore transfer learning to perform object detection and classification
  • Build self-driving car applications using instance and semantic segmentation
  • Understand data encoding for image, text and recommender systems
  • Implement text analysis using sequence-to-sequence learning
  • Leverage a combination of CNN and RNN to perform end-to-end learning
  • Build agents to play games using deep Q-learning
Estimated delivery fee Deliver to Spain

Premium delivery 7 - 10 business days

€17.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Feb 28, 2019
Length: 568 pages
Edition : 1st
Language : English
ISBN-13 : 9781789346640
Category :
Languages :
Concepts :
Tools :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Estimated delivery fee Deliver to Spain

Premium delivery 7 - 10 business days

€17.95
(Includes tracking information)

Product Details

Publication date : Feb 28, 2019
Length: 568 pages
Edition : 1st
Language : English
ISBN-13 : 9781789346640
Category :
Languages :
Concepts :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 98.97
Neural Networks with Keras Cookbook
€32.99
Hands-On Neural Networks with Keras
€32.99
Neural Network Projects with Python
€32.99
Total 98.97 Stars icon

Table of Contents

17 Chapters
Building a Feedforward Neural Network Chevron down icon Chevron up icon
Building a Deep Feedforward Neural Network Chevron down icon Chevron up icon
Applications of Deep Feedforward Neural Networks Chevron down icon Chevron up icon
Building a Deep Convolutional Neural Network Chevron down icon Chevron up icon
Transfer Learning Chevron down icon Chevron up icon
Detecting and Localizing Objects in Images Chevron down icon Chevron up icon
Image Analysis Applications in Self-Driving Cars Chevron down icon Chevron up icon
Image Generation Chevron down icon Chevron up icon
Encoding Inputs Chevron down icon Chevron up icon
Text Analysis Using Word Vectors Chevron down icon Chevron up icon
Building a Recurrent Neural Network Chevron down icon Chevron up icon
Applications of a Many-to-One Architecture RNN Chevron down icon Chevron up icon
Sequence-to-Sequence Learning Chevron down icon Chevron up icon
End-to-End Learning Chevron down icon Chevron up icon
Audio Analysis Chevron down icon Chevron up icon
Reinforcement Learning Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Half star icon Empty star icon 3.3
(8 Ratings)
5 star 37.5%
4 star 12.5%
3 star 12.5%
2 star 12.5%
1 star 25%
Filter icon Filter
Top Reviews

Filter reviews by




krishna Sep 23, 2019
Full star icon Full star icon Full star icon Full star icon Full star icon 5
The Word embedding concepts were clear, application oriented and neatly explained.The book is structured appropriate to the readers learning curve. The author is reachable on linkedin and is has an amazing intuitive way of explaining complicated NN architectures. This book is a must buy if you are into deep learning applications.
Amazon Verified review Amazon
drew lubz Oct 25, 2019
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book is perfect for practitioners in the intermediate stage of machine learning.. The different examples help understanding how to create real word applications....the organization of it ('how to do it' sections) are really good..
Amazon Verified review Amazon
janga reddy Aug 31, 2019
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I have gone through multiple books in vain, trying to understand the working details of various neural networks architectures. This book does an amazing job of detailing the steps involved in building neural network architectures step by step.The structure of book is easy for the reader to follow, with a logical flow from one chapter to another and from one use case to another.The code is easy to follow with commentary about each line of code. However, there are a couple of use cases where in the book, the imported libraries are to be upgraded to have the code working – sufficient information has been provided in the code’s github repository.The combination of detail in book and the corresponding github code for use cases ranging across the spectrum makes this a MUST-HAVE book for anyone beginning with neural networks.
Amazon Verified review Amazon
JJG Jan 29, 2020
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
I love the content of this book, the best of many books I have purchased on the subject of machine learning. Want to give a 5, but there are many places where the formating just sucks. See my image for example.
Amazon Verified review Amazon
Melanie L. Jan 04, 2020
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
It might be a valuable book if you're looking for Keras examples and references. But it is hard to read and lacking details. (Yes, this is a cookbook not a deep dive book after all.)
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela