One of the most fundamental skills required to get oneself setup with deep learning is a foundational understanding of linear algebra. Though linear algebra itself is a vast subject, and covering it in full is outside the scope of this book, we will go through some important aspects of linear algebra in this chapter. Hopefully, this will give you a sufficient understanding of some core concepts and how they interplay with deep learning methodologies.

In this section, we will look at core data structures and representations used most commonly across different linear algebra tasks. This is not meant to be a comprehensive list at all but only serves to highlight some of the prominent...

As the name suggests, deep learning involves learning a deeper representation of data, which requires large amounts of computational power. Such massive computational power is usually not possible with modern day CPUs. GPUs, on the other hand, lend themselves very nicely to this task. GPUs were originally designed for rendering graphics in real time. The design of a typical GPU allows for the disproportionately larger number of arithmetic logical unit (ALU), which allows them to crunch a large number of calculations in real time.

GPUs used for general purpose computation have a high data parallel architecture, which means they can process a large number of data points in parallel, leading to higher computational throughput. Each GPU is composed of thousands of cores. Each of such cores consists of a number of functional units which contain cache and ALU...

Every good deep learning application needs to have several components to be able to function correctly. These include:

• A model layer which allows a developer to design his or her own model with more flexibility
• A GPU layer that makes it seamless for application developers to choose between GPU/CPU for its application
• A parallelization layer that can allow the developer to scale his or her application to run on multiple devices or instances

As you can imagine, implementing these modules is not easy. Often a developer needs to spend more time on debugging implementation issues rather than the legitimate model issues. Thankfully, a number of software frameworks exist in the industry today which make deep learning application development practically the first class of its programming language.

These frameworks vary in architecture, design, and feature...

In this section, we will show two different ways of setting up a deep learning system using Amazon Web Services (AWS).

In this section, we will illustrate how to set up a deep learning environment on an AWS EC2 GPU instance g2.2xlarge running Ubuntu Server 16.04 LTS. For this example, we will use a pre-baked Amazon Machine Image (AMI) which already has a number of software packages installed—making it easier to set up an end-end deep learning system. We will use a publicly available AMI Image ami-b03ffedf, which has following pre-installed packages:

• CUDA 8.0
• Anaconda 4.20 with Python 3.0
• Keras / Theano

1. The first step to setting up the system is to set up an AWS account...

In this chapter, we have summarized key concepts required to get started with the real-world implementation of deep learning systems. We described core concepts from linear algebra that are central to understanding the foundations of deep learning technology. We provide a hardware guide to deep learning by covering various aspects of GPU-based implementation and what is a right hardware choice for application developers. We outline a list of most popular deep learning software frameworks that exist today and provide a feature-level parity as well as a performance benchmark for them. Finally, we demonstrate how to set up a cloud-based deep learning application on AWS.

In the next chapter, we will introduce neural networks and outline a self-start module to understanding them in greater details.