In machine learning (ML), we try to automatically discover rules for mapping input data to a desired output. In this process, it's very important to create appropriate representations of data. For example, if we want to create an algorithm to classify an email as spam/ham, we need to represent the email data numerically. One simple representation could be a binary vector where each component depicts the presence or absence of a word from a predefined vocabulary of words. Also, these representations are task-dependent, that is, representations may vary according to the final task that we desire our ML algorithm to perform.

In the preceding email example, instead of identifying spam/ham if we want to detect sentiment in the email, a more useful representation of the data could be binary vectors where the predefined vocabulary consists of words with positive...

One of the primary reasons for the widespread popularity and adoption of deep learning is the Python deep learning ecosystem, which consists of easy-to-use open source deep learning frameworks. However, the deep learning landscape is rapidly changing, considering how new frameworks keep getting launched and older ones reach the end of their life. Deep learning enthusiasts might know that Theano was the first and most popular deep learning framework, created by MILA (https://mila.quebec/), which was headed by Yoshua Bengio. Unfortunately, it was recently announced that further development and support for Theano is ending after the launch of its latest version (1.0) in 2017. Hence, it is of paramount importance to understand what frameworks are out there that can be leveraged to implement and solve deep learning. Another point to remember here is that several...

Deep learning works quite well on a standard single PC setup with a CPU. However, once your datasets start increasing in size and your model architectures start getting more complex, you need to start thinking about investing in a robust deep learning environment. The major expectations being the system can build and train models efficiently, take less time to train models, and is fault tolerant. Most deep learning computations are essentially millions of matrix operations (data is represented as matrices) and enable fast computation in parallel; GPUs have been proven to work really well in this aspect. You can consider setting up a robust cloud-based deep learning environment or even an in-house environment. Let's look at how we can set up a robust cloud-based deep learning environment in this section.

The...

Often users or organizations may not want to leverage cloud services, especially if their data is sensitive, and so focus on building an on-premise deep learning environment. The major focus here should be to invest in the right type of hardware to enable maximum performance and leverage the right GPU for building deep learning models. With regards to hardware, special emphasis goes to the following:

• Processor: You can invest in an i5 or an i7 Intel CPU, or maybe an Intel Xeon if you are looking to spoil yourself!
• RAM: Invest in at least 32 GB of DDR4 or better RAM for your memory.
• Disk: A 1 TB hard disk is excellent, and also you can invest in a minimum of 128 GB or 256 GB of SSD for fast data access!
• GPU: Perhaps the most important component for deep learning. Invest in a NVIDIA GPU, anything above a...

Let's try to familiarize ourselves with some of the basic concepts behind neural networks that make all deep learning models tick!

A linear neuron is the most basic building block of a deep neural network. It can be schematically represented as shown in the following figure. Here, represents the input vector and w_is are the weights of the neuron. Given a training set consisting a of set of input, target value pairs, a linear neuron tries to learn a linear transformation that can map the input vectors to the corresponding target value. Basically, a linear neuron approximates the input output relationship by a linear function :

We covered a lot of ground in this chapter as regards the fundamentals of deep learning. We really commend your efforts on getting this far! The idea of this chapter was to introduce you to the core concepts and terminology pertaining to the domain of deep learning. We started with a brief introduction of deep learning and then looked at the popular frameworks in today's deep learning landscape. Detailed step-by-step guides have also been included for setting up your own deep learning environments to develop and train large-scale deep learning models on GPUs.

Finally, we covered essential concepts around neural networks including linear and non-linear neurons, data representation, chain rule, loss functions, multilayer networks, and SGD. The challenges of learning in neural networks were also covered, including popular caveats surrounding local minima and exploding...