In classic regression analysis, we use a linear model to learn the relationship between a set of independent variables and a dependent variable. In finding this relationship, we expect to be able to predict the value of the dependent variable given the values of the independent variables.

A second important reason to do regression analysis is to understand the impact a single independent variable has on the dependent variable when all other independent variables are held constant. One of the great things about traditional multiple linear regression is the ceteris paribus property of linear models. We can interpret the impact a single independent variable has on the dependent variable without consideration to the other independent variable by using the learned weight associated with that independent variable. This type of interpretation...

Now that you hopefully understand why you would (and would not) want to use deep neural networks for regression, I'll show you how to do it. While it's not quite as simple as using linear regressor in scikit-learn, I think you'll find it quite easy using Keras. Most importantly, Keras will allow you to quickly iterate through model architectures without changing a lot of code.

When building a new neural network, I recommend following the same basic steps every time.

The following are the steps...

Keras uses an instance of a model object to contain a neural network. For those of you familiar with scikit-learn, this is probably quite familiar. What is somewhat different is that Keras models contain a set of layers. This set of layers needs to be defined by us. This allows for amazing flexibility in network architecture, with very little code.

Our MLP will need an input layer, a hidden layer, and an output layer.

Since we've already identified...

Changing our model is as easy as redefining our previous build_network() function. Our input layer will stay the same because our input hasn't changed. Likewise, the output layer should remain the same.

I'm going to add parameters to our network by adding additional hidden layers. I hope that by adding these hidden layers, our network can learn more complicated relationships between the input and output. I am going to start by adding four additional hidden layers; the first three will have 32 neurons and the fourth will have 16. Here's what it will look like:

And here's the associated code for building the model in Keras:

def build_network(input_features=None):
    inputs = Input(shape=(input_features,), name="input")
    x = Dense(32, activation='relu', name="hidden1")(inputs)
    x = Dense...

It's unlikely that you'll train a deep neural network and then apply it in the same script. Most likely, you will want to train your network and then save the structure and weights so that they can be used in a production-facing application designed to score new data. To do so, you'll need to be able to save and load your models.

Saving a model in Keras is very straightforward. You can use the model instance's .save() method to save the network structure and weights to an hdf5 file, as shown in the following code:

model.save("regression_model.h5")

That's really all there is to it. Loading a model from disk is just as simple. The code for doing this is given here for your reference:

from keras.models import load_model
model = load_model("regression_model.h5")

When you think about deep learning, you probably think about impressively complex computer vision problems, but deep neural networks can prove useful even for simple regression problems like this one. Hopefully, I've demonstrated that, while also introducing the Keras syntax and showing you how to build a very simple network.

As we continue, we will encounter much more complexity. Bigger networks, more complicated cost functions, and highly dimensional input data. However, the process I used in this chapter will remain same for the most part. In each case, we will outline the problem, identify the inputs and outputs, choose a cost function, create a network architecture, and finally train and tune our model.

Bias and variance can often be manipulated and reduced independently in deep neural networks if the following factors are taken care of:

• Bias: This can be reduced...