We will be predicting diabetes on a of patients by using a deep learning algorithm, which we will optimize with a grid search to find the optimal hyperparameters. We are going to be doing this project in Jupyter Notebook, as follows:

1. Start by opening up Command Prompt in Windows or Terminal in Linux systems. We will navigate to our project directory using the cd command.
2. Our next step is to open the Jupyter Notebook by typing the following command:

jupyter notebook

3. Once the Notebook is open, we will rename the unnamed file to Deep Learning Grid Search.
4. We will then import our packages using general import statements. We will print the version numbers, as shown in the following screenshot:

Keras has two options...

Our next step is to import the Pima Indians diabetes dataset, which contains the details of about 750 patients:

1. The dataset that we need can be found at https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv. We can import it by using the following line:

url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"

2. If we navigate to the preceding URL, we can see a lot of raw information. Once we have imported the dataset, we have to define column names. We will do this using the following lines of code:

names = ['n_pregnant', 'glucose_concentration', 'blood_pressure (mm Hg)', 'skin_thickness (mm)', 'serum_insulin (mu U/ml)', 'BMI', 'pedigree_function', 'age', 'class']

As can...

We'll now start building our Keras model, which is a deep learning algorithm:

1. The first thing that we're going to do is import the necessary packages and layers. We will do that by running the following lines of code:

from sklearn.model_selection import GridSearchCV, KFold
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.optimizers import Adam

In the preceding code snippet , GridSearchCV is the function we will use to perform a grid search, and KFold will be used for performing the k-fold cross-validation. KerasClassifier is used as the wrapper. Adam is the optimizer that we will be using for this model; the rest of the functions are all general functions used to define the model:

2. Let's start by defining the model. We will use a create_model() function, because...

It's now time to prepare our grid search algorithm. We will follow a step-by-step process to make it easier to understand and execute:

1. The first thing that we will do is copy the create_model() function, which we created in the Building a Keras model section, and paste it into a new cell, as shown in the following screenshot:

2. Now, we will define a random seed through NumPy. This helps us to create results that are reproducible. We are also going to add random initialization of weights and random divisions of data into different groups. We will set a starting point so that we have the same initialization and the same divisions for all the data. This can be done by adding a few lines of code above the create_model() function, as shown in the following screenshot:

3. Our next step is to initialize the KerasClassifier that we imported...

We will now use the information we gained in the Performing a grid search using scikit-learn section to optimize other aspects of our model. It looks like we might be overfitting the data a little bit, as we are getting better results on our training data than our testing data. We're now going to look at adding in dropout regularization:

1. Our first step is to copy the code that is present in the grid search cell that we ran in the previous section, and paste it in a fresh cell. We will keep the general structure of the code and play around with some of the parameters present.

2. We will then import the Dropout function from keras.layers using the following line:

from keras.layers import Dropout

3. We will now convert the learning rate into a variable by defining it in the Adam optimizer code block. We will use learn_rate as...

We're now going to optimize the weight initialization that we're applying to the end of each of these neurons:

1. To do this, we will first copy the code from the cell that we ran in the previous Reducing overfitting using dropout regularization section, and paste it into a new one. In this section, we won't be changing the general structure of the code; instead, we will be modifying some parameters and optimizing the search.
2. We now know the best learn_rate and dropout_rate, so we are going to hardcode these and remove them. We are also going to remove the Dropout layers that we added in the previous section. We will modify the learning rate of the Adam optimizer to 0.001, as this is the best value that we found.
3. Since we are trying to optimize the activation and init variables, we will define them in the create_model() function...

Let's now move on to tuning the number of neurons in each of these layers. Since we are following the same steps as the preceding sections, we will go through all these steps and do a recap with the code snippet at the end. So, let's get started with the following steps:

1. We will start by copying the code from the cell used in the Finding the optimal hyperparameters section, and paste it in a new cell. In this new cell, we will play around with the number of neurons by modifying some of the variables.
2. We will convert the total number of neurons present in each hidden layer into variables, such as neuron1 and neuron2. We will also define these variables in the create_model() function, so that they are called every time we execute it.
3. We will also change the kernel_initializer and activation values to tanh and normal, since those were the...

We know now some optimal hyperparameters for our grid search. We will use these to predict the onset of diabetes for the patients in our dataset. To do this, we will carry out the following steps:

1. We will predict whether diabetes will occur for every example in the dataset by using the predict() function, as shown in the following code snippet:

# generate predictions with optimal hyperparameters
y_pred = grid.predict(X_standardized)

2. We will then use the .shape command to see what the predictions look like. The following screenshot shows the output for this step:

From the preceding screenshot, we can see that there are 392 predictions with a numerical value for each.

3. Let's print off the first five and see what they look like. We get the following output:

4. We are now going to do a classification report and get an...

In this chapter, we built a deep neural network in Keras and we found the optimal hyperparameters using the scikit-learn grid search. We also learned how to optimize a network by tuning the hyperparameters. Note that the results that we get might not be the same for all of us, but as long as we get similar predictions, we can consider our model a success. When you start training on new data, or if you're trying to address a different problem with a different dataset, you will have to go through this process again. In this chapter, we also learned about deep learning and hyperparameter optimization and explored how to apply them to the network to predict the onset of diabetes on a huge dataset of patients.

In the next chapter, we will look at how to classify DNA using machine algorithms.