Chapter 6: Decoding Images
Activity 17: Predict if an Image Is of a Cat or a Dog
Solution:
- If you look at the name of the images in the dataset, you will find that the images of dogs start with dog followed by '.' and then a number, for example – "dog.123.jpg". Similarly, the images of cats start with cat. So, let's create a function to get the label from the name of the file:
def get_label(file):
class_label = file.split('.')[0]
if class_label == 'dog': label_vector = [1,0]
elif class_label == 'cat': label_vector = [0,1]
return label_vector
Then, create a function to read, resize, and preprocess the images:
import os
import numpy as np
from PIL import Image
from tqdm import tqdm
from random import shuffle
SIZE = 50
def get_data():
data = []
files = os.listdir(PATH)
for image in tqdm(files):
label_vector = get_label(image)
img = Image.open(PATH + image).convert('L')
img = img.resize((SIZE,SIZE))
data.append([np.asarray(img),np.array(label_vector)])
shuffle(data)
return data
SIZE here refers to the dimension of the final square image we will input to the model. We resize the image to have the length and breadth equal to SIZE.
Note
When running os.listdir(PATH), you will find that all the images of cats come first, followed by images of dogs.
- To have the same distribution of both the classes in the training and testing sets, we will shuffle the data.
- Define the size of the image and read the data. Split the loaded data into training and testing sets:
data = get_data()
train = data[:7000]
test = data[7000:]
x_train = [data[0] for data in train]
y_train = [data[1] for data in train]
x_test = [data[0] for data in test]
y_test = [data[1] for data in test]
- Transform the lists to numpy arrays and reshape the images to a format that Keras will accept:
y_train = np.array(y_train)
y_test = np.array(y_test)
x_train = np.array(x_train).reshape(-1, SIZE, SIZE, 1)
x_test = np.array(x_test).reshape(-1, SIZE, SIZE, 1)
- Create a CNN model that makes use of regularization to perform training:
from keras.models import Sequential
from keras.layers import Dense, Dropout, Conv2D, MaxPool2D, Flatten, BatchNormalization
model = Sequential()
Add the convolutional layers:
model.add(Conv2D(48, (3, 3), activation='relu', padding='same', input_shape=(50,50,1)))
model.add(Conv2D(48, (3, 3), activation='relu'))
Add the pooling layer:
model.add(MaxPool2D(pool_size=(2, 2)))
- Add the batch normalization layer along with a dropout layer using the following code:
model.add(BatchNormalization())
model.add(Dropout(0.10))
- Flatten the 2D matrices into 1D vectors:
model.add(Flatten())
- Use dense layers as the final layers for the model:
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(2, activation='softmax'))
- Compile the model and then train it using the training data:
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics = ['accuracy'])
Define the number of epochs you want to train the model for:
EPOCHS = 10
model_details = model.fit(x_train, y_train,
batch_size = 128,
epochs = EPOCHS,
validation_data= (x_test, y_test),
verbose=1)
- Print the model's accuracy on the test set:
score = model.evaluate(x_test, y_test)
print("Accuracy: {0:.2f}%".format(score[1]*100))
Figure 6.39: Model accuracy on the test set
- Print the model's accuracy on the training set:
score = model.evaluate(x_train, y_train)
print("Accuracy: {0:.2f}%".format(score[1]*100))
Figure 6.40: Model accuracy on the train set
The test set accuracy for this model is 70.4%. The training set accuracy is really high, at 96%. This means that the model has started to overfit. Improving the model to get the best possible accuracy is left for you as an exercise. You can plot the incorrectly predicted images using the code from previous exercises to get a sense of how well the model performs:
import matplotlib.pyplot as plt
y_pred = model.predict(x_test)
incorrect_indices = np.nonzero(np.argmax(y_pred,axis=1) != np.argmax(y_test,axis=1))[0]
labels = ['dog', 'cat']
image = 5
plt.imshow(x_test[incorrect_indices[image]].reshape(50,50), cmap=plt.get_cmap('gray'))
plt.show()
print("Prediction: {0}".format(labels[np.argmax(y_pred[incorrect_indices[image]])]))
Figure 6.41: Incorrect prediction of a dog by the regularized CNN model
Activity 18: Identifying and Augmenting an Image
Solution:
- Create functions to get the images and the labels of the dataset:
from PIL import Image
def get_input(file):
return Image.open(PATH+file)
def get_output(file):
class_label = file.split('.')[0]
if class_label == 'dog': label_vector = [1,0]
elif class_label == 'cat': label_vector = [0,1]
return label_vector
- Create functions to preprocess and augment images:
SIZE = 50
def preprocess_input(image):
# Data preprocessing
image = image.convert('L')
image = image.resize((SIZE,SIZE))
# Data augmentation
random_vertical_shift(image, shift=0.2)
random_horizontal_shift(image, shift=0.2)
random_rotate(image, rot_range=45)
random_horizontal_flip(image)
return np.array(image).reshape(SIZE,SIZE,1)
- Implement the augmentation functions to randomly execute the augmentation when passed an image and return the image with the result.
This is for horizontal flip:
import random
def random_horizontal_flip(image):
toss = random.randint(1, 2)
if toss == 1:
return image.transpose(Image.FLIP_LEFT_RIGHT)
else:
return image
This is for rotation:
def random_rotate(image, rot_range):
value = random.randint(-rot_range,rot_range)
return image.rotate(value)
This is for image shift:
import PIL
def random_horizontal_shift(image, shift):
width, height = image.size
rand_shift = random.randint(0,shift*width)
image = PIL.ImageChops.offset(image, rand_shift, 0)
image.paste((0), (0, 0, rand_shift, height))
return image
def random_vertical_shift(image, shift):
width, height = image.size
rand_shift = random.randint(0,shift*height)
image = PIL.ImageChops.offset(image, 0, rand_shift)
image.paste((0), (0, 0, width, rand_shift))
return image
- Finally, create the generator that will generate images batches to be used to train the model:
import numpy as np
def custom_image_generator(images, batch_size = 128):
while True:
# Randomly select images for the batch
batch_images = np.random.choice(images, size = batch_size)
batch_input = []
batch_output = []
# Read image, perform preprocessing and get labels
for file in batch_images:
# Function that reads and returns the image
input_image = get_input(file)
# Function that gets the label of the image
label = get_output(file)
# Function that pre-processes and augments the image
image = preprocess_input(input_image)
batch_input.append(image)
batch_output.append(label)
batch_x = np.array(batch_input)
batch_y = np.array(batch_output)
# Return a tuple of (images,labels) to feed the network
yield(batch_x, batch_y)
- Create functions to load the test dataset's images and labels:
def get_label(file):
class_label = file.split('.')[0]
if class_label == 'dog': label_vector = [1,0]
elif class_label == 'cat': label_vector = [0,1]
return label_vector
This get_data function is similar to the one we used in Activity 1. The modification here is that we get the list of images to be read as an input parameter, and we return a tuple of images and their labels:
def get_data(files):
data_image = []
labels = []
for image in tqdm(files):
label_vector = get_label(image)
img = Image.open(PATH + image).convert('L')
img = img.resize((SIZE,SIZE))
labels.append(label_vector)
data_image.append(np.asarray(img).reshape(SIZE,SIZE,1))
data_x = np.array(data_image)
data_y = np.array(labels)
return (data_x, data_y)
- Now, create the test train split and load the test dataset:
import os
files = os.listdir(PATH)
random.shuffle(files)
train = files[:7000]
test = files[7000:]
validation_data = get_data(test)
- Create the model and perform training:
from keras.models import Sequential
model = Sequential()
Add the convolutional layers
from keras.layers import Input, Dense, Dropout, Conv2D, MaxPool2D, Flatten, BatchNormalization
model.add(Conv2D(32, (3, 3), activation='relu', padding='same', input_shape=(50,50,1)))
model.add(Conv2D(32, (3, 3), activation='relu'))
Add the pooling layer:
model.add(MaxPool2D(pool_size=(2, 2)))
- Add the batch normalization layer along with a dropout layer:
model.add(BatchNormalization())
model.add(Dropout(0.10))
- Flatten the 2D matrices into 1D vectors:
model.add(Flatten())
- Use dense layers as the final layers for the model:
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(2, activation='softmax'))
- Compile the model and train it using the generator that you created:
EPOCHS = 10
BATCH_SIZE = 128
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics = ['accuracy'])
model_details = model.fit_generator(custom_image_generator(train, batch_size = BATCH_SIZE),
steps_per_epoch = len(train) // BATCH_SIZE,
epochs = EPOCHS,
validation_data= validation_data,
verbose=1)
The test set accuracy for this model is 72.6%, which is an improvement on the model in Activity 21. You will observe that the training accuracy is really high, at 98%. This means that this model has started to overfit, much like the one in Activity 21. This could be due to a lack of data augmentation. Try changing the data augmentation parameters to see if there is any change in accuracy. Alternatively, you can modify the architecture of the neural network to get better results. You can plot the incorrectly predicted images to get a sense of how well the model performs.
import matplotlib.pyplot as plt
y_pred = model.predict(validation_data[0])
incorrect_indices = np.nonzero(np.argmax(y_pred,axis=1) != np.argmax(validation_data[1],axis=1))[0]
labels = ['dog', 'cat']
image = 7
plt.imshow(validation_data[0][incorrect_indices[image]].reshape(50,50), cmap=plt.get_cmap('gray'))
plt.show()
print("Prediction: {0}".format(labels[np.argmax(y_pred[incorrect_indices[image]])]))
