A self-driving car (also called an autonomous/automated vehicle or driverless car) is a robotic vehicle that is capable of traveling between destinations and navigating without human intervention. To enable autonomy, self-driving cars detect and interpret environments using a variety of techniques such as radar, GPS and computer vision; and they then plan appropriate navigational paths to the desired destination.

In more detail, the following is how self-driving cars work in general:

• The software plans the routes based on the destination, traffic, and road information and starts the car
• A Light Detection and Ranging (LiDAR) sensor captures the surroundings in real time and creates a dynamic 3D map
• Sensors monitor lateral movement to calculate the car's position on the 3D map
• Radar systems exploit information on distances...

As always, we begin by exploring the German Traffic Sign Recognition Benchmark (GTSRB) dataset at http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset.

The GTSRB dataset, compiled and generously published by the real-time computer vision research group in Institut für Neuroinformatik, was originally used for a competition of classifying single images of traffic signs. It consists of a training set of 39,209 labeled images and a testing test of 12,630 unlabeled images. The training dataset contains 43 classes—43 types of traffic signs. We will go through all classes and exhibit several samples for each class.

The dataset can...

We have been very fortunate so far to possess a large-enough training dataset with 75% of 39,209 samples. This is one of the reasons why we are able to achieve a 99.3% to 99.4% classification accuracy. However, in reality, obtaining a large training set is not easy in most supervised learning cases, where manual work is necessary or the cost of data collection and labeling is high. In our traffic signs classification project, can we still achieve the same performance if we are given a lot less training samples to begin with? Let's give it a shot.

We simulate a small training set with only 10% of the 39,209 samples and a testing set with the rest 90%:

> train_perc_1 = 0.1 
> train_index_1 <- createDataPartition(data.y, p=train_perc_1, list=FALSE) 
> train_index_1 <- train_index_1[sample(nrow(train_index_1...

Overfitting occurs when the model fits too well to the training set but is not able to generalize to unseen cases. For example, a CNN model recognizes specific traffic sign images in the training set instead of general patterns. It can be very dangerous if a self-driving car is not able to recognize sign images in ever-changing conditions, such as different weather, lighting, and angles different from what are presented in the training set. To recap, here's what we can do to reduce overfitting:

• Collecting more training data (if possible and feasible) in order to account for various input data.
• Using data augmentation, wherein we invent data in a smart way if time or cost does not allow us to collect more data.
• Employing dropout, which diminishes complex co-adaptations among neighboring neurons.
• Adding Lasso (L1) or/and...

We just accomplished our second computer vision project in this R and deep learning journey! Through this chapter, we got more familiar with convolutional neural networks and their implementation in MXNet, and another powerful deep learning tool: Keras with TensorFlow.

We started with what self-driving cars are and how deep learning techniques are making self-driving cars feasible and more reliable. We also discussed how deep learning stands out and becomes the state-of-the-art solution for object recognition in intelligent vehicles. After exploring the traffic sign dataset, we developed our first CNN model using MXNet and achieved more than 99% accuracy. Then we moved on to another powerful deep learning framework, Keras + TensorFlow, and obtained comparable results.

We introduced the dropout technique to reduce overfitting. We also learned how to deal with lack of training...