Introducing DAVE-2

DAVE-2 is a system designed by Nvidia to train a neural network to drive a car, intended as a proof of concept to demonstrate that, in principle, a single neural network could be able to steer a car on a road. Putting it another way, our network could be trained to drive a real car on a real road, if enough data is provided. To give you an idea, Nvidia used around 72 hours of video, at 10 frames per second.

The idea is very simple: we feed the neural network a video stream, and the neural network will simply generate the steering angle, or something equivalent. The training is created by a human driver, and the system collects data from the camera (training data) and from the steering wheel moved by the pilot (training labels). This is called behavioral cloning because the network is trying to clone the behavior of the human driver.

Unfortunately, this would be a bit too simple, as most of the labels would simply be 0 (the driver going straight), so the network...