Retraining
Sometimes, once you get a neural network that performs well, you job is done. Sometimes, however, you might want to retrain it on new samples, to get better precision (as your dataset is now bigger) or to get fresher results if your training dataset becomes obsolete relatively quickly.
In some cases, you might even want to retrain continuously, for example, every week, and have the new model automatically deployed in production.
In this case, it's critical that you have a strong procedure in place to verify the performance of your new model in the validation dataset and, hopefully, in a new, throwaway test dataset. It may also be advisable to keep a backup of all the models and try to find a way to monitor the performance in production, to quickly identify anomalies. In the case of a self-driving car, I expect a model to undergo rigorous automated and manual testing before being deployed in production, but other industries that don't have safety concerns...
