The process to find maxima or minima is based on constraints. The choice of optimization algorithm for your deep learning model can mean the difference between good results in minutes, hours, and days.
Optimization sits at the center of deep learning. Most learning problems reduce to optimization problems. Let's imagine we are solving a problem for some set of data. Using this pre-processed data, we train a model by solving an optimization problem, which optimizes the weights of the model with regards to the chosen loss function and some regularization function.
Hyper parameters of a model play a significant role in the efficient training of a model. Therefore, it is essential to use the different optimization strategies and algorithms to measure appropriate and optimum values of model's hyper parameters, which affect our Model's learning process...
