Network Architecture Search – finding the most efficient network architecture
Neural Architecture Search (NAS) is the process of finding the best organization of the layers for the given problem. As the search space of the possible network architectures is extremely large, it is not feasible to evaluate every possible network architecture. Therefore, there is a need for a clever way to identify a promising network architecture and evaluate the candidates. Therefore, NAS methods are developed along three different aspects:
- Search space: How to construct a search space of a reasonable size
- Search strategy: How to explore the search space efficiently
- Performance estimation strategy: How to estimate the performance efficiently without training the model completely
Even though NAS is a fast-growing field of research, a few tools are available for TF and PyTorch models:
- Optuna (https://dzlab.github.io/dltips/en/tensorflow/hyperoptim-optuna)
- Syne...
