You're reading from Mastering PyTorch Build powerful neural network architectures using advanced PyTorch 1.x features

Product type Paperback

Published in Feb 2021

Publisher Packt

ISBN-13 9781789614381

Length 450 pages

Edition 1st Edition

Languages

Python

Tools

PyTorch

Concepts

Deep Learning

Author (1):

Ashish Ranjan Jha

View More author details

Table of Contents (20) Chapters

Preface

1. Section 1: PyTorch Overview

2. Chapter 1: Overview of Deep Learning using PyTorch FREE CHAPTER

3. Chapter 2: Combining CNNs and LSTMs

4. Section 2: Working with Advanced Neural Network Architectures

5. Chapter 3: Deep CNN Architectures

6. Chapter 4: Deep Recurrent Model Architectures

7. Chapter 5: Hybrid Advanced Models

8. Section 3: Generative Models and Deep Reinforcement Learning

9. Chapter 6: Music and Text Generation with PyTorch

10. Chapter 7: Neural Style Transfer

11. Chapter 8: Deep Convolutional GANs

12. Chapter 9: Deep Reinforcement Learning

13. Section 4: PyTorch in Production Systems

14. Chapter 10: Operationalizing PyTorch Models into Production

15. Chapter 11: Distributed Training

16. Chapter 12: PyTorch and AutoML

17. Chapter 13: PyTorch and Explainable AI

18. Chapter 14: Rapid Prototyping with PyTorch

19. Other Books You May Enjoy

Leave a review - let other readers know what you think

Using Optuna for hyperparameter search

Optuna is one of the hyperparameter search tools that supports PyTorch. You can read in detail about the search strategies used by the tool, such as TPE (Tree-Structured Parzen Estimation) and CMA-ES (Covariance Matrix Adaptation Evolution Strategy) in the Optuna paper, at https://arxiv.org/pdf/1907.10902.pdf. Besides the advanced hyperparameter search methodologies, the tool provides a sleek API, which we will explore in a moment.

Tool citation

Optuna: A Next-Generation Hyperparameter Optimization Framework.

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama (2019, in KDD).

In this section, we will once again build and train the MNIST model, this time using Optuna to figure out the optimal hyperparameter setting. We will discuss important parts of the code step by step, in the form of an exercise. The full code can be found here:

https://github.com/PacktPublishing/Mastering-PyTorch/blob/master/Chapter12...