Exercises

1. Choosing learning rate is not a solved problem. There is no one “right” learning rate. What happens if you make the learning rate 10 times bigger? 10 times smaller? What if you start with a large learning rate and then reduce it after some training?
2. We used the same loss function and optimizer as the original code, which was first released back in 2018, but there are a lot of options now that weren’t available then. Try replacing the loss function with others from PyTorch’s extensive collection (https://pytorch.org/docs/stable/nn.html#loss-functions). Some of them will work without any change, but some won’t work for our situation at all. Try different ones, or even try combinations of loss functions!
3. We defined a model that downscaled from a 64x64 pixel image and re-created that same image. But with some tweaks, this same architecture can instead create a 128x128 or 256x256 pixel image. How would you make changes to the model...