There are many other interesting deep learning models and architectures that are difficult to classify, other than the ones we mentioned in previous chapters, and at the same time, they are the new trends in deep learning and will have a huge impact in years to come. In NLP, BERT (which stands for Bidirectional Encoder Representations from Transformers) became the state-of-the-art language model (for more details, refer to the following paper, which was published by Google: https://arxiv.org/pdf/1810.04805.pdf). As for computer vision, GANs continue to gain popularity and improve. Their inventor, Ian Goodfellow, proposed Attention Generative Adversarial Networks for generating images in finer detail, which includes three new trends, as follows:
- Bayesian neural networks
- Capsule networks
- Meta-learning
