More Variants of RNNs
We've seen quite a few variations of RNNs in this chapter – covering all the prominent ones and the major upcoming (in terms of popularity) variations. Sequence modeling and its associated architectures are a hot area of research, and we see plenty of developments coming in every year. Many variants aim to make lighter models with fewer parameters that aren't as hardware hungry as current RNNs. Clockwork RNNs (CWRNNs) are a recent development and show great success. There are also Hierarchal Attention Networks, built on the idea of attention, but ultimately also propose that you shouldn't use RNNs as building blocks. There's a lot going on in this exciting area, so keep your eyes and ears open for the next big idea.
Activity 6.01: Sentiment Analysis of Amazon Product Reviews
So far, we've looked at the variants of RNNs and used them to predict sentiment on movie reviews from the IMDb dataset. In this activity, we will build...