Language modeling
The goal of language models is to compute a probability of a sequence of words. They are crucial to a lot of different applications, such as speech recognition, optical character recognition, machine translation, and spelling correction. For example, in American English, the two phrases wreck a nice beach and recognize speech are almost identical in pronunciation, but their respective meanings are completely different from each other. A good language model can distinguish which phrase is most likely correct, based on the context of the conversation. This section will provide an overview of word- and character-level language models and how RNNs can be used to build them.
Word-based models
A word-based language model defines a probability distribution over sequences of words. Given a sequence of words of length m, it assigns a probability P(w 1 , ... , w m ) to the full sequence of words. The application of these probabilities are two-fold. We can use them to estimate the...
