Penetration into other research fields
Next we will discuss three different areas, which have blended with NLP to produce some interesting machine learning tasks. We will be discussing three specific areas:
NLP and computer vision
NLP and reinforcement learning
NLP and generative adversarial networks
Combining NLP with computer vision
First we will discuss two applications where NLP is combined with various computer vision applications to process multimodal data (that is, images and text).
Visual Question Answering (VQA)
VQA is a novel research area, where the focus is to produce an answer to a textual question about an image. For example, consider these questions about Figure 11.5:
Q1: What color is the sofa?
Q2: How many black chairs are there?
Figure 11.5: The image about which we've asked questions
With this type of information provided to the system, the system should output the following (preferably):
Answer Q1: The color of the sofa is black
Answer Q2: There are two black chairs in the room
The...
