Penetration into other research fields

Next we will discuss three different areas, which have blended with NLP to produce some interesting machine learning tasks. We will be discussing three specific areas:

• NLP and computer vision

• NLP and reinforcement learning

• NLP and generative adversarial networks

Combining NLP with computer vision

First we will discuss two applications where NLP is combined with various computer vision applications to process multimodal data (that is, images and text).

Visual Question Answering (VQA)

VQA is a novel research area, where the focus is to produce an answer to a textual question about an image. For example, consider these questions about Figure 11.5:

Q1: What color is the sofa?

Q2: How many black chairs are there?

Figure 11.5: The image about which we've asked questions

With this type of information provided to the system, the system should output the following (preferably):

Answer Q1: The color of the sofa is black

Answer Q2: There are two black chairs in the room

The...