Answering questions from a document corpus in an abstractive manner

In the previous recipe, we learned how to build a QA system based on the document corpora. The answers that were retrieved were extractive in nature (i.e., the answer snippet was a piece of text copied verbatim from the document source). There are techniques to generate an abstractive answer too, which is more readable by end users compared to an extractive one.

Getting ready

For this recipe, we will build a QA system that will provide answers that are abstractive in nature. We will load the bilgeyucel/seven-wonders dataset from the Hugging Face site and initialize a retriever from it. This dataset has content about the seven wonders of the ancient world. To generate the answers, we will use the PromptNode component from the Haystack framework to set up a pipeline that can generate answers in an abstractive fashion. You can use the 9.4_abstractive_qa_on_document_corpus.ipynb notebook from the code site if you...