Methodology

Question-answering is mostly presented as an NLP exercise involving a transformer and a dataset that contains the ready-to-ask questions and provides the answers to those questions. The transformer is trained to answer the questions asked in this closed environment.

However, in more complex situations, reliable transformer model implementations require customized methods.

Transformers and methods

A perfect and efficient universal transformer model for question-answering or any other NLP task does not exist. The best model for a project is the one that produces the best outputs for a specific dataset and task.

Chapter 6, Text Generation with OpenAI GPT-2 and GPT-3 Models, showed that the Pattern-Exploiting Training (PET) method applied to a small ALBERT model exceeded the performance of the much larger GPT-3 model.

The method outperforms models in many cases. A suitable method with an average model often will produce more efficient results than a flawed...