Auto-BIG-bench

Will AI soon be able to evaluate itself? Let’s take a step forward into the future and see what is most probably coming.

Open Auto-BIG-bench.ipynb from this chapter’s folder in the repository. The program will feed GPT-4 a sample of 140+ BIG-bench tasks with a two-part prompt.

The first part contains the following instructions:

"1.Explain the following task
2.Provide an example
Solve it":

Note that the instructions do not require punctuation, only a whitespace.

The second part is the description of BIG-bench, for example:

Given a narrative, choose the most related proverb

GPT-4 will then:

1. Read the first part of the instructions.
2. Read the BIG-bench NLP task to be performed.
3. Create an example of the task.
4. Solve it.

This aspect is another step toward functional AGI. In the future, another AI model will probably evaluate and improve the response.

To illustrate this potential leap...