Early LLMs

There are many LLMs today and they can be put into a family tree; see Figure 1.1. The figure shows the evolution from word2vec to the most advanced LLMs in 2023: GPT-4 and Gemini [Bard].

Figure 1.1: Family tree of LLMs from word2vec to GPT-4 and Bard, from Yang2023 with permission

So, that’s all of them but, for now, we’ll look at the earlier LLMs that lead to the most advanced technologies today. We’ll start with GPT.

GPT lineage

The development of GPT is a constantly changing and iterative process, with each new model building upon the strengths and weaknesses of its ancestors. The GPT series, initiated by OpenAI, has undergone a great deal of evolution, leading to advancements in natural language processing (NLP) and understanding.

GPT-3, the third iteration, brought a significant leap in terms of size and complexity, with an impressive 175 billion parameters. This allowed it to generate pretty human-like text...