Key concepts/terms used in LLMs
LLMs are a complex field of NLP, and there are several terms associated with them.
Some key terms and concepts used in the context of LLMs are the following:
- Transformer architecture: The foundational architecture for most LLMs, known for its self-attention mechanism, which allows the model to weigh the importance of different words in a sentence.
- Pre-training: The initial phase in which the LLM is trained on a massive corpus of text data from the internet to learn language patterns and context. This pre-trained model is often referred to as the “base model.”
- Fine-tuning: The subsequent phase where the pre-trained model is adapted to perform specific NLP tasks, such as text classification, translation, summarization, or question answering. Fine-tuning helps the model specialize in these tasks.
- Parameters: These are the trainable components of the LLM, represented by numerical values. The number of parameters is a...
