Introduction to PEFT

Before we start, let’s ask ourselves the following question: In the era of ChatGPT, we know that LLMs can solve many problems without the need for any additional updates or fine-tuning operations; so, do we still need fine-tuning operations?

Yes! We currently use models such as ChatGPT to efficiently solve general problems such as sentiment analysis, named-entity recognition (NER), and summarization. However, our industry or academia requires very specific natural language processing (NLP) tasks that are influenced by factors such as culture, domain, time, and geography. Studies (see Is ChatGPT a General-Purpose Natural Language Processing Task Solver?, Chengwei Qin et al.) have shown that fine-tuning a model outperforms ChatGPT-like language models. Therefore, it is better to fine-tune the model.

Additionally, it is possible to use relatively smaller models such as BERT or T5 by fine-tuning them for on-premises use within the company. This approach...