Prompting large language models

I’ve said this before: I am a huge fan and big advocate of Hugging Face. I’ve learned a lot about natural language processing (NLP) from and with them, so I’d be remiss if I didn’t call out their book as a great source for prompt engineering tips and techniques. (10) Most of those practices center around picking the right hyperparameters for your model, with each type of model offering slightly different results.

However, I would argue that the rise of ChatGPT has now almost completely thrown that out of consideration. In today’s world, the extremely accurate performance of OpenAI’s model raises the bar for all NLP developers, pushing us to deliver comparable results. For better or worse, there is no going back. Let’s try to understand how to prompt our large language models (LLMs)! We’ll start with instruction fine-tuning.

Instruction fine-tuning

First, it’s helpful to really understand...