Summary

This chapter’s goal was to show that as we accumulate RAG data, some data is dynamic and requires constant updates, and as such, cannot be fine-tuned easily. However, some data is static, meaning that it will remain stable for long periods of time. This data can become parametric (stored in the weights of a trained LLM).

We first downloaded and processed the SciQ dataset, which contains hard science questions. This stable data perfectly suits fine-tuning. It contains a question, answer, and support (explanation) structure, which makes the data effective for fine-tuning. Also, we can assume human feedback was required. We can even go as far as imagining this feedback could be provided by analyzing generative AI model outputs.

We converted the data we prepared into prompts and completions in a JSONL file following the recommendations of OpenAI’s preparation tool. The structure of JSONL was meant to be compatible with a completion model (prompt and completion...