Summary
Tools on both ends were covered: analytic tools, such as RAGAs with metrics (sorry for the math!), and usability tools to monitor ongoing improvements from the customer’s perspective. This and monitoring conversations will flow into refinements needed for RAG materials, instructions/prompts, fine-tuning, and swapping to newer models while improving integrations with services and APIs. To do this well, this needs to fit into a process that can handle the dynamic nature of LLMs.
The next chapter offers insights into implementing processes conducive to LLM development and integrating with engineering to improve LLM solutions continuously.
