Implementing a continuously improving system
Let’s implement monitoring and continuous improvement for our web page Q&A application powered by the LLM we’ve trained in previous chapters. Initially, the model provided basic answers to frequently asked questions but struggled with more nuanced queries and user-specific issues. Let’s improve this by incorporating a continuous improvement journey, integrating robust human feedback mechanisms, and closely monitoring performance metrics to refine the model iteratively.
Metrics used and performance improvements observed
When we began, the model’s accuracy in delivering correct answers was around 70%. With continuous feedback and iterative training, we’ve seen substantial gains, with accuracy improving to 92%. Similarly, precision, which gauges the relevance of the model’s answers to posed questions, has improved significantly from 65% to 90%. Furthermore, user satisfaction, as measured through direct feedback forms embedded within the plugin, has risen from an average rating of 3.5 out of 5 to 4.8 out of 5. Here’s the partial pipeline we used to fine-tune the model using Quantization and Low-Rank Adapters (QLoRA) and LangChain, incorporating detailed mechanisms for obtaining user feedback:
# Live user feedback is collected on the UI. Users and internal human-in-the-loop teams contribute feedback. user_feedback = collect_user_feedback_for_query_id(query_id) query = extract_query_from_query_id(query_id) response = extract_response_from_query_id(query_id) # Incorporate feedback into the model's training regimen qlora_model.update(query, response) performance_metrics = qlora_model.evaluate(query)
In this pipeline, we added a collect_user_feedback function that fetches the collection of real-time feedback from users interacting with the model. This feedback is typically obtained through user interfaces that include mechanisms such as a Satisfaction button, textboxes, or a rating system where users can indicate their happiness with the model’s response. This direct user feedback is invaluable as it provides immediate insights into user satisfaction and areas needing improvement. Here’s our Azure infrastructure for continuously collecting and monitoring these metrics:
- Azure Monitor: This service is fundamental for collecting, analyzing, and acting on telemetry data from our cloud environments. Azure Monitor allows us to track application health, performance, and other custom metrics crucial for the LLM’s operation.
- Application Insights: Integrated with Azure Monitor, Application Insights provides deeper analytics on application performance and user behavior. It helps in understanding dependencies, tracking exceptions, and profiling performance bottlenecks. This service is instrumental in collecting detailed performance metrics and logs, ensuring our models perform as expected.
- Log Analytics: As part of Azure Monitor, Log Analytics processes and queries vast amounts of operational data collected, including logs and performance metrics. We use queries to extract insights from data, which helps in proactive decision-making and continuous performance improvement.
- Azure Data Explorer: For more complex analytical requirements, Azure Data Explorer allows us to perform real-time analysis on large volumes of data. It’s particularly useful for identifying patterns, anomalies, and trends across the metrics we collect, enabling us to refine our models continuously.
- Azure Automation: This service automates repetitive, manual tasks involved in managing and monitoring our LLMs. Azure Automation helps us ensure compliance with policies, manage resource deployment, and handle fault remediation, all crucial for maintaining system integrity and performance.
Additionally, internal “human-in-the-loop” feedback processes involve expert reviewers who periodically assess the model’s outputs and provide detailed corrections and suggestions. This form of feedback is especially useful for handling more complex queries or when the model encounters new types of questions, ensuring that the training data remains robust and the model’s accuracy continuously improves. This combination of live user feedback and expert review forms a comprehensive feedback mechanism that drives the ongoing refinement of the model.
For the next optimization, we focused on the response time, which initially averaged 8 seconds per query. This has been reduced to just 1 second, dramatically enhancing the user experience by providing quicker answers. To accomplish this, we used a 7B version of the foundation model (FM) compared to the 13B-parameter Llama2 model. Additionally, we used 16-bit float model parameters instead of 32-bit parameters. Finally, we leveraged the Open Neural Network Exchange (ONNX) runtime to speed up Llama2 inference speeds. All these changes resulted in nearly an 8x speedup. Here’s the code that helped most:
convert_graph_to_onnx.convert(framework="pt", model=model,
tokenizer=tokenizer, output=onnx_path, opset=13,
use_external_format=False,
pipeline_name='feature-extraction',
precision="float16")
providers = [
('CUDAExecutionProvider', {
'device_id': 0,
'arena_extend_strategy': 'kNextPowerOfTwo',
'gpu_mem_limit': 2 * 1024**3,
'cudnn_conv_algo_search': 'EXHAUSTIVE',
'do_copy_in_default_stream': True,
}) if ort.get_device() == 'GPU' else 'CPUExecutionProvider'
]
session = ort.InferenceSession(onnx_path, providers=providers)
inputs = tokenizer(input_text, return_tensors="np")
start_time = time.time()
outputs = session.run(None, {k: v for k, v in inputs.items()})
latency = time.time() - start_time The preceding code creates an ONNX session from our fine-tuned Llama2 model, specifies a 16-bit float, and measures the latency. This speedup really helped the engagement metrics; the ignore rate on the tool has decreased by 40%, and the time spent on the tool has increased by 50%, indicating higher engagement and greater content relevance.
From the implementation of this Q&A application, we’ve learned several lessons that can provide valuable insights for LLM projects, each contributing to a deeper understanding of effective model management:
- Start small and scale gradually: Our experience emphasizes the importance of beginning with a manageable scope and complexity. This approach allows for more controlled testing and refinement, helping to identify core areas that benefit most from the LLM application. Understanding these areas thoroughly before scaling to broader use cases ensures a solid foundation for expansion and prevents overextension.
- Incorporate diverse feedback early: One of the critical strategies we adopted was engaging a diverse group of users in the feedback process from the early stages. This diversity captures a wide range of use cases and linguistic nuances, enhancing the model’s robustness against varied inputs and ensuring it meets a broad spectrum of user needs.
- Monitor continuously: Continuous monitoring has been critical in maintaining the model’s effectiveness. By implementing real-time monitoring tools to track performance metrics, we could quickly detect issues and make immediate adjustments. This responsiveness is important in keeping the model performing at its best in a dynamic environment.
- Emphasize data privacy in feedback collection: It’s imperative to ensure that feedback collection processes comply with data privacy laws and ethical standards, especially when handling user-generated content. Safeguarding user data protects privacy and builds trust in the application, enhancing user engagement.
- Iterate on feedback mechanisms: As the model evolves, mechanisms for collecting and integrating feedback must adapt accordingly. Periodic reviews of these systems are necessary to ensure they remain effective and efficient. This iterative approach to feedback integration helps keep the model current and responsive to new challenges and opportunities.
These lessons and practical advice underscore the dynamic and iterative nature of managing LLMs, highlighting the importance of adaptability and rigorous process management in achieving long-term success and relevance.
