Modern-day LLMs are impressive, but they have never seen your company’s private data (hopefully!). This means the ability of an LLM to help your company fully utilize its data is very limited. This very large barrier has given rise to the concept of RAG, where you are using the power and capabilities of the LLM but combining it with the knowledge and data contained within your company’s internal data repositories. This is the primary motivation for using RAG: to make new data available to the LLM and significantly increase the value you can extract from that data.

Beyond internal data, RAG is also useful in cases where the LLM has not been trained on the data, even if it is public, such as the most recent research papers or articles about a topic that is strategic to your company. In both cases, we are talking about data that was not present during the training of the LLM. You can have the latest LLM trained on the most...

Now is as good a time as any to review some vocabulary that should help you become familiar with the various concepts in RAG. In the following subsections, we will familiarize ourselves with some of this vocabulary, including LLMs, prompting concepts, inference, context windows, fine-tuning approaches, vector databases, and vectors/embeddings. This is not an exhaustive list, but understanding these core concepts should help you understand everything else we will teach you about RAG in a more effective way.

LLM

Most of this book will deal with LLMs. LLMs are generative AI technologies that focus on generating text. We will keep things simple by concentrating on the type of model that most RAG pipelines use, the LLM. However, we would like to clarify that while we will focus primarily on LLMs, RAG can also be applied to other types of generative models, such as those for images, audio, and videos. We will focus on these other types of models and how they are used...

It could be argued that understanding vectors and all the ways they are used in RAG is the most important part of this entire book. As mentioned previously, vectors are simply the mathematical representations of your external data, and they are often referred to as embeddings. These representations capture semantic information in a format that can be processed by algorithms, facilitating tasks such as similarity search, which is a crucial step in the RAG process.

Vectors typically have a specific dimension based on how many numbers are represented by them. For example, this is a four-dimensional vector:

[0.123, 0.321, 0.312, 0.231]

If you didn’t know we were talking about vectors and you saw this in Python code, you might recognize this as a list of four floating points, and you aren’t too far off. However, when working with vectors in Python, you want to recognize them as a NumPy array, rather than lists. NumPy arrays are generally more machine-learning...

RAG is rapidly becoming a cornerstone of generative AI platforms in the corporate world. RAG combines the power of retrieving internal or new data with generative language models to enhance the quality and relevance of the generated text. This technique can be particularly useful for companies across various industries to improve their products, services, and operational efficiencies. The following are some examples of how RAG can be used:

• Customer support and chatbots: These can exist without RAG, but when integrated with RAG, it can connect those chatbots with past customer interactions, FAQs, support documents, and anything else that was specific to that customer.
• Technical support: With better access to customer history and information, RAG-enhanced chatbots can provide a significant improvement to current technical support chatbots.
• Automated reporting: RAG can assist in creating initial drafts or summarizing existing articles...

Conventional generative AI has already shown to be a revolutionary change for companies, helping their employees reach new levels of productivity. LLMs such as ChatGPT are assisting users with a rapidly growing list of applications that include writing business plans, writing and improving code, writing marketing copy, and even providing healthier recipes for a specific diet. Ultimately, much of what users are doing is getting done faster.

However, conventional generative AI does not know what it does not know. And that includes most of the internal data in your company. Can you imagine what you could do with all the benefits mentioned previously, but combined with all the data within your company – about everything your company has ever done, about your customers and all their interactions, or about all your products and services combined with a knowledge of what a specific customer’s needs are? You do not have to imagine...

LLMs can be adapted to your data in two ways:

• Fine-tuning: With fine-tuning, you are adjusting the weights and/or biases that define the model’s intelligence based on new training data. This directly impacts the model, permanently changing how it will interact with new inputs.
• Input/prompts: This is where you use the model, using the prompt/input to introduce new knowledge that the LLM can act upon.

Why not use fine-tuning in all situations? Once you have introduced the new knowledge, the LLM will always have it! It is also how the model was created – by being trained with data, right? That sounds right in theory, but in practice, fine-tuning has been more reliable in teaching a model specialized tasks (such as teaching a model how to converse in a certain way), and less reliable for factual recall.

The reason is complicated, but in general, a model’s knowledge of facts is like a human’s long-term...

The following are the stages of a RAG process from a user’s perspective:

1. A user enters a query/question.
2. The application thinks for a little while before checking the data it has access to so that it can see what is the most relevant.
3. The application provides a response that focuses on answering the user’s question, but using data that has been provided to it through the RAG pipeline.

From a technical standpoint, this captures two of the stages you will code: the retrieval and generation stages. But there is one other stage, known as indexing, which can be and is often executed before the user enters the query. With indexing, you are turning supporting data into vectors, storing them in a vector database, and likely optimizing the search functionality so that the retrieval step is as fast and effective as possible.

Once the user passes their query into the system, the following steps occur:

1. The user query...

In this chapter, we explored RAG and its ability to enhance the capabilities of LLMs by integrating them with an organization’s internal data. We learned how RAG combines the power of LLMs with a company’s private data, enabling the model to utilize information it was not originally trained on, making the LLM’s outputs more relevant and valuable for the specific organization. We also discussed the advantages of RAG, such as improved accuracy and relevance, customization to a company’s domain, flexibility in data sources used, and expansion of the model’s knowledge beyond its original training data. Additionally, we examined the challenges and limitations of RAG, including dependency on data quality, the need for data cleaning, added computational overhead and complexity, and the potential for information overload if not properly filtered.

Midway through this chapter, we defined key vocabulary terms and emphasized the critical importance...