Understanding how LlamaIndex uses prompts
In terms of mechanics, a RAG-based application follows exactly the same rules and principles of interaction that a simple user would use in a chat session with an LLM. A major difference comes from the fact that RAG is actually a kind of prompt engineer on steroids. Behind the scenes, for almost every indexing, retrieval, metadata extraction, or final response synthesis operation, the RAG framework programmatically produces prompts. These prompts are enriched with context and then sent to the LLM.
In LlamaIndex, for each type of operation that requires an LLM, there is a default prompt that is used as a template. Take TitleExtractor as an example. This is one of the metadata extractors that we already talked about in Chapter 4, Ingesting Data into Our RAG Workflow. The TitleExtractor class uses two predefined prompt templates to get titles from text nodes inside documents. It does this in two steps:
- It gets potential titles from...
