Code lab 5.2 – Red team attack!
This code can be found in the CHAPTER5-2_SECURING_YOUR_KEYS.ipynb
file in the CHAPTER_05
directory of the GitHub repository.
Through our hands-on code lab, we will engage in an exciting red team versus blue team exercise, showcasing how LLMs can be both a vulnerability and a defense mechanism in the battle for RAG application security.
We will first take the role of red team and orchestrate a prompt probe on our RAG pipeline code. As mentioned earlier in this chapter, prompt probing is the initial step to gain insight into the internal prompts a RAG system is using to discover the system prompt(s) of a RAG application. The system prompt is the initial set of instructions or context provided to the LLM to guide its behavior and responses. By uncovering the system prompt, attackers can gain valuable insights into the inner workings of the application and this sets the foundation for designing more targeted and efficient attacks using the other...