Red-teaming, auditing, and reporting
The notion of red-teaming has been around for quite some time, from warfare and religious contexts to more recent computer systems and software and, now, generative AI/LLMs.
Red-teaming is generally described as a proactive methodology to determine the possible vulnerabilities within a system/environment by purposefully attacking the system with known threats. Subsequently, these attacks and threats are analyzed to better understand what exploits are possible for a potentially compromising system. In warfare, the enemy was described as the “red team” or the initiators of an attack, and the “blue team” thwarted such attacks.
As per the White House Executive Order on the safe and secure use of AI, the term “AI red-teaming” means a structured testing effort to find flaws and vulnerabilities in an AI system, often in a controlled environment and in collaboration with developers of AI. Artificial Intelligence...
