Generative AI – A New Frontier

Up to this point, we have primarily covered how adversarial attacks affect predictive AI and discriminative models. These are designed to distinguish between different kinds of data. They are good at understanding, classifying, and predicting specific outcomes from input data. In this section, we will look at generative AI. This is a different strand of AI that generates entirely new data, such as text and images, using the data it was trained on. Both strands of AI share the same foundations.

The security topics we have discussed so far affect generative AI. However, generative AI’s different functions and outputs significantly change how adversarial AI is applied.

For example, evasion is no longer relevant because there is no expected outcome to evade. Instead, manipulating the models to create malicious content is now a prevalent attack on deployed AI targeting generative AI. Furthermore, as mentioned in previous chapters when discussing...