Integrating MLSecOps with LLMOps
As discussed in Chapter 12, LLMs have significantly changed the scope of our security concerns by introducing the following:
- Non-deterministic outputs and the importance of input as a critical attack vector for prompt injections
- RAG, which expands the scope of input data significantly and introduces embeddings to model input preprocessing
- Large models cover generalist domains and have advanced capabilities and understanding. They require evaluation and benchmarking beyond the relatively simple tests against predictive AI models
- Predominantly external models and third-party hosting, which has shifted the operational focus to supply-chain validation and the previous three aspects
These differences have given rise to LLM Operations (LLMOps) to ensure that LLMs (owned or a third party) are robustly integrated into applications, providing reliable and efficient LLM services. LLMOps provide the backbone to operationalize LLM applications...
