Introducing AI orchestrators to embed LLMs into applications
Earlier in this chapter, we saw that there are two main aspects to consider when incorporating LLMs within applications: a technical aspect and a conceptual aspect. While we can explain the conceptual aspect with the brand-new category of software called Copilot, in this section, we are going to further explore how to technically embed and orchestrate LLMs within our applications.
The main components of AI orchestrators
From one side, the paradigm shift of foundation models implies a great simplification in the domain of AI-powered applications: after producing models, now the trend is consuming models. On the other side, many roadblocks might arise in developing this new kind of AI, since there are LLM-related components that are brand new and have never been managed before within an application life cycle. For example, there might be malicious actors that could try to change the LLM instructions (the system message...
