How to embed LLMs into applications: introducing AI orchestrators

In the previour paragraph, we saw that there are two main aspects to consider when incorporating Large Language Models within applications: a technical aspect and a conceptual aspect. While we can explain the conceptual aspect with the brand-new category of software called Copilot, in this paragraph we are going to further explore how to technically embed and orchestrate LLMs within our applications.From one side, the paradigm shift of foundation models implies a great simplification in the domain of AI-powered applications: from producing models, now the trend is consuming models. On the other side, many roadblocks might arise in developing with this new kind of AI, since there are LLMs-related components that are brand-new and never been managed before within an application lifecycle.The main components are:

• Models. The model is simply the type of LLM we decide to embed in our application. There are two main categories...