Defining an integration framework

Let’s define a framework to approach the integration paths through integration components. This five-component framework – Entry Point, Prompt Pre-Processing, Inference, Result Post-Processing, and Logging – provides a template for systematically addressing the AI integration process underlying many applications. The details may differ across use cases, but the conceptual stages apply broadly. Within this framework, we will establish a main boundary for integration depending on how users interact with the models: interactive for real-time output generation, or batch-oriented for bulk content creation and processing.

Figure 3.2: GenAI application integration framework

Integrating GenAI models can follow these two distinct paths – interactive user-driven approaches versus batch processing workflows. The interactive mode directly exposes model inference in real time through an application interface, where users...