Setting up an LLM application – API-based closed source models
When looking to employ models in general and LLMs in particular, there are various design choices and trade-offs. One key choice is whether to host a model locally in your local environment or to employ it remotely, accessing it via a communication channel. Local development environments would be wherever your code runs, whether that’s your personal computer, your on-premises server, your cloud environment, and so on. The choice you make will impact many aspects, such as cost, information security, maintenance needs, network overload, and inference speed.
In this section, we will introduce a quick and simple approach to employing an LLM remotely via an API. This approach is quick and simple as it rids us of the need to allocate unusual computation resources to host the LLM locally. An LLM typically requires amounts of memory and computation resources that aren’t common in personal environments.
...
