Evaluating and selecting the right FM

Now that we’ve understood the different types of FMs available in Amazon Bedrock, how do we determine which one is best suited for our specific project needs? This section will help you learn how to evaluate the model fit for your use case.

The first step is to clearly define the problem you’re trying to solve or the use case you want to build. Get as specific as possible about the inputs, outputs, tasks involved, and any other requirements. With a well-defined use case in hand, you can research which models have demonstrated capabilities relevant to your needs. Narrowing the options upfront based on capabilities will streamline the evaluation process.

Once you’ve identified some potential candidate models, the next step is to examine their performance across standardized benchmarks and use cases. Amazon Bedrock provides a capability to evaluate FMs, also called model evaluation jobs. With model evaluation jobs, users...