Exploring custom metrics and their applications
Base metrics are generally sufficient to meet the requirements of most use cases. However, custom metrics build upon base metrics and incorporate additional goals that are specific to a given scenario. It’s helpful to think of base metrics as a bachelor’s degree and custom metrics as a master’s or PhD degree. It’s perfectly fine to use only base metrics if they meet your needs and you don’t have any additional requirements.
Custom ideals often arise naturally early on in a project and are highly dependent on the specific use case. Most real use cases don’t expose their chosen metrics to the public, even when the prediction of the model is meant to be utilized publicly, such as Open AI’s ChatGPT. However, in machine learning competitions, companies with real use cases accompanied by data publish their chosen metric publicly to find the best model that can be built. In such a setting for...
