Detecting potential model fairness issues
Machine learning models can behave unfairly due to multiple reasons:
- Historical bias in society may be reflected in the data that was used to train the model.
- The decisions made by the developers of the model may have been skewed.
- Lack of representative data used to train the model. For example, there may be too few data points from a specific group of people.
Since it is hard to identify the actual reasons that cause the model to behave unfairly, the definition of a model behaving unfairly is defined by its impact on people. There are two significant types of harm that a model can cause:
- Allocation harm: This happens when the model withholds opportunities, resources, or information from a group of people. For example, during the hiring process or the loan lending example we have been working on so far, you may not have the opportunity to be hired or get a loan.
- Quality-of-service harm: This happens when the...
