Understanding bias

Detecting and mitigating bias is a crucial focus area for AI risk management. The presence of bias in ML models can expose an organization to potential legal risks but also lead to negative publicity, causing reputational damage and public relations issues. Specific laws and regulations, such as the Equal Credit Opportunity Act, also prohibit discrimination in business transactions, like credit transactions, based on race, skin color, religion, sex, nationality origin, marital status, and age. Some other examples of laws against discrimination include the Civil Rights Act of 1964 and Age Discrimination in Employment Act of 1967.

ML bias can result from the underlying prejudice in data. Since ML models are trained using data, if the data has a bias, then the trained model will also exhibit bias behaviors. For example, if you build an ML model to predict the loan default rate as part of the loan application review process, and you use race as one of the features...