Chapter 9. Fighting Bias

We like to think that machines are more rational than us: heartless silicon applying cold logic. Thus, when computer science introduced automated decision making into the economy, many hoped that computers would reduce prejudice and discrimination. Yet, as we mentioned earlier when looking at mortgage applications and ethnicity, computers are made and trained by humans, and the data that those machines use stems from an unjust world. Simply put, if we are not careful, our programs will amplify human biases.

In the financial industry, anti-discrimination is not only a matter of morality. Take, for instance, the Equal Credit Opportunity Act (ECOA), which came into force in 1974 in the United States. This law explicitly forbids creditors from discriminating applicants based on race, sex, marital status, and several other attributes. It also requires creditors to inform applicants about the reasons for denial.

The algorithms discussed in this book are discrimination machines...