Algorithmic Bias: Challenges and Solutions

A recent article in The Guardian described the issue of discrimination by algorithms with a few examples:

“There was the voice recognition software that struggled to understand women, the crime prediction algorithm that targeted black neighborhoods and the online ad platform which was more likely to show men highly paid executive jobs”.

When I worked on a fraud analytics project under the supervision of the Chief Analytics Officer of a leading fraud detection company, I learned that often times fraud algorithms can also be biased against certain races (minorities), or against immigrants, or even against men. In such situations, the Consumer Financial Protection Bureau intervenes by setting rigorous regulation. The “disparate impact” regulation suggests that any particular race, class, gender, religion, national origin, disability status etc. may not be impacted unequally in certain public domains.

The formal definition is as follows: “Disparate impact in United States labor law refers to practices in employment, housing, and other areas that adversely affect one group of people of a protected characteristic more than another, even though rules applied by employers or landlords are formally neutral”. As a result in a lot of cases it is illegal to be discriminating by criteria like ethnicity, etc. even if the model suggests that a particular ethnicity is more or less likely to achieve an outcome.

Solution to this challenge:

In the case of fraud detection, or banking in general, banks/fraud agencies are forced to ensure that their models are actually not biased against several relevant criteria laid out under the relevant disparate impact provision. Hence, models have to be changed, and tests have to be run to measure if “disparate impact” is in fact taking place, and if so, the model needs to ensure that it doesn’t happen. Often times, the bankers, data scientists, and legal folks have to sit together and come to an agreement on what makes the most business sense (ensures profitability) for the institution, without compromising any legal provisions. Thus, much to the dislike of data scientists, they often have to settle for models with lower R-squared statistics, or lower predictive power.

Additionally, certain models are black box models, like neural nets/deep learning and these are much harder to explain, and to actually understand what criteria is being used for decision making. On the other hand, a model like logistic regression, although simple, allows for very good inference and we know exactly which factors are used in the model predictions. Thus, while machine learning methods are advancing rapidly, one challenge is to work with legal institutions to find ways to ensure that thing such as disparate impact do not take place, as the models become more “black-box”.

Google has introduced a new method to test for and ensure that such discrimination does not take place. “Their approach is called the “Equality of Opportunity in Supervised Learning” works on the basic principle that when an algorithm makes a decision about an individual – the decision should not reveal anything about the individual’s race or gender beyond what might be gleaned from the data itself”

Such approaches will allow us to ensure algorithmic fairness in the rapidly advancing machine learning/AI world.

 

 

 

 

 

 

Works cited:-

https://www.theguardian.com/technology/2016/dec/19/discrimination-by-algorithm-scientists-devise-test-to-detect-ai-bias

https://en.wikipedia.org/wiki/Disparate_impact