The danger of biased algorithmic systems and how to solve it
What you give is what you get. This saying seems to be perfectly applicable to the realm of AI systems as well. Or as IBM puts it (3), artificial intelligence is only “as effective as the data it was trained on” and, therefore, machine intelligence is “accurately reflecting the prejudices of the people it drew its training from”. Such an inherent flaw may at first be invisible in algorithms powering services like loan-worthiness, medical diagnosis, and job candidate selection. However, as AI is increasingly changing the business world and making its way into other vital customer services, we must ensure that “these transformative technologies will have a net positive impact on society” (4).
One of the domains in which the footprint of the millions of everyday life decisions taken by machine intelligence can be identified is image recognition. In 2015, Google faced a backlash from the African American community as the Google Photos service wrongly labeled people of African American descent as ‘gorillas’ (5).
Another tech-giant that was dealing with similar problems is IBM with its Watson Visual Recognition. Earlier this year a study was published that scrutinizes the error scores of different AI systems and, unsurprisingly, IBM’s visual analysis tools share the same drawbacks with its counterparts over at Google. The error rates for recognizing faces of darker females and males ranged from 2% to 3.5% whereas the facial recognition capabilities for individuals with lighter skin colour were nearly perfect (4).
Of course, Google and IBM have improved their AIs immensely in the past few years but, nevertheless, such stories give you a good idea of how artificial intelligence has not yet reached the role of the ‘great equalizer’ that solely relies on “math and cold calculations, uncoloured by the bias or prejudices we may hold as people” (1).
Unfortunately, repairing biased algorithms will only get harder and more complex as time passes. Recent developments in technologies such as deep-learning and artificial neural networks are making it hard for the creators of AI to come to grips with the logic of their networks. “Creating its own new correlations, much like the human brain, in order to make a decision” (1), flawed human observations that were added to the ‘black box’ at the start of training processes could well be too enmeshed in AIs of companies to fully trace down and eradicate. Therefore, technology companies all around the world need to embrace this daunting challenge as fast as possible and collaborate deeply with everyone in society taking into consideration various “industries, specialties and backgrounds” (3).
Initial steps towards fairer and more transparent artificial decision-making are creating transparency standards, open-source code and reducing the inscrutability of AI. Its developers must be able to explain why algorithms act in a distinct manner and, additionally, outsiders should be granted access to an AI’s black box. The imperative here is quite clear: if an algorithmic decision is unexplainable, you should not be able to use the underlying algorithms. In this respect, there are already positive regulatory changes being made by the regulatory sector in many countries worldwide. Especially the European Union is very keen on implementing its new transparency standards under its pioneering data protection regulation (GDPR) that is supposed to put human consumers back in the driver’s seat.
Furthermore, and probably most importantly, companies must try to boost the diversity represented in teams that are working on building complex algorithmic systems. Individuals across all races, genders, cultures and socioeconomic backgrounds are needed if we really want to mitigate the bias problem in the world of AI and tech in general. And, fortunately, it seems that tech firms are getting more interested in solving diversity issues as we might believe. All of a sudden, the business world has come to realise that “when you exclude or fail to serve under-represented groups, you leave massive profits on the table” (1).
In conclusion, one must admit that today we are still not creating artificial intelligence that can be characterized as infallible and unbiased. People need to be more critical about how exactly AI is handling our decision making on a daily basis and if we put too much faith in the capabilities of current algorithms we could be in for an unpleasant surprise. Until now, we were “just obfuscating our own flawed observations inside of a black box” (2) and, as a consequence, we are still to reach ‘algorithmic equality’. However, as recent technological developments render artificial intelligent systems increasingly complex, our time to act is now. In no time, AI “will sleep in all corners of our lives” (1) and without introducing more transparency and diversity in the training processes of our AIs, the flaws of prejudiced algorithms may soon become too hard to tackle.
Bibliography:
- https://www.fastcompany.com/40536485/now-is-the-time-to-act-to-stop-bias-in-ai
Retrieved on 19th July 2018.
- https://thenextweb.com/artificial-intelligence/2018/04/10/human-bias-huge-problem-ai-heres-going-fix/
Retrieved on 19th July 2018.
- https://www.ibm.com/blogs/policy/bias-in-ai/
Retrieved on 19th July 2018.
- https://www.ibm.com/blogs/research/2018/02/mitigating-bias-ai-models/
Retrieved on 19th July 2018.
- https://www.usatoday.com/story/tech/2015/07/01/google-apologizes-after-photos-identify-black-people-as-gorillas/29567465/
Retrieved on 19th July 2018.
Users who have LIKED this post:
6 comments on “The danger of biased algorithmic systems and how to solve it”
Comments are closed.
I find this topic very interesting, too. Thanks for a great post!
This is truly a significant problem for AI and ML solutions, and may even cause distrust in applications if there would be any bad cases of biased AI to go public. As you said, this must be addressed and you suggested e.g. increasing diversity of ethnicity, gender and personal background in AI teams and also transparency of AI systems. These are good suggestions, since most that can be done lies in the applications and systems, not in the data. That is because the data describes the current state of the world, and to make e.g. predictions and recommendations in this current world you need data that describes it. As our guest lecturer Craig Martell from LinkedIn said, manipulating data may lead to screwing up the application for everyone. Furthermore, as we all know, research purposes do not allow manipulation of data to reach desired results. In conclusion, there are very few possibilities to correct biased data on the data side instead of the application side, especially when it is usually very hard to detect.
This article is such an interdisciplinary piece that leads us to think about a potential bias problem in AI tech. However, there are some facts that don’t really make sense to me so I just want to share with everyone.
I can see “bias” in the composition of AI teams but don’t see the “danger”. The author used an example of the error rate of facial recognition on African American to try to address a bias issue. Why bother with such minutia when you know AI can do a lot more? Before we criticize the biased algorithm, it could very well be a statistical fact that African American faces tend to be less identifiable. Before we can conduct a control experiment, we can’t blame this fact on technicians who write the algorithm. Additionally, this single story doesn’t tell us anything, and it doesn’t imply an underlying bias in the AI team as well.
One thing I really like the tech world is that it is the fairest playground for participants. No matter what social standing you are in, what nationality you have, where you were born, as long as you can make things happen, you deserve the applaud and reward.
Hi xwang01
I don’t think Adam or anyone else is blaming the team that wrote the algorithm for the bias. It is the data that is inherently biased because it comes from our biased world. But data scientists can identify these biases and try to adjust the model.
I think the example Craig Martell gave works better to illustrate this. If I remember correctly, LinkedIn was suggesting high level management jobs preferably to men because in the training data men made up most of the current management positions. The model was working correctly but reinforced the problem of too few women in top management. The “danger” is that these models can have a very real impact on people’s lives. So it is very important to recognized these issues which can be very hard. In the case of gender or race it might be obvious but in other areas we may often not be aware of the biases.
I’m going to have to disagree with xwang01, and throw my weight behind Adam Dada in emphasizing the importance of thinking of the “danger” of AI, or as Craig Martell put it, Big Data with sexy statistics. Likely, the reason why in 2015 Google Photos incorrectly labeled people of African American descent as ‘gorillas’ is because of people’s bias inherent in the Big Data Google runs sexy statistics on. Relating gorillas to people of African American descent is not an uncommon comparison committed by racist individuals. This “mistake” arises currently in the same way in the late 1800s/early 1900s, if Google Photos were in place, would incorrectly label those of Irish descent as ‘gorillas,’ I think and it is foolish and naive for us to think otherwise. One of the problems with the obscurity and lack of conceptual clarity behind A.I. and with how most people have a strong enough grasp on the term to know to associate it with Big Data is the sense of objectivity the buzzword implies. Recasting A.I. as Big Data with sexy statistics, I think, does little to decouple the term from objectivity, as I believe it’s coming from this prevalent assumption that statistics, like other disciplines like math and physics (housed in the hard sciences) is objective. It’s a real problem, I think, that this is advancing so rapidly. Laws, regulations, etc… have always had to play catchup, especially during phases of rapidly changing technology. It’s not wise to think of these matters lightly.
I great point highlighted is the potential bias of AI. Today’s compute resources combined with sophisticated algorithms deliver great value, especially when drawing inferences from large data sets. It is important to keep in mind that the decisions artificial intelligence makes are derived from datasets. Artificial intelligence is not human intelligence, even if human decisions are often times based on data as well. Even if datasets are incredibly large the decisions made by artificial intelligence might be skewed. The main issues can be observed when looking at social media feeds. They do a great job in terms of recommending items of interest to the specific user profile. The question remains if such artificial intelligence-fueled feeds are just mirroring current beliefs of the user. Thus, preventing them from becoming more intelligent by reinforcing the bias of the system and user at the same time.
Thanks for picking up on such an important topic and stressing the importance to tackle it.
A few thoughts:
1) One big trouble with requiring open source and explainability of algorithms are a) proprietarity of tools – it could basically screw up the business models, but probably mitigatable, b) as far as I know (at this might be limited) many algorithms that perform superbly in predicting carry incredibly little value to interpret and vice versa.
2) I’m not sure I understand how the diversity of the team building the algorithms helps reducing the bias, vs. making sure the training sets are up to (probably not yet existing) diversity standards?
3) A point that doesn’t make biased AI better, but I want to throw it in, because of the direction public discussions sometimes lean to that seems as if only with AI we had bias: we humans have huge biases, too. Our bias might seem less systemic than AI’s, but so far it is above all less tangible. Even the fact that we have identified the (or at least part of the) problem with AI bias (datasets) seems to be much more actionable than what we know about and can influence what is going on in the human mind. I would love to see a comparison of AI bias and human bias, for instance accuracy rates, in hiring, judicial cases, and so on that have been discussed publicly. Human bias and psychological fallacies underlying them are incredibly strong (like judges decisions in correlation to their food and break schedule, prejudice, availability heuristics – Thinking Fast and Slow gives a pretty scary overview on this), but for some reason we are biased toward human bias, trust it more than AI bias. Just a thought & while I have a bit of an understanding of ML algorithms etc, I am not an AI professional or alike, so I’d love to hear other peoples views on that.