Man vs machine: IBM India using software to root out racist, sexist bias

Venkatesh Ganesh What is common between Bollywood movies and characters in Man Booker Prize winning novels? Stereotypical biases, it seems. So, in a dataset that involved analysing 275 books and 3,200 Bollywood movies, technology giant IBM found that the stereotypes of men being portrayed as strong and violent and, on the other hand, women associated with home and considered to be gentle and less active compared to men, continues even in the 21st century.

These biases have started to creep into IT systems and IBM India is building software tools to tackle such biases that data throws up. “We are developing software tools which we can assist humans to flag where there are biases in data,” said Sriram Raghavan, Vice President, IBM Research and CTO, IBM India. Basically it is like a football referee overseeing a game.

Data bias is assuming significance due to increased use of technology in every walk of life- both personal and professional. Increased affordability of smartphones and internet access has resulted in a data deluge. This has made corporations across the world use this complex and vast data set to assist in their business decisions using techniques like machine learning and AI.

However, a spate of recent books — such as

Automating Inequality written by Associate Professor of Political Science Virginia Eubanks at the University at Albany, and Algorithms of Oppression by Safiya Umoja Noble, a faculty member of the University of Southern California — have raised concerns on how data is being used to discriminate against people. Noble points out in her book that racist and sexist bias, misinformation, and profiling are frequently unnoticed by-products of algorithms. And unlike public institutions (like the library), Google and Facebook have no transparent curation process by which the public can judge the credibility or legitimacy of the information they propagate.

The widespread concerns that these authors mention has potential implications in India where a large section of people are from disadvantaged sections of society and by using data to act in a biased manner, there is a risk of them not getting their rightful due. However, detection of such bias is only the first step and algorithms are being developed to de-bias such text, said Raghavan.

Globally, the biases in data pools have been attributed to software programmers not having engineered this in system design. “Software developers need to address the issue of data bias when building systems and that can help in alerting biases,” said Jai Ganesh, Vice President and Head Mphasis NEXT Labs.

Globally, sociologists believe there are 180 forms of biases. While machines can build in probably all of this in algorithms, the question remains whether a human force the system to confirm to his or her bias. “The machine learns over a period of time but at least there is a system of checks or balances that can shine the light on biases continuously,” said Raghavan.