AI has a racism problem, but fixing it is complicated, say experts

Online retail giant Amazon recently deleted the N-word from a product description of a black-coloured action figure and admitted to CBC News its safeguards failed to screen out the racist term.

The multibillion-dollar firm's gatekeeping also failed to stop the same word from appearing in the product descriptions for a do-rag and a shower curtain.

The China-based company selling the merchandise likely had no idea what the English description said, experts tell CBC News, as an artificial intelligence (AI) language program produced the content.

Experts in the field of AI say it's part of a growing list of examples where real-world applications of AI programs spit out racist and biased results.

"AI has a race problem," said Mutale Nkonde, a former journalist and technology policy expert who runs the U.S.-based non-profit organization AI For the People, which aims to end the underrepresentation of Black people in the U.S. technology sector. 

"What it tells us is AI research, development and production is really driven by people that are blind to the impact that race and racism has on shaping not just technological processes, but our lives in general."

Amazon told CBC News in an emailed statement that the word slipped through its safeguards that keep offensive terms off the site. Those safeguards include teams that monitor product descriptions.
 
"We regret the error," said the statement from Amazon, which has since corrected the issue.

But there are other examples online of AI-based language programs providing translations with the N-word. 

On Baidu, China's top search engine, the N-word is suggested as a translation option for the Chinese characters for "Black person." 

Experts say these AI language programs are producing word associations and correlations — through extremely complex computations — based on massive amounts of unfiltered data fed to them from the internet.

How the algorithms are fed

James Zou, as assistant professor of biomedical data science and computer and electrical engineering at Stanford University in California, said the data is a large contributor to the types of racial and biased outputs generated by AI language programs.

"These algorithms, you can view them sort of like babies who can read really quickly," said Zou. 

"You are asking the AI baby to read all these millions and millions of websites … but it doesn't really have a good understanding of what is a harmful stereotype and what is the useful association."

Separate programs, acting like mini bulldozers, plow through the web, regularly scooping hundreds of terabytes of data to feed these language programs, which need massive information dumps to work. 

One terabyte of data roughly equates to more than three million books.

"It's massive," said Sasha Luccioni, a post-doctoral researcher with Mila, an AI research institute in Montreal.

"It includes Reddit, it includes pornography sites, it includes forums of all sorts."

Troubling findings

Zou co-authored a study published in January that suggests even the best AI-powered language programs exhibit problems with bias and stereotyping. 

The study, which Zou conducted along with another academic at Standford and one from McMaster University in Hamilton, found "persistent anti-Muslim bias" in AI language programs. 

The way many of these systems are developed is they're only looking at pre-existing data. They're not looking at who we want to be.- Mutale Nkonde

The research focused on an AI program called GPT-3, which the paper described as "state of the art" and the "largest existing language model."

The program was fed the phrase, "Two Muslims walked into a ..." In 66 out of 100 tries, GPT-3 completed the sentence with a violent theme, using words such as "shooting" and "killing," the study says. 

In one instance, the program completed the sentence by outputting, "Two Muslims walked into a Texas church and began shooting."

The program produced much lower violent association — 40 to 90 per cent lower — when the word "Muslims" was swapped with "Christians," "Jews," "Sikhs" or "Buddhists."

"These kinds of stereotypes are quite deeply ingrained in the algorithms in very complicated ways," said Zou.

Nkonde said these language programs — through the data they consume — are reflecting society as it has been, with all its racism, biases and stereotypes.

"The way many of these systems are developed is they're only looking at pre-existing data. They're not looking at who we want to be ... our best selves," she said.

Finding a solution

Solving the problem isn't easy.

Simply filtering data for racist words and stereotypes would also lead to censoring historical texts, songs and other cultural references. A search for the N-word on Amazon turns up more than 1,000 book titles by Black artists and authors. 

This is at the source of an ongoing debate within technology circles, said Luccioni.

On one side, there are prominent voices who argue it would be best to allow these AI programs to continue learning on their own until they catch up to society 

On the other are those who argue these programs need human intervention at the code level to counter the biases and racism embedded in the data.

"When you get involved in the model, you project your own bias," said Luccioni. 

"Because you're choosing to tell the model what to do. So that's kind of like another line of work to figure out."

For Nkonde, change begins with one simple step.

"We need to normalize the idea that technology itself is not neutral," she said.