Gemini AI controversy highlights AI racial bias challenge

title placeholder | GZERO AI

Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she questions whether big tech companies can be trusted to tackle racial bias in AI, especially in the wake of Google's Gemini software controversy. Importantly, should these companies be the ones designing and deciding what that representation looks like?

This was a week full of AI-related stories. Again, the one that stood out to me was Google's efforts to correct for bias and discrimination in its generative AI model and utterly failing. We saw Gemini, the name of the model, coming up with synthetically generated images of very ethnically diverse Nazis. And of all political ideologies, this white supremacist group, of course, had few, if any, people of color in them historically. And that's the same, unfortunately, as the movement continues to exist, albeit in smaller form today.

And so, lots of questions, embarrassing rollbacks by Google about their new model, and big questions, I think, about what we can expect in terms of corrections here. Because the problem of bias and discrimination has been well researched by people like Joy Buolamwini with her new book out called “Unmasking AI,” her previous research “Codes Bias,” you know, well established how models by the largest and most popular companies are still so flawed with harmful and illegal consequence.

So, it begs the question, how much grip do the engineers developing these models really have on what the outcomes can be and how could this have gone so wrong while this product has been put onto the markets? There are even those who say it is impossible to be fully representative in a in a fair way. And it is a big question whether companies should be the ones designing and deciding what that representation looks like. And indeed, with so much power over these models and so many questions about how controllable they are, we should really ask ourselves, you know, when are these products ready to go to market and what should be the consequences when people are discriminated against? Not just because there is a revelation of an embarrassing flaw in the model, but, you know, this could have real world consequences, misleading notions of history, mistreating people against protections from discrimination.

So, even if there was a lot of outcry and sometimes even sort of entertainment about how poor this model performed, I think there are bigger lessons about AI governance to be learned from the examples we saw from Google's Gemini this past week.

More from GZERO Media

Small businesses play an outsized role in driving economic output on the local and global levels. But smaller margins and fewer resources than larger companies make them far more vulnerable to climate shocks. We must reduce the climate risks facing smaller enterprises while accelerating their ability to deliver climate solutions and play a central role in the transition to a low-carbon, regenerative economy. A new report, produced by Christensen Global and supported by the Mastercard Center for Inclusive Growth, reveals three things small businesses need to unlock their power to accelerate climate-smart inclusive growth: data, capital, and wrap-around support.

US President Joe Biden delivers remarks in the Rose Garden of the White House in Washington after Israel and Lebanon accepted a ceasefire deal on Nov. 26, 2024.
Yuri Gripas/ABACAPRESS.COM via Reuters

The Israeli Security Cabinet has approved a ceasefire for Lebanon, President Joe Biden announced on Tuesday, welcoming the opportunity to start reestablishing peace in the Middle East. Early Wednesday, 13 months of fighting ended as the ceasefire took hold, and thousands of displaced Lebanese civilians began returning to their homes in the South.

Mexican President Claudia Sheinbaum speaks after reading a letter to be sent to U.S. President-elect Donald Trump, warning that tariffs would cause inflation and job losses in both countries, at National Palace in Mexico City, Mexico, November 26, 2024.
Carlos Ramos Mamahua/Presidencia de Mexico/Handout via REUTERS
Art by Annie Gugliotta/GZERO Media

With the US Thanksgiving holiday approaching, millions of American families will soon sit down to a turkey dinner. That makes it as good a time as any to ask an important question: Why are turkeys, which are not actually from Turkey, called turkeys? No other animal is named for so many countries that it's not actually from.

Security force personnel walk as smoke billows from tear gas shells fired to prevent an anti-government protest by supporters of the former Pakistani Prime Minister Imran Khan's party Pakistan Tehreek-e-Insaf (PTI) demanding the release of Khan, in Islamabad, Pakistan, November 26, 2024
REUTERS/Akhtar Soomro

Supporters of jailed former Pakistani Prime Minister Imran Khan broke through numerous barricades to clash with police in the capital, Islamabad, on Tuesday, leaving at least six members of security forces dead. On Wednesday, Khan's party said demonstrations were “temporarily suspended” owing to “government brutality.”