Gemini AI controversy highlights AI racial bias challenge

title placeholder | GZERO AI

Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she questions whether big tech companies can be trusted to tackle racial bias in AI, especially in the wake of Google's Gemini software controversy. Importantly, should these companies be the ones designing and deciding what that representation looks like?

This was a week full of AI-related stories. Again, the one that stood out to me was Google's efforts to correct for bias and discrimination in its generative AI model and utterly failing. We saw Gemini, the name of the model, coming up with synthetically generated images of very ethnically diverse Nazis. And of all political ideologies, this white supremacist group, of course, had few, if any, people of color in them historically. And that's the same, unfortunately, as the movement continues to exist, albeit in smaller form today.

And so, lots of questions, embarrassing rollbacks by Google about their new model, and big questions, I think, about what we can expect in terms of corrections here. Because the problem of bias and discrimination has been well researched by people like Joy Buolamwini with her new book out called “Unmasking AI,” her previous research “Codes Bias,” you know, well established how models by the largest and most popular companies are still so flawed with harmful and illegal consequence.

So, it begs the question, how much grip do the engineers developing these models really have on what the outcomes can be and how could this have gone so wrong while this product has been put onto the markets? There are even those who say it is impossible to be fully representative in a in a fair way. And it is a big question whether companies should be the ones designing and deciding what that representation looks like. And indeed, with so much power over these models and so many questions about how controllable they are, we should really ask ourselves, you know, when are these products ready to go to market and what should be the consequences when people are discriminated against? Not just because there is a revelation of an embarrassing flaw in the model, but, you know, this could have real world consequences, misleading notions of history, mistreating people against protections from discrimination.

So, even if there was a lot of outcry and sometimes even sort of entertainment about how poor this model performed, I think there are bigger lessons about AI governance to be learned from the examples we saw from Google's Gemini this past week.

More from GZERO Media

People attend a rally to protest against the arrest of Istanbul Mayor Ekrem Imamoglu as part of a corruption investigation in Istanbul, Turkey, on March 29, 2025.
REUTERS/Umit Bektas

Hundreds of thousands of people flooded the streets of Istanbul this weekend to protest the detainment of Istanbul Mayor Ekrem Imamoglu, a popular contender for the next presidential election.

Democratic-backed Dane County Circuit Judge Susan Crawford and Republican-backed Waukesha County Circuit Judge Brad Schimel square off in their only debate until their April 1 election.
Brian Cahn/ZUMA Press Wire via Reuters

Elections are back in the United States — and so is the money. Six months after the 2024 US presidential vote, Wisconsinites will head to the polls Tuesday to decide whether liberal candidate Susan Crawford or her opponent, conservative Brad Schimel,will tip the ideological balance of the state Supreme Court. The liberals currently have a 4-3 advantage.

US Secretary of Defense Pete Hegseth shakes hands with Japanese Prime Minister Shigeru Ishiba at the Prime Minister's office in Tokyo on March 30, 2025.
POOL via ZUMA Press Wire via Reuters

In his first trip to Asia this weekend, US Secretary of Defense Pete Hegseth called for greater military cooperation between Tokyo and Washington.

People walk by as a painter repaints an anti-US mural in Tehran, Iran, on Saturday, March 29, 2025.
Majid Asgaripour/WANA via Reuters

On Sunday, US President Donald Trump issued a stark warning to Iran, threatening to bomb the country and impose secondary tariffs if Tehran fails to reach a new agreement on its nuclear program. In a telephone interview with NBC News, Trump stated, “If they don’t make a deal, there will be bombing. It will be bombing the likes of which they have never seen before.”

President Donald Trump waves as he walks before departing for Florida from the South Lawn at the White House in Washington, D.C., U.S., on March 28, 2025.

REUTERS/Evelyn Hockstein

Is the bloom off the bromance between US President Donald Trump and Russian President Vladimir Putin? On Sunday, Trump took Putin to task over Russia’s foot-dragging on a ceasefire in Ukraine and threatened to tariff Russian oil and impose more sanctions on the country.

Rescuers work at the site of a building that collapsed after the strong earthquake in Mandalay, Myanmar, on Sunday, March 30, 2025.
REUTERS/Stringer

The death toll continues to rise in Myanmar after a devastating 7.7-magnitude earthquake struck near the central city of Mandalay on March 28. Approximately 1,700 people are dead and over 3,400 injured, with the US Geological Service estimating that casualties could top 10,000. Relief operations are further complicated by Myanmar’s ongoing civil war, though a two-week ceasefire was declared on Sunday.

Listen: Elon Musk, the world’s richest man, made his fortune-breaking industries—space, cars, social media—and is now trying to break the government… in the name of fixing it. But what happens when Silicon Valley’s ‘move fast and break things’ ethos collides with the machinery of federal bureaucracy? On the GZERO World Podcast, Ian Bremmer sits down with WIRED Global Editorial Director Katie Drummond to unpack the implications of Musk’s deepening role in the Trump administration and what’s really behind his push into politics.

France's President Emmanuel Macron speaks during a press conference following a summit for the "coalition of the willing" at the Elysee Palace in Paris on March 27, 2025.

LUDOVIC MARIN/Pool via REUTERS

At the third summit of the so-called “coalition of the willing” for Ukraine on Thursday, French President Emmanuel Macron proposed a multinational “reassurance force” to deter Russian aggression once a ceasefire is in place – and to engage if attacked.