Gemini AI controversy highlights AI racial bias challenge

title placeholder | GZERO AI

Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she questions whether big tech companies can be trusted to tackle racial bias in AI, especially in the wake of Google's Gemini software controversy. Importantly, should these companies be the ones designing and deciding what that representation looks like?

This was a week full of AI-related stories. Again, the one that stood out to me was Google's efforts to correct for bias and discrimination in its generative AI model and utterly failing. We saw Gemini, the name of the model, coming up with synthetically generated images of very ethnically diverse Nazis. And of all political ideologies, this white supremacist group, of course, had few, if any, people of color in them historically. And that's the same, unfortunately, as the movement continues to exist, albeit in smaller form today.

And so, lots of questions, embarrassing rollbacks by Google about their new model, and big questions, I think, about what we can expect in terms of corrections here. Because the problem of bias and discrimination has been well researched by people like Joy Buolamwini with her new book out called “Unmasking AI,” her previous research “Codes Bias,” you know, well established how models by the largest and most popular companies are still so flawed with harmful and illegal consequence.

So, it begs the question, how much grip do the engineers developing these models really have on what the outcomes can be and how could this have gone so wrong while this product has been put onto the markets? There are even those who say it is impossible to be fully representative in a in a fair way. And it is a big question whether companies should be the ones designing and deciding what that representation looks like. And indeed, with so much power over these models and so many questions about how controllable they are, we should really ask ourselves, you know, when are these products ready to go to market and what should be the consequences when people are discriminated against? Not just because there is a revelation of an embarrassing flaw in the model, but, you know, this could have real world consequences, misleading notions of history, mistreating people against protections from discrimination.

So, even if there was a lot of outcry and sometimes even sort of entertainment about how poor this model performed, I think there are bigger lessons about AI governance to be learned from the examples we saw from Google's Gemini this past week.

More from GZERO Media

U.S. President Donald Trump delivers remarks on tariffs in the Rose Garden at the White House in Washington, D.C., U.S., April 2, 2025.
REUTERS/Carlos Barria

During a speech in the White House Rose Garden on Wednesday, Donald Trump announced a universal 10% tariff on all US imports, 25% tariffs on foreign-made cars and parts, as well as a naughty list of trading partners that were hit with “reciprocal tariffs” on top – to the tune of 20% for the EU, 54% for China, and 46% for Vietnam, to name a few of the hardest-hit.

Palestinians travel in vehicles between the northern and southern Gaza Strip along the Rashid Road on April 2, 2025.

Majdi Fathi/NurPhoto via Reuters

Israeli PM Benjamin Netanyahu said Wednesday that Israel was seizing more territory in Gaza to “divide up” the besieged enclave. He spoke as Israeli forces increased the intensity of their assault on Hamas in Gaza, which resumed two weeks ago after phase one of the ceasefire agreed to in January ended.

Vice President JD Vance and his wife, Usha Vance, tour the US military's Pituffik Space Base in Greenland on March 28, 2025.
JIM WATSON/Pool via REUTERS

How much would it cost for the United States to maintain Greenland as its territory? And what are the revenue possibilities from the Arctic island’s natural resources? Those are two questions the White House is reportedly looking into in the surest sign yet that Trump’s interest in Greenland is genuine.

Protesters demanded the ouster of South Korean President Yoon in central Seoul on March 29, 2025.
Lee Jae-Won/AFLO via Reuters

South Korea’s Constitutional Court will tie the legal bow on what has been a tumultuous period for the country as it rules Friday on whether to formally dismiss or reinstate impeached President Yoon Suk-yeol.

After voters elected her to the Wisconsin Supreme Court, liberal candidate Judge Susan Crawford celebrates with Wisconsin Supreme Court Judge Ann Walsh Bradley at her election night headquarters in Madison, Wisconsin, on April 1, 2025.

REUTERS/Vincent Alban

Republicans expanded their lean House majority after a pair of special elections in Florida, but a conservative candidate lost badly in a Wisconsin judicial race — despite a huge cash injection from Elon Musk.