Gemini AI controversy highlights AI racial bias challenge

title placeholder | GZERO AI

Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she questions whether big tech companies can be trusted to tackle racial bias in AI, especially in the wake of Google's Gemini software controversy. Importantly, should these companies be the ones designing and deciding what that representation looks like?

This was a week full of AI-related stories. Again, the one that stood out to me was Google's efforts to correct for bias and discrimination in its generative AI model and utterly failing. We saw Gemini, the name of the model, coming up with synthetically generated images of very ethnically diverse Nazis. And of all political ideologies, this white supremacist group, of course, had few, if any, people of color in them historically. And that's the same, unfortunately, as the movement continues to exist, albeit in smaller form today.

And so, lots of questions, embarrassing rollbacks by Google about their new model, and big questions, I think, about what we can expect in terms of corrections here. Because the problem of bias and discrimination has been well researched by people like Joy Buolamwini with her new book out called “Unmasking AI,” her previous research “Codes Bias,” you know, well established how models by the largest and most popular companies are still so flawed with harmful and illegal consequence.

So, it begs the question, how much grip do the engineers developing these models really have on what the outcomes can be and how could this have gone so wrong while this product has been put onto the markets? There are even those who say it is impossible to be fully representative in a in a fair way. And it is a big question whether companies should be the ones designing and deciding what that representation looks like. And indeed, with so much power over these models and so many questions about how controllable they are, we should really ask ourselves, you know, when are these products ready to go to market and what should be the consequences when people are discriminated against? Not just because there is a revelation of an embarrassing flaw in the model, but, you know, this could have real world consequences, misleading notions of history, mistreating people against protections from discrimination.

So, even if there was a lot of outcry and sometimes even sort of entertainment about how poor this model performed, I think there are bigger lessons about AI governance to be learned from the examples we saw from Google's Gemini this past week.

More from GZERO Media

- YouTube

Fifty years after the fall of Saigon (or its liberation, depending on whom you ask), Vietnam has transformed from a war-torn battleground to one of Asia’s fastest-growing economies—and now finds itself caught between two superpowers. Ian Bremmer breaks down how Vietnam went from devastation in the wake of the Vietnam War to becoming a regional economic powerhouse.

Eurasia Group and GZERO Media are seeking a highly creative, detail-oriented Graphic and Animation Designer who lives and breathes news, international affairs, and policy. The ideal candidate has demonstrated experience using visual storytelling—including data visualizations and short-form animations—to make complex geopolitical topics accessible, social-friendly, and engaging across platforms. You will join a dynamic team of researchers, editors, video producers, and writers to elevate our storytelling and thought leadership through innovative multimedia content.

The body of Pope Francis in the coffin exposed in St. Peter's Basilica in Vatican City on April 24, 2025. The funeral will be celebrated on Saturday in St. Peter's Square.
Pasquale Gargano/KONTROLAB/ipa-agency.net/IPA/Sipa USA

While the Catholic world prepares for the funeral of Pope Francis on Saturday – the service begins at 10 a.m. local time, 4 a.m. ET – certain high-profile attendees may also have other things on their mind. Several world leaders will be on hand to pay their respects to the pontiff, but they could also find themselves involved in bilateral talks.

A Ukrainian rescue worker sits atop the rubble of a destroyed residential building during rescue operations, following a Russian missile strike on a residential apartment building block in Kyiv, Ukraine, on April 24, 2025.
Photo by Justin Yau/ Sipa USA
Members of the M23 rebel group stand guard at the opening ceremony of Caisse Generale d'epargne du Congo (CADECO) which will serve as the bank for the city of Goma where all banks have closed since the city was taken by the M23 rebels, in Goma, North Kivu province in the East of the Democratic Republic of Congo, April 7, 2025.
REUTERS/Arlette Bashizi

The Democratic Republic of the Congo and an alliance of militias led by the notorious M23 rebels announced a ceasefire on Thursday after talks in Qatar and, after three years of violence, said they would work toward a permanent truce.

Students shout slogans and burn an effigy to protest the Pahalgam terror attack in Guwahati, Assam, India, on April 24, 2025. On April 22, a devastating terrorist attack occurs in Pahalgam, Jammu and Kashmir, resulting in the deaths of at least 28 tourists.
Photo by David Talukdar/NurPhoto

Prime Minister Narendra Modi has blamed Pakistan for Tuesday’s deadly terrorist attack in Kashmir, and he’s takenaggressive action against its government.

- YouTube

“When things are going fine, nobody really tests the skills and talents of their financial advisor, but this is a moment where really good advice can be extraordinarily powerful,” says Margaret Franklin, CFA Institute's CEO and President. In conversation with GZERO’s Tony Maciulis, Franklin describes the current financial climate as “maximum uncertainty,” rating it a 10 out of 10 on the risk scale.

President Donald Trump at a bilateral meeting with China's President Xi Jinping during the G20 leaders summit in Osaka, Japan, on June 29, 2019.
REUTERS/Kevin Lamarque/File Photo

On Wednesday, Donald Trump said he would deliver a “fair deal” with China and that he’d be “very nice” to the country after meeting with major retailers. But Beijing denies that there are any ongoing talks and has told the US it must cancel its unilateral tariffs before China will broker any negotiations.