OpenAI is risk-testing Voice Engine, but the risks are clear

- YouTube
Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she says that while OpenAI is testing its new Voice Engine model to identify its risks, we've already experienced the clear dangers of voice impersonation technology. What we need is a more independent assessment of these new technologies applying equally to companies who want to tread carefully and those who want to race ahead in developing and deploying the technology.

About a year ago, I was part of a small meeting where I was asked to read a paragraph, sort of random text to me, it seemed. But before I knew it, I heard my own voice very convincingly, saying things through the speakers of the conference room that I had never said and would never say.

And it was really, you know, a sort of goosebump moment because I realized that generative AI used for voice was already very convincing. And that was a prototype of the voice engine, which is now being reported by the New York Times as having been this new product by OpenAi that the company is choosing to only release to a limited set of users as it's still testing the risky uses.

And I don't think this testing with a limited set of users is needed to understand the risks. We've already heard of fraudulent robocalls impersonating President Biden. We've heard of criminals trying to deceive parents, for example, with voice messages sounding like their children who are in trouble and asking for the parent to send money, which then, of course, benefits the criminal group, not their children.

So the risks of using voice impersonation are clear. Of course, companies will also point to opportunities of helping people who may have lost their voice through illness or disability, which I think is an important opportunity to explore. But we cannot be naive about the risks. And so in response to the political robocalls, the Federal Communications Commission at least drew a line and said that AI cannot be used for these. So there are some kind of restriction. But all in all, we need to see more independent assessment of these new technologies, a level playing field for all companies, not just those who want to choose to pace the release of their new models, but also those who want to race ahead. Because sooner or later, one or the other company will and we will all potentially be confronted with this widely accessible, voice generating artificial intelligence opportunity.

So it is a tricky moment when we see the race to bring to market and the rapid development of these technologies, which also incur a lot of risk and harm as an ongoing dynamic in the AI space. And so I hope that as there are discussions around regulation and guardrails happening around the world, that the full spectrum of use cases that we know and can anticipate will be on the table with the aim of keeping people free from crime, our democracy safe, while making sure that if there is a benefit for people in minority disabled communities, that they can benefit from this technology as well.

More from GZERO Media

- YouTube

Fifty years after the fall of Saigon (or its liberation, depending on whom you ask), Vietnam has transformed from a war-torn battleground to one of Asia’s fastest-growing economies—and now finds itself caught between two superpowers. Ian Bremmer breaks down how Vietnam went from devastation in the wake of the Vietnam War to becoming a regional economic powerhouse.

Eurasia Group and GZERO Media are seeking a highly creative, detail-oriented Graphic and Animation Designer who lives and breathes news, international affairs, and policy. The ideal candidate has demonstrated experience using visual storytelling—including data visualizations and short-form animations—to make complex geopolitical topics accessible, social-friendly, and engaging across platforms. You will join a dynamic team of researchers, editors, video producers, and writers to elevate our storytelling and thought leadership through innovative multimedia content.

The body of Pope Francis in the coffin exposed in St. Peter's Basilica in Vatican City on April 24, 2025. The funeral will be celebrated on Saturday in St. Peter's Square.
Pasquale Gargano/KONTROLAB/ipa-agency.net/IPA/Sipa USA

While the Catholic world prepares for the funeral of Pope Francis on Saturday – the service begins at 10 a.m. local time, 4 a.m. ET – certain high-profile attendees may also have other things on their mind. Several world leaders will be on hand to pay their respects to the pontiff, but they could also find themselves involved in bilateral talks.

A Ukrainian rescue worker sits atop the rubble of a destroyed residential building during rescue operations, following a Russian missile strike on a residential apartment building block in Kyiv, Ukraine, on April 24, 2025.
Photo by Justin Yau/ Sipa USA
Members of the M23 rebel group stand guard at the opening ceremony of Caisse Generale d'epargne du Congo (CADECO) which will serve as the bank for the city of Goma where all banks have closed since the city was taken by the M23 rebels, in Goma, North Kivu province in the East of the Democratic Republic of Congo, April 7, 2025.
REUTERS/Arlette Bashizi

The Democratic Republic of the Congo and an alliance of militias led by the notorious M23 rebels announced a ceasefire on Thursday after talks in Qatar and, after three years of violence, said they would work toward a permanent truce.

Students shout slogans and burn an effigy to protest the Pahalgam terror attack in Guwahati, Assam, India, on April 24, 2025. On April 22, a devastating terrorist attack occurs in Pahalgam, Jammu and Kashmir, resulting in the deaths of at least 28 tourists.
Photo by David Talukdar/NurPhoto

Prime Minister Narendra Modi has blamed Pakistan for Tuesday’s deadly terrorist attack in Kashmir, and he’s takenaggressive action against its government.