OpenAI is risk-testing Voice Engine, but the risks are clear

- YouTube
Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she says that while OpenAI is testing its new Voice Engine model to identify its risks, we've already experienced the clear dangers of voice impersonation technology. What we need is a more independent assessment of these new technologies applying equally to companies who want to tread carefully and those who want to race ahead in developing and deploying the technology.

About a year ago, I was part of a small meeting where I was asked to read a paragraph, sort of random text to me, it seemed. But before I knew it, I heard my own voice very convincingly, saying things through the speakers of the conference room that I had never said and would never say.

And it was really, you know, a sort of goosebump moment because I realized that generative AI used for voice was already very convincing. And that was a prototype of the voice engine, which is now being reported by the New York Times as having been this new product by OpenAi that the company is choosing to only release to a limited set of users as it's still testing the risky uses.

And I don't think this testing with a limited set of users is needed to understand the risks. We've already heard of fraudulent robocalls impersonating President Biden. We've heard of criminals trying to deceive parents, for example, with voice messages sounding like their children who are in trouble and asking for the parent to send money, which then, of course, benefits the criminal group, not their children.

So the risks of using voice impersonation are clear. Of course, companies will also point to opportunities of helping people who may have lost their voice through illness or disability, which I think is an important opportunity to explore. But we cannot be naive about the risks. And so in response to the political robocalls, the Federal Communications Commission at least drew a line and said that AI cannot be used for these. So there are some kind of restriction. But all in all, we need to see more independent assessment of these new technologies, a level playing field for all companies, not just those who want to choose to pace the release of their new models, but also those who want to race ahead. Because sooner or later, one or the other company will and we will all potentially be confronted with this widely accessible, voice generating artificial intelligence opportunity.

So it is a tricky moment when we see the race to bring to market and the rapid development of these technologies, which also incur a lot of risk and harm as an ongoing dynamic in the AI space. And so I hope that as there are discussions around regulation and guardrails happening around the world, that the full spectrum of use cases that we know and can anticipate will be on the table with the aim of keeping people free from crime, our democracy safe, while making sure that if there is a benefit for people in minority disabled communities, that they can benefit from this technology as well.

More from GZERO Media

Director Sean Baker, producers Alex Coco, and Samantha Quan, and cast and crew members win the Oscar for best picture for "Anora" during the Oscars show at the 97th Academy Awards in Hollywood, Los Angeles, on March 2, 2025.

REUTERS/Carlos Barria

It was a big night for independent filmmaking and a film with Russian themes at the Oscars on Sunday as “Anora” took home five Academy Awards, including best screenplay, best editing, best director, best actress, and best picture.

Britain's Prime Minister Keir Starmer, President Volodymyr Zelenskiy, France's President Emmanuel Macron, Canada's Prime Minister Justin Trudeau, Germany's Chancellor Olaf Scholz, European Commission President Ursula von der Leyen, European Council President Antonio Costa, Denmark's Prime Minister Mette Frederiksen, Sweden's Prime Minister Ulf Kristersson, Norway's Prime Minister Jonas Gahr Store, Finland's President Alexander Stubb and other officials attend the European leaders' summit to discuss European security and Ukraine, at Lancaster House in London, Britain, on March 2, 2025.
NTB/Javad Parsa/via REUTERS

British Prime Minister Keir Starmer, European leaders from France, Italy, Germany, and other nations, as well as Ukrainian President Volodymyr Zelensky and Canadian Prime Minister Justin Trudeau, spent the weekend in London crafting a European-led plan to bring peace to Ukraine.

Syrian Kurds gather with flags as Turkey's jailed militant leader Abdullah Ocalan calls on his Kurdistan Workers Party to lay down its arms last week in Hasakah, Syria.

REUTERS/Orhan Qereman

After a 40-year conflict with Turkey that has killed 40,000 people, the Kurdistan Workers’ Party, known as the PKK, declared a ceasefire on Saturday following a call from its leader, Abdullah Ocalan, to dissolve the group. Ocalan, imprisoned since 1999, called the move a “historical responsibility” – but one that brings no apparent concessions from Ankara.

Israeli tanks are seen inside Gaza amid a ceasefire breakdown between Israel and Hamas on March 2, 2025.
REUTERS/Amir Cohen

Israeli Defense Forces blocked aid trucks from entering Gaza on Sunday, just one day after the first phase of the ceasefire between Israel and Hamas expired, bringing negotiations over a permanent truce to a standstill.

A man gestures toward security forces during an anti-government rally in Bucharest, Romania, March 1, 2025.
REUTERS/Andreea Campeanu

Tens of thousands of far-right demonstrators gathered in Bucharest on Saturday to protest the Romanian government’s decision to call off a second round of national elections, deeming it an assault on democracy. A rerun of the first round is now scheduled for May 4, but the protesters want the government to reinstate the original result and hold a run-off instead.

The Kremlin

China and Russia are reportedly looking to exploit US federal workforce cuts by targeting recently fired or at-risk federal employees in national security roles for recruitment, according to sources familiar with US intelligence. The quarries? Employees with top security clearances and information about America’s critical infrastructure and government operations.

- YouTube

On GZERO World with Ian Bremmer, Latvian Foreign Minister Baiba Braže discusses Ukraine's fate and Baltic security in the face of Russian aggression. Former Russian colonel Dmitri Trenin offers a starkly different perspective from Moscow, arguing that negotiations over Ukraine should be decided primarily by the US and Russia—not Ukraine or Europe.