OpenAI is risk-testing Voice Engine, but the risks are clear

- YouTube
Marietje Schaake, International Policy Fellow, Stanford Human-Centered Artificial Intelligence, and former European Parliamentarian, co-hosts GZERO AI, our new weekly video series intended to help you keep up and make sense of the latest news on the AI revolution. In this episode, she says that while OpenAI is testing its new Voice Engine model to identify its risks, we've already experienced the clear dangers of voice impersonation technology. What we need is a more independent assessment of these new technologies applying equally to companies who want to tread carefully and those who want to race ahead in developing and deploying the technology.

About a year ago, I was part of a small meeting where I was asked to read a paragraph, sort of random text to me, it seemed. But before I knew it, I heard my own voice very convincingly, saying things through the speakers of the conference room that I had never said and would never say.

And it was really, you know, a sort of goosebump moment because I realized that generative AI used for voice was already very convincing. And that was a prototype of the voice engine, which is now being reported by the New York Times as having been this new product by OpenAi that the company is choosing to only release to a limited set of users as it's still testing the risky uses.

And I don't think this testing with a limited set of users is needed to understand the risks. We've already heard of fraudulent robocalls impersonating President Biden. We've heard of criminals trying to deceive parents, for example, with voice messages sounding like their children who are in trouble and asking for the parent to send money, which then, of course, benefits the criminal group, not their children.

So the risks of using voice impersonation are clear. Of course, companies will also point to opportunities of helping people who may have lost their voice through illness or disability, which I think is an important opportunity to explore. But we cannot be naive about the risks. And so in response to the political robocalls, the Federal Communications Commission at least drew a line and said that AI cannot be used for these. So there are some kind of restriction. But all in all, we need to see more independent assessment of these new technologies, a level playing field for all companies, not just those who want to choose to pace the release of their new models, but also those who want to race ahead. Because sooner or later, one or the other company will and we will all potentially be confronted with this widely accessible, voice generating artificial intelligence opportunity.

So it is a tricky moment when we see the race to bring to market and the rapid development of these technologies, which also incur a lot of risk and harm as an ongoing dynamic in the AI space. And so I hope that as there are discussions around regulation and guardrails happening around the world, that the full spectrum of use cases that we know and can anticipate will be on the table with the aim of keeping people free from crime, our democracy safe, while making sure that if there is a benefit for people in minority disabled communities, that they can benefit from this technology as well.

More from GZERO Media

More than 60% of Walmart suppliers are small businesses.* Through a $350 billion investment in products made, grown, or assembled in the US, Walmart is helping these businesses expand, create jobs, and thrive. This effort is expected to support the creation of over 750,000 new American jobs by 2030, empowering companies like Athletic Brewing, Bon Appésweet, and Milo’s Tea to grow their teams, scale their production, and strengthen the communities they call home. Learn more about Walmart's commitment to US manufacturing. *See website for additional details.

Last month, Microsoft released its 2025 Responsible AI Transparency Report, demonstrating the company’s sustained commitment to earning trust at a pace that matches AI innovation. The report outlines new developments in how we build and deploy AI systems responsibly, how we support our customers, and how we learn, evolve, and grow. It highlights our strengthened incident response processes, enhanced risk assessments and mitigations, and proactive regulatory alignment. It also covers new tools and practices we offer our customers to support their AI risk governance efforts, as well as how we work with stakeholders around the world to work towards governance approaches that build trust. You can read the report here.

Ukraine President Volodymyr Zelenskyy meets with U.S President Trump in the Hague during the NATO Summit on Wednesday June 25, 2025.
(Ukraine Presidential Press Office/EYEPRESS)

In a major White House U-turn, US President Donald Trump declared that his government would resume sending key defensive weapons – including Patriot missiles – to Ukraine.

Demonstration of AI innovation at the AI for Good Summit in Geneva, Switzerland, on July 7, 2025.
Photo courtesy of ITU

Since ChatGPT burst onto the scene in late 2022, it’s been nearly impossible to attend a global conference — from Davos to Delhi — without encountering a slew of panels and keynote speeches on artificial intelligence. Will AI make our lives easier, or will it destroy humanity? Can it be a force for good? Can AI be regulated without stifling innovation? At the ripe old age of eight, the AI for Good Summit is now a veteran voice in this rapidly-evolving dialogue. It kicks off today in Geneva, Switzerland, for what promises to be its most ambitious edition yet.

- YouTube

Frederic Werner discusses the importance of AI for global impact at the 2025 AI for Good Summit in Geneva, in an exclusive Global Stage interview with GZERO's Tony Maciulis. They discuss the future of AI and its role in solving humanity's challenges, from harnessing quantum computing to closing the digital divide.

- YouTube

Elon Musk wants to start a new political party and it’s already making waves. In this episode of Ian Bremmer’s Quick Take, Ian unpacks Musk’s so-called “America Party,” driven by Musk’s frustration with both Republicans and Democrats.

UK Prime Minister Keir Starmer and Mayor of London Sadiq Khan leave the St Paul’s Cathedral, where a service of commemoration took place to mark the 20th anniversary of the deadly July 7, 2005, London bombings in which four suicide bombers targeted London's public transport system, in London, United Kingdom, on July 7, 2025.
REUTERS/Chris J. Ratcliffe