What is “safe” superintelligence?

​Ilya Sutskever, co-Founder and Chief Scientist of OpenAI speaks during a talk at Tel Aviv University in Tel Aviv, Israel June 5, 2023.
Ilya Sutskever, co-Founder and Chief Scientist of OpenAI speaks during a talk at Tel Aviv University in Tel Aviv, Israel June 5, 2023.
REUTERS/Amir Cohen

OpenAI co-founder and chief scientist Ilya Sutskever has announced a new startup called Safe Superintelligence. You might remember Sutskever as one of the board members who unsuccessfully tried to oust Sam Altman last November. He has since apologized and hung around OpenAI before departing in May.

Little is known about the new company — including how it’s funded — but its name has inspired debate about what’s involved in building a safe superintelligent AI system. “By safe, we mean safe like nuclear safety as opposed to safe as in ‘trust and safety,’” Sutskever disclosed. (‘Trust and safety’ is typically what internet companies call their content moderation teams.)

Sutskever said that he won’t actually build products en route to superintelligence — so no ChatGPT competitor is coming your way.

“This company is special in that its first product will be the safe superintelligence, and it will not do anything else up until then,” Sutskever told Bloomberg. “It will be fully insulated from the outside pressures of having to deal with a large and complicated product and having to be stuck in a competitive rat race.”

Sutskever also hasn’t said what exactly he wants this superintelligence to do though he said he wants it to be more than a smart conversationalist and to help people with more ambitious tasks. But building the underlying tech and keeping it “safe” seems to be his only stated priority.

Sutskever’s view is still rather existentialist — as in, will the AI kill us all or not? Is it still a safe system if it perpetuates racial bias, hallucinates answers, or deceives users? Surely there should be better safeguards than,“Keep the AI away from our nukes!”

More from GZERO Media

Jordan Bardella, president of Rassemblement National or National Rally, gives a speech and flies French flags at a rally in support of Marine le Pen after her conviction on April 6, 2025.

Bastien Ohier/Hans Lucas via Reuters Connect

Jordan Bardella, the 29-year-old president of France’s far-right National Rally, aka RN, has announced his readiness to run for the country’s presidency in 2027 if current party leader Marine Le Pen remains barred from contesting the race.

Ukrainian President Volodymyr Zelensky and US President Donald Trump meet while they attend the funeral of Pope Francis at the Vatican on April 26, 2025.
TPX Images via Reuters

At the Vatican on Saturday, US President Donald Trump sat down with Ukrainian President Volodymyr Zelensky for a meeting the White House described as “very productive,” and which Zelensky said had the “potential to become historic, if we achieve joint results.”

President Donald Trump raises a fist during a ceremony where he signs two executive orders that will lead to reciprocal tariffs against other countries that charge tariffs on US goods.

Andrew Leyden/ZUMA Press Wire via Reuters

Barely three months in, President Trump has bashed America’s closest European allies and spooked NATO into worrying about its survival, taken a chainsaw to US foreign aid programs, pulled the rug out from under Ukraine, threatened to expand US territory for the first time since the 19th century, and started a global trade war that’s pushed protectionism to its highest levels since the Great Depression. That’s a lot for 100 days, and it seems chaotic, but there are a few basic aspects of Trump’s worldview and commitment to “America First” that are consistent and worth understanding.

- YouTube

Fifty years after the fall of Saigon (or its liberation, depending on whom you ask), Vietnam has transformed from a war-torn battleground to one of Asia’s fastest-growing economies—and now finds itself caught between two superpowers. Ian Bremmer breaks down how Vietnam went from devastation in the wake of the Vietnam War to becoming a regional economic powerhouse.

Eurasia Group and GZERO Media are seeking a highly creative, detail-oriented Graphic and Animation Designer who lives and breathes news, international affairs, and policy. The ideal candidate has demonstrated experience using visual storytelling—including data visualizations and short-form animations—to make complex geopolitical topics accessible, social-friendly, and engaging across platforms. You will join a dynamic team of researchers, editors, video producers, and writers to elevate our storytelling and thought leadership through innovative multimedia content.