What is “safe” superintelligence?

​Ilya Sutskever, co-Founder and Chief Scientist of OpenAI speaks during a talk at Tel Aviv University in Tel Aviv, Israel June 5, 2023.
Ilya Sutskever, co-Founder and Chief Scientist of OpenAI speaks during a talk at Tel Aviv University in Tel Aviv, Israel June 5, 2023.
REUTERS/Amir Cohen

OpenAI co-founder and chief scientist Ilya Sutskever has announced a new startup called Safe Superintelligence. You might remember Sutskever as one of the board members who unsuccessfully tried to oust Sam Altman last November. He has since apologized and hung around OpenAI before departing in May.

Little is known about the new company — including how it’s funded — but its name has inspired debate about what’s involved in building a safe superintelligent AI system. “By safe, we mean safe like nuclear safety as opposed to safe as in ‘trust and safety,’” Sutskever disclosed. (‘Trust and safety’ is typically what internet companies call their content moderation teams.)

Sutskever said that he won’t actually build products en route to superintelligence — so no ChatGPT competitor is coming your way.

“This company is special in that its first product will be the safe superintelligence, and it will not do anything else up until then,” Sutskever told Bloomberg. “It will be fully insulated from the outside pressures of having to deal with a large and complicated product and having to be stuck in a competitive rat race.”

Sutskever also hasn’t said what exactly he wants this superintelligence to do though he said he wants it to be more than a smart conversationalist and to help people with more ambitious tasks. But building the underlying tech and keeping it “safe” seems to be his only stated priority.

Sutskever’s view is still rather existentialist — as in, will the AI kill us all or not? Is it still a safe system if it perpetuates racial bias, hallucinates answers, or deceives users? Surely there should be better safeguards than,“Keep the AI away from our nukes!”

More from GZERO Media

Pete Hegseth, Donald Trump's nominee to be secretary of defense, gestures as he leaves a Senate Committee on Armed Services confirmation hearing on Capitol Hill in Washington, on Jan. 14, 2025.
REUTERS/Evelyn Hockstein

Pete Hegseth, Donald Trump’s pick for defense secretary, appeared before the Senate Armed Services Committee on Tuesday. Dems grilled him on allegations of misconduct and his history of opposing women in the military, while Republicans applauded his performance. Is he likely to be confirmed?

President Donald Trump talks with Canada's Prime Minister Justin Trudeau during a North Atlantic Treaty Organization Plenary Session at the NATO summit back in 2019.
REUTERS/Kevin Lamarque

Incoming US President Donald Trumpposted on Tuesday that he will create an “External Revenue Service” to oversee his planned trade tariffs, underlining his commitment to these policies – much to the chagrin of America’s neighbors.

Palestinians inspect the site of an Israeli strike on a beachfront cafe amid the ongoing conflict between Israel and Hamas in Deir Al-Balah, Gaza Strip, on January 14, 2025.
(Photo by Majdi Fathi/NurPhoto) via Reuters

After months of negotiations mediated by the US, Egypt, and Qatar, Hamas on Tuesday accepted a draft ceasefire agreement that could bring an end to the fighting in Gaza – at least temporarily – if Israel’s cabinet approves it.

President-elect Donald Trump speaks during a meeting with House Republicans at the Hyatt Regency hotel in Washington, DC, on Nov. 13, 2024.

ALLISON ROBBERT/Pool via REUTERS

In just under a week, Donald Trump will be inaugurated as the 47th president of the United States. These final days of Biden’s administration mark the very end of the calm before the storm.

- YouTube

A Gaza ceasefire has gained momentum. What is the likelihood a deal will be reached soon? What do I make of a potential sale of TikTok to Elon Musk? What does Lebanon's new president mean for Hezbollah? Ian Bremmer shares his insights on global politics this week on World In :60.

When his daughter was born, Johnny was able to use Walmart’s paid parental leave to spend six weeks bonding with her: “I’m a living example of the benefits Walmart provides.” Walmart’s comprehensive benefits — including paid parental leave, healthcare, tuition coverage, and more — help associates live better at work and at home. With a $1 billion investment in career-driven training and development, Walmart is creating pathways to higher-paying, higher-skilled jobs, so associates like Johnny can build better lives for themselves and their families. Learn why it pays to work at Walmart.

As we look ahead to the next four years, there are many reasons to be optimistic about the role of American AI. The path forward demands collaboration across all sectors, and Microsoft is ready to help chart this course. At Microsoft, we envision a three-pronged approach. First, the US must advance AI technology and infrastructure to maintain its global leadership. Second, it’s critical to expand skilling initiatives to equip the workforce for an AI-powered future. Finally, exporting American AI innovation to allied nations will not only strengthen international partnerships but also bolster domestic economic growth. Realizing this vision requires a unified effort from government, industry, and educational institutions, and Microsoft is committed to playing a key role in this journey. You can learn more here.