Canadian artificial intelligence (AI) pioneers Geoffrey Hinton and Yoshua Bengio have signed onto yet another open letter, this time advocating for the prohibition of “superintelligence” development.
“We … need to make sure the public has a much stronger say in decisions that will shape our collective future.”
Bengio and Hinton have long expressed concern about dangers stemming from artificial general intelligence (AGI), a hypothetical future scenario in which AI achieves human-level intelligence across many subjects. “Superintelligence” refers to one step beyond AGI, when AI surpasses human-level intelligence, a feat many tech companies like Meta are chasing.
The open letter, organized by the Future of Life Institute, has been signed by over 1,500 people as of this writing, including other computer scientists, politicians, religious leaders, and artists.
“[M]any leading AI companies have the stated goal of building superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks,” the letter reads. “This has raised concerns, ranging from human economic obsolescence and disempowerment, losses of freedom, civil liberties, dignity, and control, to national security risks and even potential human extinction.”
Signatories are calling for a prohibition on the development of superintelligence until there is “broad scientific consensus” that it will be done safely and controllably, and “strong public buy-in.”
Bengio launched a non-profit earlier this year, LawZero, to build guardrails for agentic AI systems. Many of its contributors, as well as members of the Bengio-founded AI institute Mila, also signed the letter.
“To safely advance toward superintelligence, we must scientifically determine how to design AI systems that are fundamentally incapable of harming people, whether through misalignment or malicious use,” Bengio wrote alongside his signature. “We also need to make sure the public has a much stronger say in decisions that will shape our collective future.”
In addition to Hinton and Bengio, dozens of Canadians signed the call-to-action, including former Canadian heritage minister Pascale St-Onge, Mila president and CEO Valerie Pisano, and the musician Grimes.
The open letter brings together a unique assortment of backers, including American right-wing commentators Steve Bannon and Glenn Beck, Apple co-founder Steve Wozniak, and billionaire Virgin founder Richard Branson. The Duke and Duchess of Sussex, Prince Harry and Meghan Markle, also joined actors Stephen Fry and Joseph Gordon-Levitt in signing the letter.
Hinton, who won a Nobel Prize for his pioneering work in AI systems, has joined Bengio in calling for restrictions on the development of AI in recent years. The duo have signed multiple open letters warning of the dangers of unfettered AI development. In his Nobel Prize speech, Hinton decried tech companies chasing “short-term profits” and called for “urgent and forceful attention” to combat AI safety risks.
Feature image courtesy LawZero via LinkedIn.