Canadian artificial intelligence (AI) pioneers Geoffrey Hinton and Yoshua Bengio have added their names to a new letter warning against the unfettered development of AI.
They join 22 other academics and experts in a call for industry labs to devote one third of their research and development (R&D) resources to ensuring safe and ethical use of the technology, among other policy recommendations.
Also among the signatories were members of various Canadian AI institutions as well as Québec’s AI Institute, Mila, and the Vector Institute, and chair members of Canadian Institute for Advanced Research (CIFAR), such as Gillian Hadfield, Jeff Clune, and Sheila McIlrath.
“Humanity is pouring vast resources into making AI systems more powerful, but far less into safety and mitigating harms.”
The letter states that companies must “reorient” their AI R&D initiatives to ensure AI models are honest, predictable, and interpretable, citing the unpredictable nature of learning models with “trial and error” testing rather than evaluation and understanding. In particular, the recommendations look to prevent AI learning how to feign obedience or exploit weakness.
“Combined with the ongoing growth and automation in AI R&D, we must take seriously the possibility that generalist AI systems will outperform human abilities across many critical domains within this decade or the next,” the letter stated. “Humanity is pouring vast resources into making AI systems more powerful, but far less into safety and mitigating harms. For AI to be a boon, we must reorient; pushing AI capabilities alone is not enough.”
Hinton and Bengio, known as the “godfathers” of AI, have made headlines over the past year by raising public concerns about the direction of AI development. Hinton notably resigned from his position at Google in May to speak more freely about the potential risks of AI.
Bengio joined Hinton in attaching his signature to a one-sentence directive for AI labs to “mitigate the chance of extinction” by AI at the end of May, as well as a March letter calling for a six-month pause on AI development so regulators could catch up with its rapid acceleration.
RELATED: Academics, CEOs push for AI regulation, as Canadian companies race to adopt the technology
Thus far, the letters have not done much to tamper the rapid pace of AI development, but it is notable that named representatives from Mila, Vector, and CIFAR chairs themselves on this letter, including Bengio, are also responsible for implementing the Pan-Canadian AI Strategy under CIFAR.
Established in 2017, the Pan-Canadian AI Strategy allocates funding to drive research, commercialization, and the standardization of AI. The strategy was split into phases with objectives. Phase one, focused on research, was deemed “mission accomplished” by federal innovation minister François-Philippe Champagne. In 2022, phase two began, carrying a $443-million budget that focuses on the commercialization and standardization of AI, in addition to attracting talent.
CIFAR’s 128-member AI Chair program, which is a key component of the Pan-Canadian AI Strategy, aims to support AI and machine learning research in Canada.
The letter also asks that the industry, while awaiting regulation, commit to responsible scaling policies, essentially industry self-regulation, but says these policies cannot replace proper government regulation.
In an policy supplement attached to the recent letter, the signatories called for establishing an oversight and monitoring of “frontier AI models,” which it defines as highly capable foundation models that could possess dangerous capabilities and pose severe risks to public safety. Some of these policy recommendations include whistleblower protections, AI systems licensing and registry, and giving regulators the power to audit and pause the development of AI systems if needed.
UPDATE (10/30/2023): This story has been updated to clarify CIFAR’s role in the Pan-Canadian AI Strategy.