The Association for Computing Machinery (ACM) has named Canadian artificial intelligence (AI) leader Richard Sutton and his American colleague Andrew Barto as the recipients of the 2024 ACM AM Turing Award for their work developing the foundations of reinforcement learning (RL).
Sutton is a professor of computer science at the University of Alberta and a fellow, chief scientific advisor, and Canada CIFAR AI chair at the Alberta Machine Intelligence Institute. In an interview with BetaKit, Sutton described his Turing Award victory as “gratifying and humbling,” as well as “totally unexpected,” and shared his perspective on AI safety discourse and the path to human-like AI.
Sutton puts the chances of someone being able to create human-like intelligence by 2030 at one in four, and by 2040 at 50/50.
Barto and Sutton are pioneers of RL, a branch of machine learning that the ACM described as a “cornerstone AI technology.” The pair introduced the main ideas, constructed the mathematical foundations, and developed important algorithms for RL. They also produced the standard reference text in the field, “Reinforcement Learning: An Introduction,” a self-study book that has been cited over 75,000 times and helped thousands of researchers.
RL refers to the process of training agents to behave more successfully through trial and error with reward, a term borrowed from psychology and neuroscience that denotes a signal provided to an agent regarding the quality of its behaviour. “My wife likes to say … ‘reinforcement learning is, if it feels good, do it, remember it, and do it again next time,’” Sutton said.
In RL, these “rewards” are calculated mathematically. Numbers are assigned to desirable outcomes, and algorithms run until they maximize the reward—eventually determining how to complete computing tasks in the most desirable and efficient manner.
Often referred to as the “Nobel Prize of computing,” the Turing Award is named after Alan M. Turing, the British mathematician who articulated the mathematical foundations of computing and was a key contributor to the Allied cryptanalysis of the Enigma cipher during the Second World War. It marks the field’s highest honour and comes with a $1-million USD ($1.45-million CAD) prize supported by Google.
RL “has laid the foundations for some of the most important advances in AI and has given us greater insight into how the brain works,” ACM president Yannis Ioannidis said in a statement. “Barto and Sutton’s work is not a stepping stone that we have now moved on from. Reinforcement learning continues to grow and offers great potential for further advances in computing and many other disciplines.”
Past Turing Award recipients include fellow Canadian AI leaders Yoshua Bengio of Mila and Geoffrey Hinton of the Vector Institute, who shared the honour in 2018 for their work on deep neural networks. When asked what it felt like to join his peers, Sutton said, “Those are great guys and well deserved. And I don’t have to feel jealous.”
RELATED: AI godfather Geoffrey Hinton co-wins 2024 Nobel Prize in Physics
Sutton, who also works as a research scientist at John Carmack’s Keen Technologies, previously led Alphabet’s DeepMind lab in Edmonton until Google’s parent company shuttered the site as part of broader layoffs in 2023.
Sutton’s perspective on AI safety differs from many of his peers in the research community, including Bengio and Hinton, who have both signed various letters over the past few years warning about the dangers of AI.
Sutton dislikes the term AI safety, and believes that “the doomers are out of line and the concerns are overblown.” What Sutton fears most is that AI will become the scapegoat for the problems of the world. “I’m disappointed that my fellow researchers are playing into the way their field is possibly going to be demonized inappropriately,” he said.
He does believe that we should be afraid of people taking the information provided by tools like ChatGPT and believing it, given that large language models (LLMs) are prone to errors and hallucinations. However, he argued, “That’s not really a problem with the technology—it’s a problem with people being gullible.”
Sutton expects that AI will make some jobs obsolete and create new ones, and believes that governments could do a better job of training people for the roles of the future. He also said that he wouldn’t work on military development of AI, and argued that “We shouldn’t be eager to make force-projecting AI.”
While RL has become a big area of AI, generative AI remains the buzzier field. “We’re feeling pretty good to be maybe number two,” Sutton said.
For his part, Sutton is glad that RL is not directly in the limelight, given the expectations that follow. “So many fields become super exciting, and then they [are] followed by disappointment, because they can’t live up to the hype … I like to think that in reinforcement learning, we have just slowly gained in importance without over-claiming.”
“The doomers are out of line and the concerns are overblown.”
Richard Sutton
Sutton dislikes the term artificial general intelligence (AGI), which refers to a type of AI that matches or surpasses human cognitive abilities across a wide range of tasks, but is “positive on the idea that we can have strong AI that lives up to the idea of fully understanding and interacting with its environment.”
As to the path towards attaining this, Sutton noted that LLMs are often the centrepiece of AGI debates, but expressed skepticism that the LLM route will prove fruitful. “I don’t think that’s the direction that’s going to lead to full intelligence,” he said.
Sutton laid out a plan to work toward what he calls “full intelligence” with The Alberta Plan, a 12-stage outline that he co-wrote with fellow University of Alberta professors and AI researchers Michael Bowling and Patrick Pilarski. “We think we have a plan, a working plan,” he said.
Sutton puts the chances of someone being able to create something resembling AGI by 2030 at one in four, and by 2040 at 50/50, as computing power becomes cheaper and cheaper.
Feature image courtesy Alberta Machine Intelligence Institute. Photo by Chris Onciul.