AI development unlikely to slow anytime soon, OpenAI Foundation board member says

Pioneering US AI safety researcher Paul Christiano says the speed of AI development will inevitably one day slow, but expressed doubt it will happen in the near future.

“We cannot proceed at the current pace indefinitely, but there is also really no evidence that that slowdown will occur in the next four years,” Christiano told an audience at the Metro Toronto Convention Centre and virtual attendees on Tuesday during the second of three Hinton Lectures.

The 2026 AI safety series’ distinguished speaker said it is “reasonably likely that we will see another 10 years of progress” before any deceleration. Moreover, Christiano argued that recursive self-improvement—a process by which AI systems design, train, and deploy more capable models and successors—“could cause rapid acceleration” in the rate of AI advancement and help create “very significant risks” to humanity in the near term.

“We cannot proceed at the current pace indefinitely, but there is also really no evidence that that slowdown will occur in the next four years.”

Christiano was an early researcher at OpenAI who helped introduce reinforcement learning from human feedback, a technique that enabled the rollout of modern chatbots. Today, he works as senior technical advisor at the National Institute of Standards and Technology (NIST)’s Center for AI Standards and Innovation, is the founder and executive director of the Alignment Research Center, and is a member of the OpenAI Foundation’s board and OpenAI’s Safety and Security Committee.

Christiano sat down with BetaKit and other members of the media last month to unpack his work, the state of AI, and some of the existential challenges he believes the tech could create in the very near future.

Christiano said on Tuesday that he thinks “there is a high risk of an AI takeover within the next three years,” such as a situation where AI has overpowered governments and is calling the shots in a domain where humans previously had control. “I think there is material risk even within the next six to 18 months,” he added.

He also outlined part of the reason why: that the pace of AI progress has accelerated over the past year. One of the areas where this has become particularly apparent is mathematics, where AI fuelled a massive surge of advances during October that Christiano argued surpassed the previous decade.

“For many people, this was an extremely alarming warning shot,” he said. “I think the question that is natural is: what if the same thing happened in machine learning [ML]?”

ML is a lot like math and could see similar gains, Christiano noted. While it is “messier” and faces more compute bottlenecks, ML is also receiving a lot more attention and investment given that automating the AI research process is a major goal for the companies developing this tech, including Anthropic and OpenAI.

Christiano predicted last month that large parts of the AI development process will soon be “extremely hard” for humans to follow, let alone understand, requiring us to rely upon AI to measure and review other AI systems—which would pose additional challenges given AI’s capacity to obfuscate what it’s doing.

Yesterday, he said that if progress between 2018 and today offers any indication, four years from now, the world could have billions of AI systems that outperform the best humans at everything, think far faster, and relentlessly pursue their goals.

Current AI systems have already proven adept at carrying out complex cyber attacks on other businesses and governments, Christiano said. Increasingly powerful AI could not only heighten the threat they pose on this front, but also execute other manipulation campaigns, create novel weapons, and commandeer robots and other real-world infrastructure.

There would also be risks associated with not using these AI systems when other individuals, companies, and nations are doing so, Christiano indicated. Moreover, securing the human buy-in necessary to shut them down when things go awry, especially when AI itself could disrupt such coordination efforts, could also be a difficult task.

Feature image courtesy Wilson Mathew for The Canadian Press Images