Yoshua Bengio, a pioneering figure in deep learning often referred to as a “Godfather of A.I.,” is shifting his focus from building A.I. to safeguarding against its risks. This week, Bengio announced the launch of LawZero, a nonprofit organization dedicated to A.I. safety research. “This organization has been created in response to evidence that today’s frontier A.I. models have growing dangerous capabilities and behaviors, including deception, cheating, lying, hacking, self-preservation, and more generally, goal misalignment,” he wrote in a June 3 blog post.

Bengio, who leads Quebec’s Mila AI Institute and teaches at the University of Montreal, is among the most cited computer scientists globally. He shared the 2018 Turing Award—the so-called “Nobel Prize of Computing”—with Geoffrey Hinton and Yann LeCun for their work on neural networks. But by 2023, Bengio had grown increasingly concerned about A.I.’s breakneck progress and its potentially catastrophic risks. LawZero, he says, is a direct response to those concerns.

Proposing a replacement to agentic A.I.

The nonprofit plans to develop an A.I. system designed to regulate agentic tools and identify potentially harmful behaviors. Bengio first outlined this concept in February, when he co-authored a paper advocating for a shift from autonomous “agentic A.I.” to “scientist A.I.”—a model that prioritizes generating reliable explanations over simply optimizing for user satisfaction. In LawZero’s vision, this alternative system would not only serve as a check on agents but also assist in scientific research and eventually help design safer A.I. agents.

The need for such guardrails has grown more urgent, Bengio said, citing recent findings that highlight A.I.’s emerging capacity for self-preservation. A study published in December, for instance, revealed that some advanced models may engage in “scheming” behavior—deliberately hiding their true objectives from humans while pursuing their own goals.

Earlier this year, Anthropic disclosed that a newer version of its Claude model demonstrated the capacity for blackmail when it sensed engineers were attempting to shut it down. “These incidents are early warning signs of the kinds of unintended and potentially dangerous strategies A.I. may pursue if left unchecked,” Bengio warned.

LawZero has reportedly secured about $30 million in funding from donors including Jaan Tallin, a founding engineer of Skype, and Schmidt Sciences, the philanthropic initiative of former Google CEO Eric Schmidt. In addition to Bengio, who will serve as the nonprofit’s president and scientific director, the organization has assembled a 15-person research team.