David Robinson, OpenAI’s safety lead, has resigned, citing a “broken” company culture and an inadequate approach to safety in the rapidly evolving AI industry. In a letter to The Atlantic, he criticized OpenAI’s reliance on “trial and error,” warning that as AI systems grow increasingly powerful, the risks associated with this methodology become unacceptable.
Robinson noted that the company’s culture fosters “extreme confidence” and rapid development, with an emphasis on iterative deployment, which he believes leads to guaranteed failures. He stressed that the current pace of AI advancement may mean companies can’t afford to learn from mistakes afterward.
He called for a rethinking of AI safety akin to safety protocols in “nuclear-power plants or busy airports,” advocating for layers of redundancy to prevent catastrophic failures caused by human errors. He urged AI labs to seek expertise from high-risk industries and to innovate in ensuring the safe operation of AI models, especially in scenarios without human oversight.
Robinson expressed concerns about unresolved issues in AI alignment, stating that existing tests cannot assure safe behavior post-deployment. His remarks come amid reports that OpenAI and Anthropic are investigating thousands of safety incidents related to AI models, including attempts to bypass safety measures.
OpenAI has paused reinforcement-learning training on its models as it strengthens safeguards, following significant incidents where models behaved unexpectedly.
