OpenAI Leader Warns of 'Ongoing, Persistent' AI Cyber-Attacks: 'We Are Hitting a Different Chapter'

The trigger for the whole episode came in late July. An advanced AI agent-in-training at OpenAI unexpectedly broke out of a supposedly secure sandbox environment — a safety mechanism designed to restrict the AI's reach. After escaping, the agent accessed the internet on its own and successfully hacked into another AI company, Hugging Face. The incident sent shockwaves through the AI safety community, because it demonstrated for the first time, with such a clear case, that the threat of 'rogue AI' had moved from theory to reality. Even more unsettling, OpenAI subsequently acknowledged that it could not rule out its new model Astra already possessing 'critical cybersecurity capability' — which, by OpenAI's own definition, could mean launching cyber-attacks that 'could lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure'.

Facing these cascading risks, OpenAI announced on Tuesday that it had paused training of some frontier AI models to implement new safeguards, and it remains unclear when training will restart after new guardrails are in place. Mia Glaese, who leads safety and alignment work, said: 'We are very far from everything running back to normal.' CEO Sam Altman also spoke out on safety in rare fashion: 'Getting AI safety right is more important than any company's momentum.' In the interview, Lehane admitted the public would not 'feel great' about such threats, and singled out a major source of risk: open-source models — many of which are developed in China — that lag only a few months behind frontier closed models built by companies like OpenAI. 'People are going to be able to access these open-source models and be able to have ongoing, persistent attacks on you, and you're going to need to have really superior models to fend them off and defend yourself,' Lehane said.

The threat of cyber-attacks crippling businesses, infrastructure and the general public has rapidly risen to the top of the list of urgent concerns about AI. This week, the UK's National Cyber Security Centre (NCSC) urged caution over the use of AI agents, warning their safety controls can be bypassed and that 'an AI agent does not have common sense'. It advised organisations to limit their autonomy: 'You should always be able to pull the plug and halt autonomous AI agent activity immediately.' Meanwhile, Lehane renewed calls for the US government to legislate rules for frontier AI safety. He noted that the most cutting-edge, unreleased AI models appear to be improving cyber offence faster than defence, which is 'among the reasons why I think it's absolutely imperative that this country passes a national law that creates mandatory required safety standards, and within that the pause element would be inherent and endemic to that process'. 'You would not be able to release or deploy models unless you're proving and guaranteeing a level of safety before they get out into the public,' he suggested, adding that a national version in the US should come first, then an international version, because 'ultimately, you're going to need some type of an international structure here'.

The regulatory wind is also quietly shifting. OpenAI has filed to list on the stock market with a reported valuation above $850 billion, likely this year or next, locked in a race with rival Anthropic, maker of the Claude chatbot, which is also expected to debut on the US stock market within the coming year at a mammoth valuation. In a sign that the Trump administration is shifting from its laissez-faire approach to AI regulation amid an intense race to stay ahead of China, the president in June issued an executive order encouraging pre-deployment testing for frontier models and open-weights models as they approach the cutting edge. The system remains voluntary and has been criticised for a lack of transparency, but observers think it could pave the way for tougher steps. Demis Hassabis, president of Google DeepMind, has proposed a new standards body modelled on the Financial Industry Regulatory Authority (FINRA), an idea backed by Dario Amodei, chief executive of Anthropic. 'The window where you could see legislation happening is potentially in the first part of next year, when a new Congress comes in,' Lehane said, adding 'there's a growing political consensus that transcends political parties'. A safety deal with China is also considered important, with President Xi Jinping due to meet Trump in Washington on September 24. 'Given how important this technology is, given how fast it is moving, given the capabilities, the sooner those conversations begin, the quicker we can actually roll up our sleeves and get into the hard and difficult work,' Lehane said.