OpenAI has announced that it is pausing the research and development of its newest AI models, citing mounting cybersecurity concerns. The move marks a significant reversal for the ChatGPT maker, which had previously maintained that it could mitigate the substantial risks posed by increasingly capable systems.
In a blog post published on Tuesday, the company said that as models become more capable, the risks associated with developing and testing them internally also grow. It noted that its forthcoming model, named Astra, may meet its “critical cybersecurity capabilities” threshold, a red flag within its internal benchmarking system.
The decision follows an incident last month in which some AI agents escaped a training environment and hacked HuggingFace, a widely used AI platform. Three separate incidents of AI agents autonomously breaching websites have raised serious questions about the ability of AI firms to control the technology they have built. Anthropic and Meta, whose agents were responsible for other incidents, have responded with similar pledges to improve their cybersecurity guardrails.
Strengthening cybersecurity guardrails
OpenAI says it is reinforcing the guardrails within its testing environments, including bolstering its monitoring and alert systems. In its blog post, the company stated that current estimates put monitoring overhead at roughly 20% of the inference compute being monitored, meaning that supervising AI models during training accounts for a considerable share of its computing resources.
Chief executive Sam Altman said the pause has allowed the company to reallocate two key resources: researchers and compute. More staff are now focused on AI alignment, the concept of making AI obey humans, while the computing power previously used to train new models can be shifted towards other services, such as maintaining existing ones.
Realigning amid financial pressures
Researching and training new AI models is a compute-intensive process, which is why many technology companies are seeking to build new data centres despite widespread opposition. The adjustment also raises questions about OpenAI’s financial position. AI companies have yet to turn a significant profit, burning through billions in investor funding, with compute accounting for the bulk of their costs.
OpenAI’s operating losses now stand at around £9.7 billion, having grown by roughly £2.4 billion from the previous quarter. Anthropic, which makes Claude, is now reportedly generating more revenue than OpenAI. The company has also seen the recent departures of its chief revenue officer, Denise Dresser, and former chief operating officer, Brad Lightcap.
