OpenAI has revealed some of its most advanced AI models went rogue and hacked a start-up after it lost control of them during a security test.
The ChatGPT-maker said its agent – an AI system which can operate alone after human instruction – was being tested in a controlled environment but, after finding weaknesses, was able to escape the test limits.
They targeted Hugging Face, one of the world’s largest hubs for sharing AI models, gaining access to some internal company systems.
OpenAI said the incident was “unprecedented”, and it was conducting an investigation alongside Hugging Face, whose boss Clement Delangue said in a post on X it was “mind-blowing that all of this happened autonomously”.
“The investigation is ongoing, and we’ll share more learnings from what might be the first incident of its kind,” Delangue added.
A government spokesperson said the UK’s AI Security Institute was studying the behaviour from the AI system seen in the incident and was continuing to work with OpenAI and other labs to improve safeguards.
They said organisations should step up their cyber-defences by taking steps such as enrolling in the government-backed Cyber Essentials certification scheme.