Anthropic's chilling SEC warning: 'Runaway' AI model that can slip out of human control almost ready

Anthropic has claimed that it is extremely close to creating an AI model that can keep building itself, becoming so powerful that it completely slips out of human control. Anthropic revealed in its revised S-1/A, an amendment to a Form S-1 registration statement submitted to the US Securities and Exchange Commission (SEC) before the IPO, that the model is capable of recursive self-improvement (RSI), TechCrunch reported.

RSI is seen as a tipping point from which humanity won't be able to pull back what it has put out there in the name of technology. It is when the AI model has grown so good at training itself that its AI capabilities expand by leaps and bounds. Critics call it the point of no return where AI wouldn't care about the negative fallouts of any of its actions to achieve a goal.

Also Read: Anthropic hacked by AI startup comprising former DOGE staff in just 12 hours

What the amendment states

Anthropic's S-1 states, "Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress." It adds that as part of the company's Responsible Scaling Policy, they have plans to "monitor and mitigate the risks as we approach what has been termed ‘recursive self-improvement.'"

Also Read: Trick or Treat -Anthropic's IPO sales pitch is the perfect Halloween scare for AI enthusiasts

What's surprising is that Anthropic admits that while using AI to push forward research and develop new models, it didn't even realise it was creating something so powerful, and that it may soon meet its own definition of RSI. Relying on AI “to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures.”

AI could develop itself faster than human ability to design controls

This means AI is moving ahead faster than humans can catch up, leading to them not being able to timely identify "a significant acceleration in our rate of development until after it has occurred." This scenario can continue to unfold in the future, in what will be a scary turn of events.

But Anthropic doesn't seem to be too alarmed about it, saying that it could update its Responsible Scaling Policy "to keep humans in control of frontier model development at Anthropic." It further clarified that none of the models has crossed the RSI threshold yet, and that it is not thinking about safely using "Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision."

Anthropic's claims can be seen as alarm bells ringing so loudly that humans can no longer afford to sit back and let AI keep growing. The company will update its Responsible Scaling Policy to keep a check. Governments around the world should also take action soon, lest AI gets ahead of us and does what everyone has feared for a long time. Experts have lately been flagging the risks AI can pose to humanity, even wiping us off the face of Earth one day.