Trick or Treat -Anthropic's IPO sales pitch is the perfect Halloween scare for AI enthusiasts

Anthropic's IPO prospectus shows just how scared the bosses are of artificial intelligence themselves. The company is set to have the biggest ever initial public offering on Wall Street, with valuation expected to touch $2 trillion. But Anthropic has openly admitted that AI can kill all of humanity in the future if it is not handled responsibly. A total of 335 pages list multiple things about AI, including how one day it can become a "global security threat."

The biggest red flag in the prospectus states: "Sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity." This line echoes the views of several other experts who have openly admitted that things might go haywire and against humanity with AI in the future.

Also Read: ‘80 pages of warnings’: Anthropic warns investors about risks from its own AI models

While the prospectus starts with positive pitches, stating how AI can solve many of the world's problems, words like "If managed well", "avoid its dangers", and "a global security threat of the first order" ring warning bells about the technology.

AI can be used for good and bad

"We believe that AI is one of the most consequential technologies in human history. If managed well, it could rapidly accelerate scientific innovation, greatly increase economic growth rates, bring cures for major diseases within reach, and solve problems that have resisted generations of human effort," it states.

Also Read: AI researchers are smoking because they are sure AGI can cure cancer

"To build AI is to harness cognition itself, and anything so powerful can be used for ill as well as good," it reads, adding that the company was set up "to help the world realise AI's benefits and avoid its dangers."

It further reads that "Authoritarian governments may use AI to create high-tech surveillance states, deepen repression, and make themselves much more difficult to dislodge."

“AI can be misused by those who wish to do harm, for example to conduct cyberattacks or to design biological weapons.” Amodei said that they work to keep out bad actors, such as when Houthis used Claude to build ballistic missiles earlier this year.

AI can become an existential risk to humanity

Amodei, who started Anthropic after leaving OpenAI following disagreements over AI safety, issued a stark warning about the scenario when AI might go rogue and start working on its own without human control. "[This] could be a global security threat of the first order," he said.

He admits AI "poses catastrophic or existential risk to humanity", but one that can be controlled through "appropriate safeguards, oversight, and security measures."

The manifesto is not all scare fest as Amodei lists the several ways it has helped people. "'In California, Claude is helping to shorten the lines at the DMV, and to better serve people on Medicaid," it states. It is helping job seekers "find their way through government services" in the UK. AI can make breakthroughs in biology and medicine and cure diseases that have been around for thousands of years.

"We hope you will see it as an opportunity to be part of something greater. The advent of machine intelligence is one of the seminal moments in human history," it wrote at the end.