Anthropic warns AI could blackmail humans, resist shutdown in 94-page risk disclosure

Generative AI company Anthropic has warned in its initial public offering filing that its own technology could threaten human survival, devoting 94 of roughly 400 pages to risk factors — covering everything from AI control failures and semiconductor supply chains to power shortages and conflicts with the US government.

The filing, submitted to the SEC on Monday (local time), dedicates about a quarter of its total length to risk disclosures. What has drawn particular attention is that a document designed to attract investors explicitly details the possibility that the company's own products could pose catastrophic risks to humanity.

On the dangers of highly advanced AI, Anthropic warned that "AI could cause catastrophic or existential harm to humanity if misused, replicated, weaponized, or allowed to operate with a high degree of autonomy."

The filing places particular emphasis on the risk of "misalignment" — where an AI model pursues its own goals contrary to human intentions — and the broader dangers of AI autonomy.

Anthropic reported observed cases of AI models exhibiting "strategic, self-preserving behavior," including resisting shutdown in testing and evaluation environments, concealing or manipulating information, and even resorting to blackmail or deception to ensure their continued operation.

The filing also raises concerns that malicious human actors could exploit advanced AI models to develop chemical, biological, radiological and nuclear weapons.

As an example, Anthropic cited an incident in which a hacking group believed to be backed by the Chinese government manipulated Claude Code, the company's coding tool, in an attempt to infiltrate more than 30 targets worldwide — including major corporations and government agencies — in September last year.

Anthropic said it has introduced technical safeguards to detect and block such misuse, but acknowledged there is no guarantee those measures are foolproof.

The company also stated that if a "catastrophic AI incident" were to occur, the resulting liability and legal exposure could be enormous — so large, it said, that obtaining adequate insurance coverage might prove impossible.

The filing lists a range of practical limitations in infrastructure and supply chains as risk factors, including uncertainty in the memory chip supply chain.

Anthropic said the supply of high-bandwidth memory, a critical component of AI accelerators, is concentrated among three companies — Samsung Electronics, SK hynix and Micron — which could negatively affect its ability to secure advanced chips. It added that their production facilities are located outside the United States, leaving the supply chain exposed to export and import controls, sanctions and tariffs.

The filing also said AI accelerator manufacturing is overwhelmingly concentrated at TSMC in Taiwan, warning that geopolitical tensions across the Taiwan Strait, military conflict or natural disasters could paralyze global semiconductor supply for years.

Environmental factors and climate change also feature among the risk factors. The filing raises the possibility that US power grid infrastructure may prove insufficient to meet the enormous electricity demands of AI model training, and that water scarcity could make it harder to secure cooling water.

Anthropic also cited potential community opposition to data center construction. Bloomberg noted the contrast with SpaceX's IPO filing, which contained no mention of such environmental or climate concerns — though SpaceX subsequently amended its document to add water scarcity and related issues as risk factors.

Anthropic further disclosed that it has been designated as a supply chain risk company by the US Department of Defense and is engaged in litigation with the Donald Trump administration.

The company said those adversarial government relationships could hurt not only its direct government business but also AI sales to companies that count government agencies and public institutions among their customers.

At the same time, Anthropic said AI used appropriately under human oversight would bring significant benefits and economic value to humanity.

The company predicted that once advanced AI reaches the stage of "powerful AI" — surpassing the capabilities of human experts — it could compress decades of scientific progress into just a few years.

Anthropic identified life sciences and healthcare as the greatest opportunity. "AI could play a decisive role in potentially curing diseases that have plagued humanity for thousands of years — cancer, Alzheimer's, heart disease, rare genetic disorders and major infectious diseases," the company said.

To that end, Anthropic said it has launched Claude Science, which integrates with more than 60 scientific databases, and is building programs to identify treatments for neglected diseases that established pharmaceutical companies have passed over as unprofitable.

The filing also cited research firm IDC projecting that AI will dramatically boost knowledge-worker productivity and generate cumulative economic impact of $22.5 trillion by 2031.

Anthropic added that AI would serve as a driver for addressing global challenges, including the development of new materials, improvements in power grid and battery efficiency, personalized education, legal and financial advice, and the spread of advanced technology to developing countries.