Anthropic has warned prospective investors that its AI models could pose “existential risks to humanity”, including through manipulation, blackmail or resistance to shutdowns. The disclosures appear in an IPO prospectus shared with a small group of partners ahead of a Nasdaq listing expected this autumn, and could complicate efforts to justify a valuation above $2 trillion.

The filing also highlights the cost and financial risks of the company’s rapid expansion: Anthropic plans to spend $518 billion on cloud, computing and infrastructure commitments, and nearly a quarter of last year’s revenue came from two customers. Revenue rose twelve-fold to almost $4.6 billion last year, while operating losses exceeded $8 billion and operating expenses approached $13 billion. Second-quarter revenue this year was $11.5 billion, and the company expects a second consecutive quarter of adjusted operating profit.