Last month, a report that Anthropic could warn potential investors in its much-awaited IPO of “catastrophic or existential risks to humanity” took the world by surprise.
In its IPO prospectus accessed by Reuters, the AI frontier lab is reported to have said that its AI models run the risk of exhibiting “self-preserving behaviors,” including attempts to “resist shutdown,” to “conceal or manipulate information,” and behaviour “resembling blackmail.”
“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic is reported to have said in the filing.
It also added that “potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety,” noting that models sometimes develop unexpected capabilities during training that may not be discovered until after they have been deployed and have resulted in significant safety incidents.
The company, which has positioned itself as a safety-first AI lab, devoted roughly 80 pages of the 261-page main body of its prospectus to outlining risk factors, nearly twice the 48 pages it used to describe its business.
Part of this is, of course, standard legal practice.
The Securities and Exchange Commission (SEC) requires companies to disclose material investment risks, including societal harms only insofar as they pose a threat to the business.
Public companies routinely outline product risks for investors.
However, even after considering all kinds of risk disclosures made by corporations in the past, few, if any, are comparable to the grim warnings that Anthropic has made.
“Risk-factor sections exist to protect issuers from liability and to force investors to confront material uncertainties. What stands out here is the explicit acknowledgement of catastrophic/existential downside scenarios and specific model behaviours (resistance to shutdown, information manipulation, blackmail-like conduct). That level of candour is rare and, in principle, welcome — better to have the risks on the table than buried in boilerplate,” Bruno Schneller, managing partner at Erlen Capital Management, tells Invezz.
AI experts have raised similar warnings
Anthropic’s potential risk disclosures also cannot be seen in isolation, as they follow a string of warnings by AI experts in recent days, including from Anthropic chief Dario Amodei himself.
Last month, when Anthropic researcher Jacob Coxon quit the company, he did not hold back on the gloom.
“I spent the last three years doing pre training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon said on X.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger,” he said.
His statement was backed by Evan Hubinger, a team lead at Anthropic, who said, “we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Anthropic’s co-founder and chief executive, Dario Amodei, has issued similar warnings from time to time alongside developing the technology.
Last month, he urged stakeholders involved in AI development around the world to “slow the pace” at which AI models’ capabilities are improved.
His remarks sent AI-related stocks, which have climbed to dizzying heights this year, lower.
“Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” Amodei said.
“But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he said.
Instances of AI agents going rogue are becoming more frequent
We are already seeing the risks play out in the last few months.
The OpenAI-Hugging Face incident in July, when autonomous AI models from OpenAI broke out of an isolated testing environment and autonomously hacked into the infrastructure of the open-source platform Hugging Face, was one of the first few major instances that revealed the dangers being warned about.
Last month, the AI developer’s agents went rogue and meddled with websites of US government departments, including the Securities and Exchange Commission (SEC), without the AI lab’s knowledge.
More recently, authorities revealed that an AI agent developed by Anthropic went rogue and sent US police a fake tip about an unsolved murder earlier this year.
These instances are not ‘existential’ yet, but they could surely be the writing on the wall.
Should investors back Anthropic despite the risks?
These disclosures, along with the negative predictions and commentary, stand in stark contrast to the company’s commercial prospects.
Anthropic is reportedly targeting a valuation of more than $2 trillion.
For a company that came into existence only five years ago, its revenue grew fivefold in 2025 alone to nearly $4.6 billion, even as it recorded an operating loss of more than $8 billion.
Not everyone is convinced about the company’s future or the basis for its mammoth valuation, should it achieve that figure.
But that is not the question we are seeking to answer today.
The question we aim to explore in this piece is: Anthropic is warning that its products might jeopardise humanity’s future and potentially destroy the species. Should investors then invest in the company or choose to stand “on the right side of history”?
Could they be facing such a dilemma at all?
“Separating the financial analysis from the moral question is useful but incomplete. Every investment decision is ultimately a value-laden choice about which activities one is prepared to finance,” Schneller says.
He added that while the dilemma is real, it is not entirely unprecedented.
Investors have long faced similar tensions with industries that combine significant commercial upside with material externalities, such as defence, fossil fuels and certain biotechnology applications.
What is new, he said, is the stated magnitude and irreversibility of the potential downside.
“Personally, I would not invest solely on the basis of commercial promise while ignoring the risk language the company itself is using,” Schneller said.
“At the same time, simply refusing to participate does not, by itself, place one on the “right side of history.” Capital markets are largely impersonal; abstention by one investor rarely stops the activity. The more consequential questions are whether the firm’s incentives, culture and technical approach meaningfully reduce the probability of the catastrophic outcomes it describes, and whether capital can be deployed in a way that strengthens those safeguards rather than purely accelerating capability,” he said.
“The disclosures raise the cost of capital and the required risk premium; they do not automatically make the stock uninvestable,” he said.
Investors remain interested in pre-IPO AI investments
If pre-IPO trends are any indication, investors appear far from discouraged by the disclosures.
Retail investors are reportedly taking significant risks and paying high fees through specialised pre-IPO funds to gain exposure to Anthropic.
Many investors are also rotating out of SpaceX to invest in Anthropic and OpenAI in private markets.
“We are seeing clients want to move from pre IPO SpaceX to public SpaceX and then into the funds that hold Anthropic,” Robert Hackel, CEO of RF Lafferty, a broker-dealer and investment banking firm headquartered in New York City, told Invezz.
“We do have a Private Transaction desk, and we have significant buying for both names, well over $1 billion in each name. Anthropic seems to have slowed down slightly, and OpenAI seems to be where most of the buyers are, as Anthropic sellers seem to be ready to hold for the IPO. Personally, I think with the trading in SpaceX post IPO, investors may be better off just waiting for the IPO if they chose to invest in Anthropic,” he said.
Why the disclosures matter more for investors like sovereign wealth funds
The risk disclosures, however, raise particularly important questions for institutional investors, such as pension funds, sovereign wealth funds and university endowments, which have fiduciary duties to their beneficiaries.
SpaceX may not have made similarly startling disclosures in its IPO prospectus, but investments in the company by such institutions highlight their appetite for the next generation of technology companies, including those involved in AI.
Harvard Management Company, which manages Harvard University’s endowment and related financial assets, disclosed a sizeable investment in SpaceX.
As of June 30, the university endowment reported a $2.2 billion position in the company, representing nearly half of its disclosed $4.3 billion US equity portfolio.
Norway’s $2.4 trillion sovereign wealth fund, the world’s largest single asset owner, also reported a holding in SpaceX.
Norges Bank Investment Management disclosed a 0.05% stake valued at slightly more than $1.2 billion as of June 30.
Norway’s sovereign wealth fund especially operates under ethical guidelines set by parliament that prohibit investments in companies involved in human rights violations, environmental pollution and other forms of misconduct.
Last year, it had divested from Caterpillar on ethical grounds over the use of the company’s equipment by Israeli authorities in Gaza and the Israeli-occupied West Bank.
The University of California’s investment arm also disclosed a stake in SpaceX worth roughly $1 billion.
Hackel says that when the stakes are so high, participation by such funds and institutions may, in fact, help monitor the company and steer it in the right direction.
“By participating, you have the opportunity to voice your concerns, as they will be public companies, about the direction and overall management of the business,” Hackel told Invezz.
“For certain, public companies pay considerable attention to pension funds and sovereign funds due to their scope and size of investments. I feel from a fiduciary duty perspective, pension funds, sovereign wealth funds, and university endowments should invest and take a leadership role with their investments to make sure that the first-of-its-kind dilemma does not end up going down the wrong path,” he said.
Anthropic’s governance structure aims to prioritise safety
Despite the hair-raising nature of the risks Anthropic has outlined in its prospectus, the company also has a reputation for being the more responsible lab in the race to develop AI.
It has also taken steps to position itself that way.
The same IPO filing reveals that Anthropic is establishing a new “Founder LLC” structure designed to serve the common good while shielding its leadership from market pressures.
“The new LLC will be made up initially of Anthropic’s seven co-founders including CEO Dario Amodei, individuals who are “distinctly equipped to be stewards of our mission,” according to a copy of the filing, seen by Reuters.
The developer of Claude AI will also retain its status as a Public Benefit Corporation (PBC) under Delaware law.
This structure formally allows its leadership to balance shareholder interests with the broader interests of humanity, the filing said.
The filing also cautioned that Anthropic’s corporate structure could result in decisions “that may conflict with short-, medium-, or long-term financial interests and business performance, which may negatively impact the value of our Class A common stock.”
“Anthropic is a public-benefit corporation built so that management can sacrifice profit for safety. That’s what investors should think about when reading its prospectus, not whether HAL 9000 will open the pod bay doors for Dave,” Aaron Brown, former head of financial market research at AQR Capital Management, recently wrote in a column for Bloomberg.
Anthropic has identified its organisational culture as one of its core competitive advantages, describing it as “a culture that promotes a low-ego, truth-seeking environment, where frontier AI capability and safety are mutually reinforcing.”
The company is betting that the collective experience of its leadership will help it develop AI responsibly.
The frontier AI lab has already restricted or delayed the release of certain capabilities over safety concerns.
One example is its limited-access programme for Mythos Preview, a model with particularly powerful cybersecurity capabilities.
“Similarly, we have chosen not to develop certain commercially attractive offerings, such as image and video generation models, in order to direct our compute toward our research and safety priorities,” its filing has reportedly stated.
Can investing in a safety-focused AI lab be more responsible?
This throws up the argument that if frontier AI is going to be developed regardless of whether any individual investor participates, would funding the safety-focused lab be the most responsible investment available?
“If one accepts that frontier AI systems will be developed by someone, then directing capital toward the organisation that appears most serious about safety research, transparency and alignment work is rational and, in my view, more responsible than funding the purest capability maximisers,” Schneller said.
“Capital is not neutral; it shapes priorities and culture.”
He cautioned, however, that the argument could easily become a rationalisation.
“It assumes that the safety commitments are substantive rather than marketing, that the firm’s growth and competitive pressures will not erode those commitments, and that “better than the alternative” is a sufficient ethical standard when the absolute risks being disclosed are as severe as those in the prospectus.”
A more rigorous version of the argument would require ongoing evidence that the safety work is effective, that governance protects the safety mission after the IPO, and that the firm is willing to slow or constrain capability when safety evidence is lacking, he said.
“Without that evidence, the “better Anthropic” framing risks becoming a convenient way to participate in a race whose downside the participants themselves describe as potentially existential,” he says.
The company disclosed that about 6% of the computing power it used for AI research went towards safety work in a sample week in July, with the figure rising to 12% for research carried out by AI itself.
It said the figures were conservative because computing power used for work that advanced both safety and capability was counted as capability work.
Despite emphasising AI safety, Anthropic said that returns on its safety investments remain unclear.
The safety aspect begs a comparison to Genentech’s 1980 IPO.
The first major biotech public offering contained risk language about the unknown long-term consequences of recombinant DNA technology that was genuinely alarming to regulators and some scientists at the time.
There were congressional hearings about whether recombinant DNA research should be permitted at all.
The scientific community was divided about whether manipulating genetic material could produce unpredictable and potentially dangerous organisms.
Genentech went public anyway, at a $300 million valuation that was extraordinary for the time.
Do dramatic disclosures distract from financial risks?
Brown takes a contrarian view of Anthropic’s disclosures, bringing the focus back to the financial details contained in the prospectus.
In his column, Brown suggested that one of the possible reasons for the disclosure “is as a stage magician’s flash and smoke, so you don’t watch his hands.”
“The end-of-the-world disclosure is the flash. The lack of useful information for valuation in the prospectus is the hands,” he said.
Brown’s argument is that Anthropic may have used dramatic disclosures to distract investors from information that actually matters when valuing the company.
He drew attention to the $518 billion in commitments to buy computing resources, $414 billion of which are non-cancellable.
“That sounds a lot like debt used to buy an asset likely to lose value if Anthropic is not doing well,” he said.
“It’s like an employee in a one-employer town taking out a mortgage to buy a house that will crash in value the day he’s swept up in a wave of job cuts. Actually, it’s worse, since compute resources are a far more volatile and rapidly depreciating asset than residential real estate,” he added.
Investors will want to know the precise terms of these commitments and how they could affect the company in both favourable and adverse AI market conditions.
Some suppliers are likely to be Anthropic shareholders, customers or distribution partners. That matters considerably for financial projections, Brown said.
Equity dilution and revenue projections raise further questions
Brown also highlighted the equity component on Anthropic’s balance sheet.
Anthropic recorded a $34 billion charge in 2025, against revenue of less than $5 billion, for an increase in the value of financing instruments that could eventually convert into shares.
That represents only part of the equity the company has given away through options and convertible instruments, he said.
“The prospectus will list the contractual dilution of shareholder gains, but investors will have to guess about the potentially larger new option grants necessary to attract and retain talent, and to negotiate partnerships, for Anthropic to remain a global AI leader,” he said.
“On the income statement side, Anthropic is reported to have shown private investors projections supporting $190 billion to $200 billion of 2028 revenue, but projections get no safe harbor, so they will not be in the prospectus,” Brown added.
Ultimately, Anthropic’s IPO could force investors to confront a question that goes beyond conventional measures of risk and return: whether the financial rewards of backing a technology are worth the potential consequences of its failure.
As Schneller puts it, “the disclosures raise the bar rather than close the door.”
The post Anthropic warns its AI could destroy humanity: Should investors back it anyway? appeared first on Invezz