American Intelligence
Dario Amodei, co-founder and CEO of Anthropic, warned that enormous AI fortunes could let a small group of people effectively control government policy. This reminds me of the famous scene in The Big Short that goes something like, "I don't get it. Why are they confessing? They're not confessing. They're bragging." In that same essay, the Anthropic CEO said every one of his co-founders had pledged 80% of their wealth to giving. He did not name the recipients. What the public record does show is the money and policy network already built around AI risk. Let's dig in. Anthropic was last valued at $965 billion and has submitted a confidential draft filing for an IPO. Reuters has reported discussion of a possible $2 trillion valuation, and the man running one of those companies creating this wealth is describing the political danger it could bring.
Dario is right, we should all be worried—particularly about effective altruism, which he represents and has pledged his fortune to. For more than a decade, wealthy donors have paid for AI risk research tests and the careers that carry these conclusions into government. Some of the same people finance frontier companies whose value may now create a much larger donor class. Wealth built in the name of making AI safe could return as power to write the rules for everyone else. That is how a few private fortunes can become a standing source of public power.
Effective altruists have long debated earning to give, pursuing a lucrative career so you can donate heavily to causes you think will do the most good. EA is much broader than AI. It also includes disease, poverty, and animal welfare, but its AI risk wing puts enormous more weight on future generations. Once a hypothetical extinction event outweighs every cost in the present, a pause on everyone else's work starts to look cheap. Do people assigning those odds gain political authority without having to prove the numbers? Just think about how this kind of thinking played during COVID. How quickly was the government willing to suspend your civil liberties and your freedom because of a hypothetical doomsday scenario?
The more extreme version of this argument is laid out by Eliezer Yudkowsky a fan fiction writer and founder of a Reddit-style blog called Less Wrong. They call themselves rationalists, or rats for short and make the argument that AI could destroy humanity. In 2013, Eliezer’s Machine Intelligence Research Institute called its safety work effective altruism. That October, he met at MIRI with Holden Karnofsky of GiveWell and Dario Amodei, who is now the founder and Anthropic CEO. The room already contained an AI risk theorist, a major grant maker, and a future frontier company leader.
Now follow the money. Dustin Moskovitz and his wife, Cari Tuna, are the perfect example of how money moved. They created Good Ventures, whose partnership with GiveWell, Holden Karnofsky's brainchild, grew into Open Philanthropy, which is now called Coefficient Giving.
Open Philanthropy's 2017 report listed a $30 million grant to the nonprofit OpenAI. MIRI reported a $3.75 million grant. Holden Karnofsky joined OpenAI's nonprofit board as part of the grant arrangement. His Carnegie biography says he is married to Anthropic president Daniela Amodei, Dario Amodei's sister. His Carnegie biography discloses financial exposure to both labs through his wife. The documented links are striking: a grant, a board seat, and family financial exposure across two leading labs.
Four years later, Moskovitz was named as an investor in Anthropic's first $124 million round let by Jaan Tallinn, a Skype co-founder and one of MIRI's major donors. Moskovitz also gave tens of millions to Democratic electoral politics. His private investment, his family's philanthropy, and his campaign contributions seem like separate decisions. But, taken together, they show a range of channels available to one technology fortune: build or back companies, fund an idea, and finance politicians. Together, those channels can turn a private worldview into public leverage.
The grant-making program kept growing. Coefficient says it committed about $50 million to technical AI safety in 2024 and spent about $25 million on tests for AI agents. Some of the tests are used by governments and frontier labs. Horizon received early Open Philanthropy funding and supports fellows working in congressional offices, agencies, and think tanks. Its 2023 class counted 20 people. The money now supports research, benchmarks, and a talent pipeline with direct access to the offices that make policy.
Coefficient Giving is frank about its next steps. In a 2025 appeal, it said that it and Good Ventures supply a concentrated share of AI safety philanthropy, and it advised more than 20 significant donors. It explained that policy groups need backers who fit their political setting, including recognizable right-of-center donors for right-of-center organizations. Moskovitz's electoral giving is lean Democratic. The funder strategy is willing to cross party lines. That is how an ideology gains staying power in Washington and how it gets its hands in the mechanizations of power.
Now, AI companies have resources of their own. Anthropic has given $40 million to Public First Action for AI policy education and advocacy. The money cannot be spent on elected candidates, but the real power in DC is found in apparati around elected officials. Anthropic says governments should have the power to slow or block deployment of models it deems catastrophically risky. The lab is building more powerful models and paying to make its preferred public policy response more influential. Amodei has pitched Anthropic's Responsible Scaling Policy as a prototype for regulation.
OpenAI's foundation holds 26% of its operating company and expects to invest at least $1 billion over the next year in programs that include AI's resilience, testing, and standards. That foundation has its own mission and board. Its equity stake shows how a frontier company's value can endow a permanent institution beside that company itself.
Here is the compounding mechanism:
All of this is rooted in the foundational priors of effective altruism and rationalism. The idea that the present needs to be discounted for the future, and anything that serves the greater good, even the death of present people, is fair game.
This compounding mechanism has already been at work for over a decade, but it could accelerate dramatically when Anthropic and OpenAI go public. Anthropic alone has been discussed as a possible $2 trillion listing. A new generation of leftist, EA millionaires will be born with pledges to give a percentage of their fortunes to non-profit groups that influence elected officials.
The agenda has already reached a senator who is the Godfather for the Democratic Socialists of America. In March, MIRI reported that Yudkowsky and its president, Nate Soares, briefed Bernie Sanders. In September, Sanders and Greg Casar introduced a bill that would:
The bill also reaches models that could be foreseeably modified into prohibited systems. For an open-weight builder, government approval could become the condition of publishing a model at all.
Sanders claims this takes on the AI oligarchs. What I see is a bill that would give a new department the power to ration competition, just as the biggest labs, their founders, and their philanthropic institutions acquire extraordinary wealth.
From a political perspective, it makes sense why the DSA would cozy up to the new donor class, but selling it as taking on the oligarchs, when you are getting in bed with the future richest men in the world is very Socialist. Right comrade?
This whole project is designed to travel across party lines. The money moves through philanthropy, private investment, corporate advocacy, and electoral donations. Coefficient has described how to fund messengers who fit different political settings. Public First Action says it works with Republicans, Democrats, and independents. Its strongest political asset is the claim to know which AI risks justify public control. Whoever persuades Washington to accept that claim helps write the rules everyone else lives by. President Trump and JD Vance have been warning about the incentive problem for a while now. In 2025, JD warned in Paris that the loudest calls for aggressive rules often come from companies with an incumbent advantage. This month, he called frontier companies begging Washington to regulate them a possible Trojan horse. President Trump asked why industry leaders would call for regulation that would supposedly bankrupt them. Days later, at the United Nations, he rejected a global scheme to control AI. They are right to ask who benefits from a rule, even when people asking for it say they are trying to save humanity.
My bet is on open weights and a field that stays open. Open weights let people run and inspect models without a frontier company deciding who may have access. Open source can share the code and other material needed to improve them. There are, of course, genuine risks in releasing powerful systems, and builders should be held accountable for concrete harms. The calls for specific, testable rules and enforcement against misuse do not justify a national pause, a standing release veto, or a licensing regime that the largest labs can afford and their challengers cannot.
The AI risk movement now has its arguments in a proposed federal law and its patrons near an extraordinary liquidity event. Congress should reject the Sanders–Casar charter and release-approval scheme.
If frontier founders believe their models pose extraordinary dangers, they should publish the evidence and pay for the independent tests, and shut down development, and maybe even their companies if they are so afraid. They can do that while other Americans remain free to build.
