The People Keeping AI Safe Are Quitting. Should You Be Worried?
The Short Version
The people whose job is to make sure AI doesn't go off the rails are leaving. Not one or two. A lot of them. All at once. At multiple companies.
On Monday, the head of Anthropic's safety team posted a public resignation letter saying "the world is in peril." The same week, half of xAI's entire founding team had officially walked out the door.
These aren't interns. These are the people who were building the guardrails.
What happened at Anthropic
Anthropic is the company behind Claude (the AI that powers Claude Code and competes with ChatGPT). It was founded by former OpenAI executives specifically to build AI safely. That's not a side mission. It's the whole pitch.
Mrinank Sharma led Anthropic's safeguards research team. His work included building defenses against AI-assisted bioterrorism and studying "sycophancy," where AI chatbots agree with everything you say instead of telling you the truth. The kind of work you want someone doing.
In his resignation letter, he wrote that employees "constantly face pressures to set aside what matters most." That "the world is in peril. And not just from AI, or bioweapons, but from a whole series of interconnected crises." His plan after quitting? Pursue a poetry degree.
He wasn't alone. Three other researchers also left that week, including Dylan Scandinaro, who went straight to OpenAI as their new head of preparedness. An Anthropic safety person going to OpenAI. Let that sink in.
All of this while Anthropic is reportedly seeking a $350 billion valuation.
What happened at xAI
At Elon Musk's AI company, things are worse.
Six out of twelve co-founders have left. Half the founding team, gone. Five of those departures happened in just the last year.
Tony Wu and Jimmy Ba both resigned within 24 hours of each other in early February. Ba handled research and safety. Wu left as xAI faces regulatory probes after its Grok chatbot enabled mass creation of non-consensual deepfake images of real people, including children.
The work culture tells its own story. The former xAI CFO lasted 102 days while working 120-hour weeks. Roughly 66% of Musk's direct reports across all his companies have departed since 2021.
The pattern
This isn't new. OpenAI went through its own safety exodus in 2024, when nearly half its AGI safety team quit. Jan Leike, one of the most respected alignment researchers in the world, left saying OpenAI wasn't prioritizing safety. He ended up at Anthropic. Now Anthropic's safety people are leaving too.
Every major AI lab has lost senior safety researchers in the past two years. Every single one.
The cycle: AI company promises safety. Company grows. Revenue becomes the priority. Safety team pushes back. Safety team leaves. Repeat.
Why you should care
You use these products. Claude powers tools at Goldman Sachs, Notion, and Uber. Grok is baked into X. These systems are writing code, handling finances, and shaping what you see online.
The safety teams make sure these tools don't generate dangerous content, don't manipulate you, and don't leak your data. When they leave, nobody replaces that expertise overnight.
The models keep shipping. The revenue keeps growing. But the question Sharma raised isn't going away. Are we building the guardrails as fast as we're building the technology?
The people who were closest to the answer didn't like what they saw. And they walked.