Newsroom
An artificial intelligence researcher has quit one of the world’s leading AI companies with a warning that is difficult to ignore: the race to build smarter machines is moving faster than efforts to keep them safe.
Jacob Coxon, 27, resigned from Anthropic on Tuesday after spending three years working on new AI models at both Anthropic and OpenAI, the company behind ChatGPT.
He said neither company was acting responsibly and accused them of “gambling with our lives.”
In a public post announcing his departure, Coxon accused the two rivals of racing toward AI that could improve itself, while still lacking a reliable way to make sure humans remain in control.
He said neither company was acting responsibly and accused them of “gambling with our lives.”
It sounds like the plot of a science-fiction film, but the concern is fairly straightforward.
Today, humans design and train AI. The fear is that future systems could become capable of helping create their own more powerful replacements. Each new version could then help build an even smarter one, speeding up development beyond the ability of people to properly understand or supervise it.
That technology does not exist in its feared form today. Coxon is warning about where he believes the industry may be heading, not saying that current chatbots are about to take over the world.
What made his resignation particularly alarming was that a senior Anthropic safety researcher publicly agreed with him.
Evan Hubinger, who works on keeping AI systems aligned with human instructions and values, said he personally believes there is a greater than 10% chance that future AI could kill humanity within the next decade.
He also acknowledged that Anthropic does not yet have a clear plan for controlling a system more intelligent than humans. At the same time, he stressed that the risk from today’s AI models remains low.
So why keep building it?
According to Coxon, every company fears that if it slows down, a rival will reach the finish line first.
He said many people at OpenAI have not fully understood the possible consequences. At Anthropic, he argued, the risks are understood more clearly, but the company keeps pushing ahead because it believes someone else may behave even less responsibly.
It is essentially an arms race in which everyone is worried about the weapon, but nobody wants to be the one who stops running.
Anthropic says it has safety policies and tests aimed at managing the dangers posed by increasingly capable models. The company has also said its present systems carry a low risk of the catastrophic scenarios being discussed.
Coxon, however, believes promises made by individual companies are not enough. He wants governments to step in or the industry to agree on a coordinated slowdown.
More than 1,300 employees from major AI companies have already signed a statement calling for international tools that would allow AI development to be slowed if it begins moving beyond human understanding or control.
For the average person in Cyprus, this may all sound far removed from asking ChatGPT to write an email or help with homework.
But AI is already moving into workplaces, schools, banks, hospitals and government services. Smaller countries such as Cyprus are likely to import most of these systems, meaning many of the decisions shaping our jobs, privacy and security will be taken elsewhere.
Coxon’s warning does not prove that AI will destroy humanity. Experts strongly disagree over whether such a scenario is likely, or even possible.
But his resignation leaves one uncomfortable question hanging in the air: if some of the people building the world’s most powerful technology admit they do not know how they will control it, who exactly is pressing the accelerator?
*With information from The Verge and Business Insider
