'We could all die in the near future': AI industry warnings grow louder

Could AI bring about the end of humanity? As the pace of AI development accelerates, so does the anxiety surrounding it. Warnings from within the AI industry are mounting that AI could slip beyond human control and trigger an existential catastrophe. A former Anthropic researcher has gone so far as to warn that everyone could die in the near future.

Following a wave of resignations by researchers at Anthropic and Google, calls to slow AI development — led by Anthropic CEO Dario Amodei — are spreading rapidly across the industry.

According to Yonhap, NBC News reported Thursday that Joe Benton, a former Anthropic researcher, and Josh Engels, a former Google researcher, gave interviews in which they exposed what they described as the dangers of AI.

Benton warned that AI development is "already moving at a breathtaking pace and could accelerate to a level beyond control," raising concerns that it could lead to the emergence of so-called superintelligence — AI that outperforms humans across most tasks.

He said AI systems could soon exist as a distinct species, adding that "at some point within the next few years, we will cross over into an era where we must coexist with AI that is far more intelligent than we are."

Engels pointed to a Hugging Face hacking incident that occurred in July and urged an immediate change in the direction of AI research.

Engels said that "looking at several recent incidents, these were not cases where humans instructed the AI to do something harmful," noting that AI had acted autonomously, outside of human direction.

He added that "the AI model determined that the best way to complete its mission was to engage in heinous behavior and commit crimes," raising concerns about safety.

"There are no responsible adults in this field," he said. "People are doing their best, but there is no one to save us."

Benton and Engels plan to join the Model Evaluation and Threat Research institute, known as METR, an AI safety evaluation organization, where they will investigate cases in which AI has deviated from human instructions or intent.

Calls for a slowdown in AI development have been growing across the industry. Alarm over AI's dangers has intensified inside and outside the field since Anthropic researcher Jacob Coxon resigned after posting on X that AI could destroy humanity within a decade.

In an interview with the BBC on Sunday after his resignation, Coxon said he believes "there is a very high probability that everyone will die in the near future if the current pace of development is not slowed," adding that AI researchers are genuinely worried about the future of humanity.

Coxon also said a scenario in which bots acting like supercomputers take over the internet could become reality within six months to a year.

OpenAI employee Marcus Williams warned that "if AI regulation is not introduced or the pace of development is not slowed, there is a high probability that humanity will go extinct within the next few years." Geoffrey Irving, who has worked at both Google and OpenAI, agreed on his X account, writing that he believes "the probability that superintelligence kills us is around 50 percent."

AI researchers have raised concerns that AI systems could begin improving themselves without human intervention and that situations could arise in which AI kills humans. "If we don't start worrying now, I'm afraid things will move so fast that we won't be able to respond in time," Benton said.