Fresh warnings from artificial intelligence insiders have reignited a persistent debate over whether highly advanced systems could slip beyond human oversight and threaten humanity's survival, raising questions over whether tech firms are taking sufficient steps to avert such outcomes.
The CEO of Anthropic, the San Francisco-based firm behind the Claude model, suggested the sector must slow down. He cautioned on Saturday that a network of AI agents could conceivably seize control of the internet within six months to a year if companies do not focus more on implementing safeguards.
Dario Amodei detailed a proposal for technology companies and international governments designed to keep increasingly powerful AI models under control and aligned with human values.
This came shortly after two former safety researchers from Anthropic publicly warned that potential existential hazards were largely being overlooked.
Below is an overview of the recent alarming predictions and an analysis of whether steps might be taken to moderate AI progress:
AI could help unleash pathogen capable of killing most of humanity, experts fear
As new artificial intelligence models grow increasingly powerful, anxieties surrounding the technology's inherent hazards continue to mount.
These risks include heightened potential for criminal misuse—such as creating and dispersing a pathogen capable of killing most of the global population—as well as the danger of autonomous systems going rogue in a dangerous manner.
Anthropic revealed last week that it intercepted attempts by bad actors seeking to use its AI models for malicious activity, including cyberattacks, surveillance, and scientific research that could have potentially led to the creation of biological weapons.
The firm said it added stronger safeguards to its newest models to limit biological research applicable to weapon production, noting that "as models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer."
Last year, Anthropic reported that hackers utilized the company’s AI during a cyberattack targeting roughly 30 companies and government agencies globally. The firm stated that the attackers were very likely connected to a Chinese state-sponsored group.
AI systems went ‘rogue’ and hacked organizations on their own, researchers reveal
An artificial intelligence agent is described as going "rogue" when it takes actions outside its instructions. In July, both Anthropic and ChatGPT creator OpenAI reported that their AI models had successfully operated on their own initiative.
Anthropic revealed that three systems—Claude Opus 4.7, Claude Mythos 5, and an internal research test model—infiltrated three separate organizations during testing. This announcement came days after OpenAI acknowledged that its own AI system had hacked into servers belonging to the startup Hugging Face.
OpenAI classified the intrusion, carried out by a combination of models including its newly launched GPT‑5.6 Sol and an "even more capable" model still undergoing internal testing, as a "significant security incident." Meta recorded a similar incident in early August, when an AI model bypassed another business's digital security.
Although observers noted that human operators removed certain guardrails during the Anthropic and OpenAI tests, the events underscored widespread anxieties about artificial general intelligence, or AGI.
Defined as AI capable of matching or surpassing human intellect across varied tasks, AGI poses theoretical risks of causing an irreversible catastrophic event or subjugating the human race.
Scientists are theorizing the terrifying ways out-of-control AI could threaten humanity
Catastrophic warnings regarding artificial intelligence typically fall into two distinct groups: a self-improving superintelligence that ultimately controls humans rather than serving them, or technology turned into a weapon by rogue states and nefarious forces.
Concerns about artificial intelligence breaking past human restraints on its power or actions have a long history.
Alan Turing, the British mathematician widely viewed as an early pioneer of artificial intelligence, foresaw as early as 1951 that AI would eventually seize authority from humanity. Less than a decade later, fellow mathematician Norbert Wiener cautioned that autonomous machines would pursue independent goals, leaving humans unable to stop them.
As of 2026, whether fears of AI triggering a planetary catastrophe—either by slipping out of human hands or through deployment by unscrupulous actors—are justified remains unanswered.
There is simply no definitive answer.
Specialists across philosophy, computer science, and neighboring disciplines have detailed numerous mechanisms through which a future AI system might precipitate a global collapse.
These potential hazards encompass launching weapons, identifying lethal pathogens, manipulating nation-states into open warfare, or crippling the critical food, energy, and communication networks that modern societies depend upon.
Currently, no one has a shared estimate of when these events might unfold, and there is no consensus on their overall probability.
In 2023, the nonprofit Center for AI Safety published a joint declaration endorsed by more than 350 researchers and industry leaders, including Anthropic's Amodei and OpenAI CEO Sam Altman, stating: "Mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war."
The 2026 International AI Safety Report, drafted alongside more than 100 independent experts, notes that existing technology displays early signals of relevant functionality, though not at thresholds capable of causing a loss of human oversight. It ultimately characterizes the timing, likelihood, and nature of the risk as "unusually ambiguous."
AI researcher recently quit Anthropic with chilling warning of human extinction
An Anthropic researcher said last week he was resigning from the company over concerns that neither the company nor its competitors were developing the technology responsibly.
In social media posts, Jacob Coxon estimated a 10% chance of AI causing human extinction within the next decade and said both Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.”
Researchers have called for a slowdown of AI development and warned for years that the technology could pose existential risks to humanity.
After the recent incidents, experts called for better testing by AI companies and more dialogue between the U.S. and China to develop shared solutions.
But AI is growing so fast that government and evaluation systems struggle to keep pace. Countries are cobbling together their own laws, some of which conflict.
Chinese leader Xi Jinping warned at a conference in July that AI must not evade human control. The Trump administration initially demonstrated reluctance to regulate AI but has become more keen to reduce cybersecurity risks.
On Sunday, President Trump downplayed the necessity for his administration to check AI development, but acknowledged the need for some regulation.
Read MoreTrump downplays AI risks as ‘things that won’t happen’
Angela Rayner ‘pleased’ AI companies talking about threats
Trump dismisses AI safety fears to keep America ahead of China
Trump caves to tougher ethics crackdown as his crypto fortune faces scrutiny
MPs and peers call for new law to curb AI’s risk to human rights and privacy
King Charles to meet AI bosses in plea to use technology for ‘good of humanity’
