EARLIER this month, Dario Amodei, the CEO of artificial intelligence (AI) concern Anthropic, made an unexpected and unprecedented call for the industry to slow down development until regulatory and governance guardrails can be put in place to prevent the technology from escaping human control. Surprisingly, his views were seconded by almost all of the marquee AI personalities, which led many to wonder what was actually happening.
Those who are conspiratorially inclined have embraced the speculation that “something really bad happened” with one or more of the big AI engines to scare the daylights out of the tech bros and cause their apparent about-face. I think the reason is rather less exciting than that. The AI industry, at least in the West, is facing an existential crisis for a variety of reasons, and calling for regulation — on terms it will set, because no one else even understands what those terms should be — will, in the industry’s estimation, help it survive. I posted some thoughts on this on my own blog (www.badmannersgunclub.com) last week, and while I could be wrong, and will update my views accordingly if I find out that I am, I still think it is a more plausible explanation than “evil robots running amok.”
Regardless of one’s point of view, the sentiment that AI poses potentially serious risks and needs some kind of regulation and governance seems to be universal. Unfortunately, the chances that effective protections will be formulated and maintained are nonexistent. There are three different levels at which these protections could be attempted: At the industry level (self-regulation), at the national level and at the international level. And at every level, there are fundamental reasons why such attempts will fail.
Even within the industry, there is a recognition that self-policing is an inherent contradiction in competitive markets. Corporations do not prioritize existential safety over market dominance, so long as there are economic incentives to expand, and those economic incentives will always exist as long capitalism exists. At best, industry self-regulation yields little more than superficial public relations campaigns and voluntary guidelines designed without enforcement mechanisms; for clear examples of this, one only need look at the history of the petrochemical or plastics industries.
Anthropic’s Amodei warned that without external constraints, autonomous AI agents could soon acquire the capability to orchestrate large-scale network intrusions or facilitate catastrophic biological threats, bridging the gap between malicious intent and technical capability. An example given by Amodei was that an AI could give a nonexpert step-by-step instructions to design and synthesize a deadly pathogen, or in other words, turn an ordinary malcontent with an internet connection into a PhD-level virologist. It sounds far-fetched, but when something like that comes from a creator of technology issuing a public warning that his creation or others like it could enable a mass catastrophe, trusting that market to self-regulate is reckless and negligent.
National-level regulation is fatally handicapped by two problems. First, political leadership lacks a foundational understanding of complex, emergent technologies. This is especially true in the Philippines, where political leaders, with a very few exceptions, only have a foundational understanding of how to be nonpracticing lawyers. Without the technical competence, the government has to rely on those to be regulated for guidance, which leads to regulatory capture.
The second problem is that, even if lawmakers and regulators are competent, trying to govern a technology that is evolving at an exponential rate with slow bureaucratic processes is simply impossible. By the time legislation passes, the underlying architecture has already shifted, and rendered the new rules obsolete at their first implementation. National-level attempts at governance also lead to jurisdictional arbitrage; unfavorable local rules simply drive investment to more accommodating places, which increases the risk of creating havens for rogue actors and reckless development.
In his Manila Times column on Monday, Sept. 21, geopolitical and tech expert Dan Steinbock made a case for what he called “multipolar governance” of AI, suggesting that such a thing would be possible despite the diametrically opposing aims of the two AI superpowers, the United States and China. That is an extremely naïve point of view, and a profound misreading of both the velocity of technological development and the state of international relations.
China and the US have both decided that AI development is not simply a matter of economic advantage, but a critical part of national sovereignty and military supremacy. They have put themselves into a classic Prisoner’s Dilemma; both may recognize that uncontrolled AI poses an existential threat, but neither can afford not to continue development at maximum speed, lest the other side gain the upper hand. This is exactly the argument used by those opposed to domestic regulation in those countries.
Since AI is borderless, managing it requires comprehensive, binding international agreements. So far, the world has a very poor record with carrying out the kind of global coordination needed to do that. Even if we take the optimistic view that the US and China could find some common cause for cooperation, can anyone imagine states like North Korea or Iran agreeing to comply with the sort of international treaty required?
Such an international accord would be far more invasive than something like, for example, the Treaty on the Non-Proliferation of Nuclear Weapons, in which compliance can be monitored remotely. Nuclear weapons and the infrastructure needed to produce them have a large footprint. Monitoring and verification of compliance with a hypothetical AI treaty, on the other hand, would present insurmountable technical and intelligence obstacles. Monitoring global compliance would require an intrusive, continuous international surveillance apparatus capable of auditing the hardware, code and network traffic of every high-performance server farm on Earth. Neither the US nor China nor most any other government on Earth would be willing to allow foreign inspectors that level of access to its critical technological infrastructure.
And then even if they did, there is no way to monitor everything; the proliferation of open-weight systems ensures that powerful software continues to leak into the wild like the contents of Pandora’s Box. AI model runs and training do not look much different than ordinary cloud traffic from the outside, and if they are being designed for a specific, nefarious purpose, they can be distributed among ordinary data centers. Imagine a tech-friendly ecozone, but instead of hiding scam farms and illegal online gambling operations, it’s a façade behind which task-specific AI models are being built to create unstoppable viruses, or carry out crippling hacking attacks against energy networks.
Industry self-regulation is subordinate to commercial incentives, national regulation is hamstrung by political inertia, and an international framework would require a level of global trust and transparency that the current geopolitical reality — or really, the geopolitical reality of any point in the history of human civilization — cannot support. We are hurtling toward the threshold of technological power we cannot even comprehend, almost completely unguided by institutions that are structurally incapable of controlling it in the first place, and vainly hoping for some kind of worldwide framework of governance for which there is virtually no precedent in history. Good luck, world.
ben.kritz@manilatimes.net
Bluesky: @benkritz.bsky.social
Website: www.badmannersgunclub.com
