You're right. I think ideally the legislation should be international. Maybe something like the Washington Naval Treaty that set an upper limit on the tonnage and armament of new battleships. Or perhaps more aptly something akin to SALT I & II where older models are taken offline to avoid derelict AI systems from falling into malicious hands and to keep the number from growing out of control. Although this parallel is somewhat weak considering the capabilities of one advanced model are more valuable than 10x models of the last generation.
Theoretical wishful thinking, I suppose, but I strongly believe that corp/govt scale ML research should be treated like advanced weaponry because it isn't a matter of if but when AI will be weaponized (whether the flavor of warfare is physical or informational).
Although of course as with weapons treaties - the major powers would likely tend to be selective in what they commit to limiting themselves in.
The world couldn't even come together on controlling 3D printed weaponry, there's no hope for an arms treaty for AI right now. The "it's not feasible to regulate even if you tried" stance applies too -- you can restrict central actors without much difficulty, and that would work for AI just as well as it works for battleships, but there's a lot of distributed compute whereas there's not a lot of distributed shipyards. Like, you just have to follow what's been done with anime image nets to see that something like GPT-3 is possible for a distributed worldwide group to achieve and is not limited to firms or governments.
Maybe when we have a disaster directly attributable to AI, nations can get on-board with something like the BWC and CWC. Until then, be even more pessimistic. (If you want a fun if rather dry book to read on material technology developments that were in the pipeline a couple decades ago, some of which have come to fruition, as well as some policy recommendations for the technologies that aren't generally good, check out Jürgen Altmann's Military Nanotechnology.)
As a small amount of metal can be added at the end to make the weapon 'legal', that act does little to address the numerous¹ problems beyond being able to sneak a gun past airport security. Hardly an important milestone in controlling anything. It didn't even affect any gun in existence at its time.
But more generally, as we all know, a ban without provisions for enforcement is useless. Compare to the CWC (Chemical Weapons Convention) which I point to as one of the best pieces of international "coming together" via treaty. It includes requirements that member countries submit to inspections from its enforcement body (OPCW) and furthermore that countries can request the OPCW inspects another member country if they suspect non-compliance. It also includes restrictions on transfer of various chemicals in order to incentivize non-member countries to become members so they can purchase chemicals for industrial purposes from other members.
¹ and bigger, if you're modeling this from assumptions where it's a problem at all -- not everyone thinks it is, "an armed society is a polite society" etc.
An AI-risk maximalist would believe AI is a near-term existential threat, with the prospect of total human extinction. In that scenario, the final backstop measure to a rogue country engaging in AI research is using nuclear weapons.
This... obviously... would be very bad. If it escalated to a full nuclear war, it would kill billions of people. But it would leave survivors, who wouldn't be interested in, or be able to, pursuing AI for decades or centuries. Better than the alternative.
Theoretical wishful thinking, I suppose, but I strongly believe that corp/govt scale ML research should be treated like advanced weaponry because it isn't a matter of if but when AI will be weaponized (whether the flavor of warfare is physical or informational).
Although of course as with weapons treaties - the major powers would likely tend to be selective in what they commit to limiting themselves in.