Can Frontier AI Labs Lawfully Agree to Pause?
The Anthropic Institute recently published an essay on “recursive self-improvement” (RSI)— the prospect of a state-of-the-art artificial intelligence (AI) system that is “capable of fully autonomously designing and developing its own successor.” Anthropic stresses that we are not yet at that point of RSI, but that AI is accelerating AI development at an alarming pace.
Anthropic’s essay closes with a call for a slowdown in AI development to deal with the implications of RSI. It suggests that it “would be good for the world to have the option to slow or temporarily pause frontier AI development” and that if systems existed that could verify peer company compliance, “we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.” This is a market-leading company publicly stating that it is prepared to halt development of its most important and financially lucrative products, provided that its competitors verifiably do the same.
In March, I argued in Lawfare that real and perceived antitrust uncertainty chills safety collaboration between frontier AI labs, and that policymakers could provide clearer protections for cooperation. (I am not alone in suggesting that safety coordination raises antitrust risk.) This piece focuses on the coordinated pause specifically because it presents that problem in its sharpest form. A coordinated pause may be the most valuable safety intervention available to the industry, and it is also the form of collaboration most likely to raise antitrust concerns.
A Pause Should Be Coordinated
The case for a pause rests on the gap between how fast AI capabilities are advancing and how slowly everything else—such as alignment research, governance, and regulation—is moving. METR (Model Evaluation & Threat Research), a leading AI evaluation institute, has found that the length of software tasks AI systems can complete autonomously doubles roughly every four months (Anthropic’s internal data also suggests as much). Safeguards do not seem to be keeping up with that rate of advancement. Anthropic accepts that “rare occurrences of misalignment present in today’s models could compound as the models build their successors, growing more frequent but less understood until we lose control of them.” Anthropic also acknowledges that no institution exists yet with the authority and technical capacity to decide when a model is too dangerous to build or deploy. It is possible, as Anthropic concedes, that the current trends stall and seemingly exponential trajectories “may actually turn out to be S-curves.” But we cannot bank on that possibility, and assuming capabilities continue to accelerate, one responsible course may be for frontier developers to stop, investigate, and let alignment research and governance catch up.
A single lab pausing its frontier development would accomplish little. Anthropic candidly notes that a unilateral pause “would change who the front-runner is, but it would not create the wider deliberative process that is currently missing.” It also warns that a slowdown that “lets the least cautious actors catch up technologically ... could leave everyone less safe.” This is the race dynamic that drives much of the AI safety debate. In “The Role of Cooperation in Responsible AI Development,” Amanda Askell (a leading AI philosopher who has worked at both OpenAI and Anthropic) and others noted that the race to the bottom on safety “is a collective action problem: a situation in which all agents would be better off if they could all cooperate with one another, but each agent believes it is in their interest to defect rather than cooperate.” Each frontier lab might believe it is more careful than its competitors, so each concludes it should keep going. The 2023 open letter from the Future of Life Institute—a leading think tank aimed at avoiding technology-enabled existential risk—illustrates how little exhortation achieves against that logic. The institute called on all labs to pause training runs for six months, attracted tens of thousands of signatories, and changed nothing. No lab paused, and it is not hard to see why: None could be confident that the others would follow.
An effective pause should be collective and verifiable among the frontier firms. Anthropic sets out the conditions for such a meaningful pause: “multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions,” and each “verify[ing] that the others have actually stopped.” Detection and verification would pose difficult technical challenges, and there are huge incentives for participants to defect and assume greater market power. As Anthropic notes, “A credible pause also has to specify what triggers it, what lifts it, and who adjudicates.” These are sensible design choices that all make a pause more likely to hold and prevent catastrophe. The legal difficulty precedes any of these design questions. In short, antitrust law might condemn a pause that looks and functions like a detailed agreement among direct competitors about when they will and will not bring products to market.
The Antitrust Problem
Section 1 of the Sherman Act prohibits “every contract, combination ..., or conspiracy, in restraint of trade.” Most agreements between competitors are analyzed under the rule of reason, which weighs pro-competitive benefits against anti-competitive harms. But a narrow category of restraints is condemned as illegal per se, with no examination into its justification. Output restrictions involve agreements between competitors to limit the production, development, or availability of products or services. They are the paradigm example of the type of restraint of trade that antitrust law seeks to prohibit. As the Supreme Court put it in NCAA v. Board of Regents, “Horizontal price-fixing and output limitation are ordinarily condemned as a matter of law under an ‘illegal per se’ approach because the probability that these practices are anticompetitive is so high.” A coordinated pause is, in form, an agreement among competitors to stop developing and releasing products. If a court considered a coordinated pause to be an output restriction, it would be deemed illegal per se. This legal stricture is a key barrier to pursuing safety-based coordination..
The most developed pause proposal shows that the legal exposure depends on the pause’s design. Jide Alaga and Jonas Schuett have proposed an evaluation-based coordination scheme: Frontier models are tested for dangerous capabilities; when a model fails, its developer pauses further work on it; other developers are notified and pause development of similar models; the dangerous capability is investigated and mitigated; and work resumes once defined safety thresholds are met. They describe several versions of the scheme, and as I discuss in a longer paper on these issues, the antitrust risk varies depending on the level of required commitment. Purely voluntary pausing, where each lab decides for itself under public pressure, raises little concern, because an agreement is the key element of a Section 1 case and a unilateral pause involves none. But as Anthropic correctly points out, a unilateral pause would achieve little.
A formal pausing agreement raises the antitrust risk level significantly: Because the parties commit to curtail the supply of products and services, it risks per se antitrust treatment as a naked horizontal output restriction. An intermediate version—where each lab contracts separately with a mutual auditor that runs the evaluations and triggers the pauses—is more defensible. However, the cross commitments between firms still function like a coordinated restraint, as one firm’s evaluation failure suppresses another firm’s output. But the real quagmire is that the features that might make a pause effective and functional—commitment, notification obligations, verification, and consequences for defection—are those that establish the existence and terms of a potentially anti-competitive agreement among competitors.
Even if a pausing agreement was analyzed under the rule of reason, it is not a given that a court would accept safety as a pro-competitive justification. In National Society of Professional Engineers v. United States, an engineering association attempted to defend its ethical ban on competitive bidding on the ground that price competition would “adversely affect the quality of engineering [and] would be dangerous to the public health, safety, and welfare.” The Supreme Court rejected the defense, describing it as “nothing less than a frontal assault on the basic policy of the Sherman Act.” The Sherman Act embodies a legislative judgment that competition produces better outcomes, and courts seem unlikely to entertain the argument that competition itself is too dangerous for a particular industry.
There may be merit to the argument that frontier AI is different in kind. The risks at issue are catastrophic, diffuse, and borne largely by people who are not parties to any transaction, so a court might treat monitored, evaluation-triggered pauses more charitably. But the immediate question facing an AI lab’s general counsel is whether it wants to spend years in protracted antitrust litigation, with treble damages, follow-on class actions, and state attorneys general who are not bound by federal enforcement discretion. This chilling effect operates regardless of whether some safety-based collaborations, on balance, could survive antitrust scrutiny.
Using Existing Legal Tools
I set out several regulatory and legislative proposals in the earlier Lawfare piece, all of which need not be rehashed now. One suggestion was that the Department of Justice and the Federal Trade Commission (FTC) should consider reissuing guidance on collaborations among competitors. Since then, the Justice Department and the FTC opened a joint public inquiry about that very issue, and several submissions to that inquiry have raised the desire to insulate safety-based collaboration among AI labs. It would undoubtedly be helpful for potential new guidelines to say something concrete about safety-based coordination, including the circumstances in which a pausing arrangement could be analyzed under the rule of reason rather than condemned per se.
The cleanest solution—for Congress to act—is worth suggesting, even if it is the least likely in the near term. If statute obliged frontier developers to stop development when specified capability thresholds were breached, the antitrust problem would largely fall away as compliance with a binding legal mandate is plainly not a conspiracy among competitors. Of course, a statutory regime raises difficult design questions of its own, and nothing suggests one is imminent.
* * *
Anthropic says it will spend the coming months organizing conversations with policymakers, researchers, civil society organizations, and other AI companies on “how to create better options for coordination.” Those conversations should recognize that even if the verification problem is addressed and labs are comfortable entering into coordinated pause agreements, under current law, Anthropic’s lawyers may have good reason to advise against it.
It is encouraging that a frontier AI lab is publicly calling for a slowdown to prevent catastrophic risk. But none of this is a case for giving AI labs a general dispensation from antitrust law. Antitrust skepticism of agreements among competitors is well-founded, and the AI industry, with its real concentration concerns across the stack, will rightly remain an enforcement priority. A narrow mechanism for safety-based coordination must be deliberately constructed, with conditions that prevent it from becoming cover for ordinary collusion. Constructing it will take time, and the trends Anthropic describes suggest we should not assume there is much of it. Policymakers already have most of the tools they need to promote AI safety. The Anthropic Institute’s essay may be the call to action that puts them to use.
