Lawfare Daily: Dan Hendrycks on National Security in the Age of Superintelligent AI

[Intro]

Dan Hendrycks: AIs, if you have a superintelligence, it creates superintelligence, broadly speaking. It can make many types of weapons research. You could have some breakthroughs in there, and that could stabilize things. Now, you know, now we could come up with a scheme for a good anti-ballistic missile system, which reduces adversaries second strike capabilities, which totally gets rid of mutual assured destruction, which is totally destabilizing.

Kevin Frazier: It's the Lawfare Podcast. I'm Kevin Frazier, the AI Innovation and Law Fellow at the UT Austin School of Law and a contributing editor at Lawfare joined by Dan Hendricks, director of the Center for AI Safety.

Dan Hendrycks: Torts will add some restrictions to, to, to these models compared to where they are now, but I think that's totally reasonable and what you would need for longer term economic stability.

Kevin Frazier: Today we're discussing his paper, Superintelligence sStrategy, coauthored with former Google CEO, Eric Schmidt and Scale AI CEO, Alexandr Wang.

[Main podcast]

As AI capabilities accelerate, we're not just talking about automation or economic shifts. We're facing potential paradigm shifts in military deterrence, cyber conflict, and global power dynamics. Some experts argue that an AI arms race is beginning to resemble the nuclear age, but with even greater unpredictability.

Our guest today has taken that analogy a step further, introducing a provocative new concept, Mutual Assured AI Malfunction, or MAIM. A deterrence regime where great powers might sabotage each other's AI advances rather than risk, unilateral domination or catastrophic loss of control. Dan and his co-authors, Eric Schmidt and Alexandr Wang argued that we need a structured approach to superintelligent strategy, drawing on historical lessons from nuclear deterrence, non-proliferation, and technological competition.

So Dan, there's so much to unpack there, but one thing I wanna start with is why now? Why did you and Eric and Alexandr choose this moment to release “Superintelligence Strategy?”

Dan Hendrycks: Well, great question. I think we're entering a different paradigm, the reasoning paradigm of AI. So they're improving quite rapidly behind the scenes, and we're seeing some of that publicly, but basically it's improvement in software engineering and other STEM skills and other domains that require a lot of reasoning is going so quickly, so it could be the case that even in the middle of the year AI becomes much more salient. So when people are thinking, what should we do? There's at least something lying around.

So that, that's some of the motivation, as well as the competition between the U.S. and China has become substantially more close. So last year there was a strategy proposed to have a Manhattan Project where the U.S. just, you know, increases its lead and then they build super intelligence, and then they get to tell all the other countries what to do, because that gives them a monopoly on, on intelligence and potentially a strategic monopoly on power.

So consequently this strategy of doing a Manhattan Project of, well, we'll just race ahead of them isn't looking like that'll be as robust. We can't assume that one will have a substantial lead over the other.

Kevin Frazier: The moat is gone, so to speak, and I think perhaps the drawbridge is up to after the Paris AI Action Summit where Vice President Vance made clear that yes, we are racing ahead, and so it's hard not to read that paper in light of these racing dynamics.

With superintelligence strategy in mind, we have so many conversations about AGI—artificial general intelligence—to transformative AI. What does superintelligence mean with respect to these other terms and how should we think about it?

Dan Hendrycks: Yeah, yeah, great question. So maybe let's just first take what is AGI? AGI, you know, people define this differently. Some people would say like, you know, when it can do the things that a typical person can do, intellectual tasks or digital types of tasks, virtual ones. That's the definition, but it keeps changing.

Transformative AI is often meaning when there's high economic impact, so it's not just, do you have the technology, but is it diffused? Because sometimes there might be a lot of, there might be a lot of red tape, that limits diffusion or procurement processes in the government may slow down diffusion substantially.

And superintelligence is when it's basically vastly better than all the world's experts everything intellectually. So it's not just at human level, it's for virtually everything is just way, way, way better. Weapons research, AI research, running a business, everything.

Those are three different distinctions. I submit a fourth that I think for in the year, coming years, it's sometimes more useful to look at specific capabilities. When is an AI capable of doing a particular risky thing? So, is it, does it have expert level virology capabilities? That's quite relevant because that can be used for both good and bad, as for dual use, or cyber capabilities for cyber-attacks. So those come at different times, those capabilities.

So it's somewhat difficult to think about terms like AGI because they're all encompassing. It can be very good in some ways and very bad than others. The intelligence frontier is quite jagged. So it would not surprise me if AIs are publishing papers—are doing the heavy lifting and publishing mathematics papers—before I have a robot that can fold my laundry at, at home and have that diffuse throughout the economy.

So thinking of it in all or nothing way tends to get people caught up and I think for the geopolitics of this, and for when we need to be coordinating, it's often useful to speak about specific capabilities. But I do think, I do think these notions are all useful, including the notion of superintelligence, because it's defined in a way that if you actually do get it, that would be plausibly quite destabilizing.

Kevin Frazier: Yeah, and what I think is really valuable about this framing of superintelligence versus AGI is it seems like everyone wants to debate precisely when and how we may reach AGI. But to your point, if we have a super intelligent system with respect to cyber or with respect to creating biological weapons, that's all we need to be concerned about to then trigger this strategy of mutual assured AI malfunction, and so–

Dan Hendrycks: Well, it would depend. It would depend.

Kevin Frazier: Okay.

Dan Hendrycks: It's for some different capabilities there.

So for, for bio, bio would be very risky from rogue actors, but be less interesting to say, a superpower, for the same reason that we have the biological weapons convention—that these are chemical and biological weapons are poor man's atom bombs, so to speak. And so it would make sense for countries to, or superpowers to cooperate so that those destructive capabilities are not widely proliferated because rogue actors could then more easily threaten their survival.

So for those capabilities, those are relevant for trying to coordinate, but in terms of incentivizing sabotage, I don't think those would necessarily do that. Maybe some form of cyber could, but I think other types of capabilities could be more destabilizing, like AI that helps make oceans more transparent, so that you can see where nuclear submarines are, know where the mobile launchers are.

Or AIs, if you have a super intelligence, it creates super intelligence, broadly speaking. It can make many types of weapons research. You could have some breakthroughs in there, and that could destabilize things. Now, you know, now we could come up with a scheme for a good anti-ballistic missile system, which reduces adversary second strike capabilities, which totally gets rid of mutual assured destruction, which is totally destabilizing.

So those are all possibilities with a super intelligence, and so you need to be betting that actually we kind of explored technology to its fullest, and there aren't going to be much returns from extra intelligence and there won't be breakthroughs that are destabilizing is, is what you would need to be betting on if, if a super intelligence is achieved and if you're thinking it's not going to be destabilizing, which I think is not something people will buy, so.

Kevin Frazier: Right, right. So, thinking of this term, Mutual Assured AI Malfunction, obviously folks are probably quickly drawn to the idea of Mutual Assured Destruction. What is the nexus between these two terms and what is the distinction that you want to draw and emphasize with MAIM as opposed to MAD?

Dan Hendrycks: Yeah. So I, I, I, I largely use the acronym just to, to allude to, to MAD. The, there, there are, of course, many differences; this isn't just a repeat of nuclear history. I mean, I'm not advocating for a Manhattan Project for, for instance, and I think that it would just be extremely escalatory.

You can't do a Manhattan Project secretly in today's day and age. You can't build a trillion dollar data center out in the desert and like, like there's these things called satellites and like people would notice if several hundred top researchers are suddenly missing, like–

Kevin Frazier: I don’t know, Dan, have you been watching Paradise on Hulu? I mean, you just blow up a rocky mountain and dig a big hole and there you go, you put a data center down there.

Dan Hendrycks: Yeah, so the, so there are some disanalogies. We’ll primarily throughout the paper be treating this as a potentially catastrophic dual use technology, because that's what it is, it is dual use. And definitionally, it is potentially catastrophic if it's expert level in virology. If it's has these sorts of cyber capabilities, that's potentially catastrophic. And it has both civilian applications and weapons applications.

And so I think seeing what the case was for bio and nuclear—which are potentially catastrophic dual use technologies— we had to do a game of managing, of capturing the civilian benefits for energy or for products or for healthcare and reducing the, downsides from the weaponization part of it.

I think we're turning to, to MAIM versus MAD. Yeah, I think there's I think there's a deterrence regime. We have, we have deterrence regimes for a lot of things for nuclear, but also for cyber attacks on critical infrastructure between superpowers, meaning destructive ones. Like China won't or is not, you know, —taking down a power grid like they could would not be the most difficult thing for them to do, but we would probably do the same back to them and trying to destroy each other's economies. Like there's so much economic interdependence or complex interdependence that's not very incentive compatible. And with nuclear as well, there's substantial deterrence to have people stave off from doing things that would just collectively harm everybody else.

 So in the case of AI, if there is an opportunity to get super intelligence if one's on the, the verge of it, if one were to initiate a process such that one might have it say a year later, this process could look like the following.

Imagine that you've got an AI that can do AI research really well. Then you can make, you know, 1,000, 10,000, 100,000 copies of these things and run them in parallel. So OpenAI maybe has got like 200 or so researchers. Now they've got 100,000 world class ones. Wow. And they're operating at 100 times the speed of a normal human. They can work around the clock.

That can be quite explosive. This isn't a new point. This is a point many, many decades old people like Alan Turing and others pointed this out. And others have for years, they said, you know, this would be quite explosive, potentially an intelligence explosion. The most three or top three most cited AI researchers, that is number one, number two, number three, in terms of citations, Bengio, Hinton and Ilya Sustkever all think that you might lose control of this process.

That would put lots of other states at risk, so they wouldn't like the sound of that. But if a state does successfully control it, that would also be something that would threaten their survival because then they could use it against them. And potentially reduce their second strike capability. So it could be quite disruptive and destabilizing, and so I think that states wouldn't get away with that. As AI becomes more salient, then states would think to deter this type of possibility and, and forestall it unless it were done under more agreeable conditions that is with, like, clear benefit sharing or something more multilateral. But that would, of course, be much later on.

So that's, that's sort of the, the, the gist of it, but all the, the, the analogy to MAD is just pointing at there's, there's shared vulnerabilities as there is for critical infrastructure and for financial systems too. So it's, it's a broader phenomenon, I think, fairly precedented. I think we'll just see a similar thing for AI.

Kevin Frazier: And I think when folks read your paper, even the kind of lay person version, you all do a great job. You have a lay person version and an expert version, which I love, and I hope more people follow because we should all be having these conversations, but I will admit anytime in a paper you see a game theory diagram come in, I think a lot of people are like, all right, well, Dan, sorry, you lost me, I'm not going back to Economics 301.

But I do just want to highlight that you all have some very practical ideas of what MAIM would actually manifest in. So can you walk us through, for example, so why would we want nations to move their data centers outside of urban areas? What does that have to do with MAIM as a theory and maybe describe some more practical steps you see?

Dan Hendrycks: Yeah. So imagine that states were wanting to create credible deterrents. They want the deterrents to be credible and they want it to be minimally escalatory as well. You know, an extremely escalatory thing would be a kinetic strike. You don't need that though. By default, you can just disable the surrounding power plant. If the surrounding power plant is affecting cities, this would be much more irritating. So in, in, in, in, in, in the nuclear age, there's the concept of city avoidance, where later on it was, let's just have our nuclear weapons point at each other's military bases instead of at each other's cities.

Initially, that was a good idea, so it would be nice if we would have sort of city avoidance for this so that Mutual Assured AI Malfunction isn't sort of mutually assured human destruction. You don't want to overlap between those. You want it to be more, more surgical.  So that, that's an example type of thing that states could do to reduce the. escalation potential of sabotage, given that they will have incentives to make a conditional threat of sabotage against each other.

There are other things that states could do, such as increase their espionage programs of each other's projects. I don't think China really would need to; the U.S. is basically an open book in terms of AI. I mean, here's how they could do it. You can just do a zero day on Slack, the Slack that communicates workspace communications, and they can see basically everything that's going on. Very easy. And so that would get you xAI, that would get you OpenAI, that would get you Anthropic, that would get you Google DeepMind.

They also have, there's a double digit percentage of the top researchers at all of these organizations are of Chinese nationals and they are extortable. This isn't saying, oh, there are spies in the beginning. I'm not saying that, but they're extortable. They have family back home. Whenever they go back home, it might be a condition before they can leave if they divulge some stuff.

So it's, it's the, the information security, I don't think is going to be fixed in that way. But it would make sense for the U.S. to improve its, its ability to, to spy on programs abroad, as well as develop cyber attacks for surrounding for the data centers and for the surrounding power plants to make the deterrent more credible so that one doesn't need to resort to more hostile actions, such as such as kinetic strikes, which I think would be pretty needless.

Now, I should note that when I'm describing this, I'm describing where the world might be at later. I'm not saying that currently China's talking about this sort of thing. I mean, the paper is new. I mean, it came out just very recently. So this, I don't think was on their radar. I'm not describing that this is how they're thinking, and here's the first introduction to the paper.

So I think that AI will become more salient this year as there's maybe some big AI agent advance. And then people start thinking through the implications and becoming more and more important decision makers will be convinced of what my coauthor Eric Schmidt calls the San Francisco school, the school that AI Is, you know, very powerful. AI Is potentially this decade and mass automation or huge automation waves is potentially this decade. Not something that's, that’s very far off, but potentially on the horizon. Even in the next, even in the next few years, you could get AGI, so to speak.

So I think more decision makers will be buying into the San Francisco school later. Whenever that happens, then then I think the strategic implications would be that a potential intelligence explosion, giving you a super intelligence or some really rapid AI research and development, giving you a super intelligence is something that we're not going to let one of.

Each other do because that could suddenly alter the balance of power in, in a destabilizing way, and so it would be prevented.

Kevin Frazier: We're speaking in mid-March, right after the request for information for the AI action plan closed and I'm hoping that somebody included in their submission, hey, Slack. You need to really bolster your cyber-

Dan Hendrycks: No, I don't know. Wait. I don't know if I was not claiming that therefore, this company should be trying to improve their, their infostack substantially to be state proof. The reason is I don't think that's very tractable. Are we going to fire all the Chinese nationals? I think that would totally undermine you as competitiveness. Are we going to require all the people to get security clearances as well? So I think you just get rid of your workforce and then, you know, transplant them in some desert or have them work out of a skiff.

And so I just don't really see that type of thing happening. So what happened in the and then in the nuclear age is things like the Open Skies Treaty, where actually for stabilizing things, they allowed mutual inspection. They reduced information and didn't try keeping things at some select things as secret from each other to improve stability.

So I think very strong state level proof or superpower proof information security, if you'd work on it, would take would be very difficult, would take many years, would undermine your competitiveness and would also further, I think, make things less stable as it happens. So there are other routes. You'd also need to call an iPhone to, you know, reduce the zero days there. You could just spy on some key decision makers on their iPhones and that'd be another way you could get a sense of what's going on. But yeah.

Kevin Frazier: So listeners may be thinking, okay, Dan, you know, I'm, I'm bought in, you've outlined this idea of MAIM and called a lot on states to think through these specific escalatory steps, making sure we don't get too high on the retaliatory measures too quickly.

They may be thinking, what's the responsibility of the labs here? Do you all call on the labs to have any role in maybe scaling things back or slowing things down, or are these racing dynamics between the companies, between these countries something that's kind of just baked into the idea of MAIM being a useful strategy or a necessary strategy?

Dan Hendrycks: I'm assuming the competition will increase at the speed that the companies can proceed at. So I'm not thinking they're going to voluntarily halt. They’re instead channeling competition later stage into other domains. So instead of competing, the first we have super intelligence should be totally stabilizing. You wouldn't get away with it. They can compete in other things like in you know, they could compete in robotics for their economy. They can compete making AI chips. They can, you know, they can, they can compete in drones or what have you. So there are many axes for competition, but the competition doesn't look like the destabilizing sort.

So there's competition between the U.S. and Soviet Union, but, you know, they end up having agreements, and they weren't trying to, you know, trying to get a nuclear monopoly relative to the other later on when both of them had it. This wasn't so they—then it was just a focus of containment with China. I think it's just strategic competition and continued strategic competition, but avoiding avoiding things that could, could destabilize that competition.

Now for, for companies, so, so what are they to do? I mean, I think clarifying, or I think they should, you know, have to test their models for malicious use applications. That seems like it takes like a week or something like that before the release, this is not—adding the safeguards are pretty easy for the catastrophic forms of malicious use like bio safeguards.

So I currently think for chatbot safety—and I speak as I'm advisor for, for XR—so we have our risk management framework. I don't think it's very difficult to implement as it happens. So I think the claim that, oh, there's a huge tension is, is is, is, is overwrought between competitiveness and, and safety adding the safeguards. Like if people think it's not, it's probably because they just don't know what they're doing. That said, this could change later. Imagine we get AI agents. There could actually be some trade off between getting the technologies out there as quickly as possible for everybody and between diffusion and safety largely due to torts.

So if AI agents can scurry around on people's behalf, and if AI companies get their way that, oh, we have no responsibility for what people instruct our agents. If they tell them to go, go harass someone or go hack or something, we, we, although it was on our server and we built the technology, we had nothing to do with it. There will actually be some trade off from, from tort law, which will put some restrictions on it and make them have to, you know, watch things more carefully and this, you know, might mean that the costs will go up a bit because they're gonna try and shift that liability onto an AI insurance company or something like that.

So, torts will add some restrictions to, to, to these models compared to where they are now, but I think that's totally reasonable and what you would need for longer term economic stability. But there's a, you know, a potential tension, but I think it's pretty, I think it's pretty reasonable if, if AIs are having to abide by the reasonable person standard if they're acting as a fiduciary on behalf of individual people.

Kevin Frazier: So we've, we've analyzed one part of the three pronged superintelligence strategy deterrence via MAIM, but you all also discuss nonproliferation and competitiveness. I wanted to drill down a little bit more on nonproliferation. What does that mean in this context? And why do you think this is an important part of that overall strategy?

Dan Hendrycks: Yep. So for other potentially catastrophic dual use technologies—chem, bio nuclear—there was some precedent of like, inspection and making sure it's not the inputs to create the weaponized versions of those are restricted. You've got the U.S. select agents list. You've got the Australia group Schedule 1 chemicals, for instance, and you have the IAEA and others that do verification for the application of fissile materials.

So this is a pretty common thing, and I think for, for AI as well you would want safeguards, making sure you're not proliferating the capabilities, like imagine the future extreme cyber capabilities to random rogue actors, because cyber offense or cyber attacks are offense dominant, with respect to critical infrastructure or extremely high virology capabilities and nonproliferation of rogue actor or two rogue actors of the things that can make those sorts of capabilities, namely AI chips. And these AI chips are, you know, 30000 plus a piece. So it's just like, don't don't, don't sell it to them. Let's make sure we know where the AI chips are at.

And I think this is a place where superpowers can coordinate. It's just the US and China that I don't think it serves either of their interests to proliferate those sorts of capabilities to rogue actors. Rogue actors I'm using as a shorthand for like imprudent or like terrorists or ones without self preservation property or ones that are not particularly prudent or rational.

Kevin Frazier: Right. And just to press on that for a second, though, about whether or not, for example, China might have an interest, one could argue, in, I don't know, allowing some pretty advanced AI systems to make their way into the hands of, let's say, some, some bad actors, some non state actors who want to wreak havoc in the Middle East and therefore draw the U.S. into a Middle East conflict. Is that a potential threat scenario that you think has some, that might detract from that sort of coordination between China and the U.S.?

Dan Hendrycks: So this is why it's useful to think about specific capabilities here. So for like bio, I don't see why that would be an incentive of theirs because those can evolve. I think China's particularly vulnerable, relatively speaking, given the homogeneity of their population to, to bioweapons relative to more diverse countries like the U.S., which is, you know, an extremely grim thing to talk about, but that's bioweapons.

It's a real horror show to think about, but it could, it could depend I, I think on terms of its incentive compatibility, and I think it'd be probably pretty imprudent given that a lot of the advanced capabilities are very tangled up with each other, so it'd be difficult for strategically enabling them with and proliferating some of the capabilities to them while not others. So I, I don't, I don't quite see that, but it'd be, it'd be an interesting thread to put, to, to, to pull on, but I, I think it's interesting to see. Probably not, not to the most concerning compared to.

Kevin Frazier: We'll save it for a future tabletop exercise, but for now, I'm curious, you know, you release this paper with Eric Schmidt with Alexandr Wang, certainly folks that other people are aware of. Is your phone just blowing up? Are senators texting you thanks, Dan, we've got MAIM now. It's all figured out. Or what's the reception been like?

Dan Hendrycks: I think the reception of the national security establishment has been has been good among people who are interested in AI stuff. Of course, there are people who are like AI, you know, what? It's a, it's a hype or something like that. And so there's nothing worth taking seriously there. And I think that will just like not be increasingly harder and harder to defend position as time goes on.

But among people who are more into this, I think the previous state of the art strategy was the sort of Manhattan project which had just like a lot of flaws and is like looking too late and would be like extremely escalatory. The lowest impact that would have the highest, the hastening other, other states is demise or humanity's demise. This is a really risky business. So, I think it's a very clear improvement over that.

So yeah, I was like in D.C. speaking to people about it last week and at the relevant places, like in Eisenhower building and so on. And so, it seems, seems, think, I think it'll I think the main thing is socializing it. And then there's some moment where people are, where the salience of AI has increased and that's the main thing lying around and it's well understood and confusions are taken care of beforehand, so.

Kevin Frazier: Yeah. So is it your assessment that generally policymakers in D.C. are still lacking a little bit of nuance in diving into the specifics of the threats posed by AI where maybe that sort of SF school of AGI is coming, superintelligence is coming, that message hasn't reached the Hill in a meaningful way.

Dan Hendrycks: So I would definitely distinguish between different agencies and different parts of government. So I think like Congress, for instance, is generally more behind on this and it's a lot harder for them to, to do stuff. In terms of the executive office of the president or the, the, we'll say that the White House and surrounding parties, I think they're generally more technically with the program. If they're handling tech, they, they know this stuff fairly well. I think in the National Security Council in the prior administration as well was very plugged into AI and many of their key people assigned were, you know, roughly in the San Francisco schools, so to speak.

Yeah. I mean, I'm, I'm, I'm very optimistic currently about the, the competence on the geopolitics of AI of, of the current administration that's, but you know, different parts of different agencies that aren't really supposed to be on top of AI. I think a lot of them, you know, still don't know as much, but I think a lot of the key actors do.

Kevin Frazier: And a prominent part of the paper centers on the idea of compute governance. And with respect to that, I think there's a connection to things like chip manufacturing at home, domestic chip manufacturing. Are you concerned about some of the pivots potentially away from the CHIPS Act, for example, diminishing the sort of political and financial support for domesticating U.S. production of chips right here in Arizona, for example, and elsewhere?

Dan Hendrycks: So, the, the China spends on the order of a chip's act a year, so roughly 50 billion in increasing their domestic semiconductor manufacturing capabilities. I think, if, if, China invades, blockades Taiwan, there goes the U.S.'s compute advantage right now in terms of the competition between them. They're roughly tied on pretty much all axes, like they've got similar models. They both have access to the internet to scrape. They've got similar data, that they, similar quantities of data that they have access to for, for text data.

They've got, China has somewhat of an energy edge because they can build powerplants and things like that more quickly, but, you know, that's not necessarily the most decisive factor. And the US's main advantage is on chips and compute AI chips. So, because 90 plus percent of the value added, the AI chip supply chain is in. Western allied countries, but the end point of the supply chain is in Taiwan. So if there is an invasion, whoopsies, there goes the US's advantage there.

So I think it would make sense to have a more secured supply chain. Maybe other allies, for instance, could start doing domestic manufacturing themselves. Two, if the U.S. would do less, but I, I would separate between the CHIPS act as implemented and a broader push to try and increase supply chain robustness. I think that's, you know, what a lot of the Trump administration is actually focusing quite a bit on, reducing economic interdependence on key things there.

So that could be possible, but I mean, the, the CHIPS Act for instance, didn't actually focus on AI chips that much. A lot of that money went to Intel, which doesn't even make serious AI chips. So it was largely a reaction to COVID where supply chains got messed up or AI chip, or excuse me, where, where chip supply chains got messed up, but I'm talking about targeting things on AI chip in particular, and I think there, there might be more that might become more salient and there might be more possibility,.

But yeah, that'll be important, otherwise you know, it's 20 percent plus chance that they invade this decade, according to most, according to many forecasters. So some put it at higher. So that's kind of a clear loss condition that we're sleepwalking into. So I should hope that something's done about it. We're not really good at risk management.

Kevin Frazier: Well, speaking of nightmare scenarios folks are probably familiar with your name as the executive director for the Center for AI Safety and you all have done a lot of work spreading awareness of potential catastrophic and existential risks posed by AI. I'm curious, we're in 2025, three years removed from the sort of ChatGPT moment. Are you sleeping better at night or worse at night? Are you more panicky? How have things evolved over time?

Dan Hendrycks: Yeah, I think it's been two years, but the It's a great question. I mean, I think there's more of a strategy now. So that's nice. I think overall things considered speaking, I don't know, maybe I'd be more like if this is, you know, a catastrophe or ends up extremely poorly for all of us or something. Maybe I'll put that at more like 50-50 as opposed to like 80 percent as it was some years ago, but, but I, I think we'll gain substantial information even in the next months of how this is going to be shaking out in terms of the, the capabilities of these AI systems with this new reasoning paradigm, much faster rate of development than with the previous generation.

So we'll see if there starts to be some substantial economic impacts on software development this year, and that that could really affect the conversation a lot. So, yeah, so that's when I get this, you know, has high uncertainty and a lot can change and will change quickly.

Kevin Frazier: And before we let you go, talk more about these important new strategies and frameworks. I'm curious if we see Congress on this struggle to pass meaningful legislation around some of these key risks. Do you continue to think that states have, have a role in maybe putting in some more robust safeguards just with respect to the things you emphasized earlier about just even week long testing of a model to ensure certain capabilities aren't too likely?

Dan Hendrycks: I think most of the risk reduction will come actually at this international or geopolitical level, and I'm not necessarily meaning through like the UN, but just like the U.S. and China and Russia, their sort of dynamics and whether they stabilize things. So one won't be bottlenecked by Congress for this, or there still would be useful role basic stuff like reporting what the capabilities are to the government. I mean, I think it's very difficult.

It's–imagine being in the Situation Room and you heard you got news from CIA that oh, China is a new big project or something. What should we do? And then they're like, well, where where are companies at? I don't know. We stopped requiring that they tell us what's going on, you know, two years ago. Okay, fantastic.

You know, China probably has more insight as to what's going on at the AI companies in our own government. You know, I think that could happen at the state level. I think even Congress requiring that, whatever you're reporting to the EU, you also report to us wherever you're reporting to foreign governments, you report to us. It's pretty reasonable to me.

But you know, obviously some political factions will say this is you're trying to take over the world or something, you know, some, some hyperbolic like that basic information stuff. So, but that could be done through other means. You wouldn't necessarily need Congress for this sort of reporting. So, like through the Defense Production Act, I think many of them would give it voluntarily, but there's just even having the appropriate agencies asking for it.

But I don't think, I think that there is a distraction or there is a, people are really fixated on the AI companies for affecting this, but I view them as basically most of their actions and behaviors of roughly predetermined by their overwhelming, overwhelming incentives and competitive pressures. So they should be like, you should think that they're going to spend 90 plus of their resources and effort on securing a next huge race that they can buy more compute. And that's their primary activity, whatever, whatever is required for that. They're solving for X, whatever lets them raise that, that amount, that that's their main objective.

And then there's some safety noises that some will make along the way in varying degrees, but those won't really make much of a difference. So I think people get very fixated on performative safety things like, oh, they put out more safety research, more safety blog posts. These don't actually make much of a difference.

What'll make a difference is whether states deter each other from the superintelligence type of stuff. Which countries have an edge in AI, like supply chain ones, if they have guaranteed supply chains or not. And if we limit proliferation of potentially catastrophic we'll use capabilities to rogue actors. I think that's a much higher thing.

So there's some role for the companies to play, but I, I, it kind of feels like TMZ or something like that, where it's like some celebrity things of, Oh, Dennis versus Sam versus this is, this is not, they, they, they do not have much ability to act otherwise. They are, they're very constrained by competitive pressures. Their personalities are interesting, but they're, they're, they, they can't, they can't go up against. Those competitive pressures in any very substantial way. So viewed as largely preset.

Kevin Frazier: Yeah. It draws to mind the idea of the transparency reports released by social media companies, isn't exactly changing the game from a content moderation standpoint.

Dan Hendrycks: Yeah. Yeah. They, they've got a better, you know, filter on their cigarettes or something like that. So they're the good one. It's, it's I think people in terms of their pressure, they should be getting on their governments to get on top of this. That's really, instead of whining at OpenAI or wherever to DeepMind, Anthropic, xAI, et cetera, to do more safety stuff or something, which, you know, maybe they'll all pay 1 percent more of their budget at best.

If they'd pressure their governments to, hey, you guys can kind of sleep at the wheel here. And you should get on top of this. That, that I think would be a more effective thing to push for if people are thinking to do advocacy.

Kevin Frazier: Alright, folks. Well, you've got your homework from Dan. Go get it and hopefully he'll give you a good grade. But for now, Dan, we'll have to let you go. Thanks again for joining.

Dan Hendrycks: Yeah, thanks for having me. The paper is at  nationalsecurity.ai.

Kevin Frazier: And we will be sure to include that in the show notes. Until next time, folks.

The Lawfare Podcast is produced in cooperation with the Brookings Institution. You can get ad free versions of this and other Lawfare podcasts by becoming a Lawfare material supporter at our website, lawfaremedia.org slash support. You'll also get access to special events and other content available only to our supporters.

Please rate and review us wherever you get your podcasts. Look for our other podcasts, including Rational Security, Allies, The Aftermath, and Escalation, our latest Lawfare Presents podcast series about the war in Ukraine. Check out our written work at lawfaremedia.org.

The podcast is edited by Jen Patja. Our theme song is from Alibi Music. As always, thank you for listening.