Speaker A
How much are you sleeping? You know, I've never been someone who slept all that much.
Anthropic CEO Dario Amodei discusses AI growth, safety, trust issues with OpenAI, and the future of AI technology in this extended Bloomberg interview.
Key Takeaways
- Rapid AI progress requires calm, rational leadership to manage risks effectively.
- Trust and aligned values are crucial in AI company leadership and partnerships.
- Anthropic prioritizes safe, responsible AI deployment over purely commercial gains.
- Silicon Valley's culture of nonconformity fosters innovation but also demands ethical vigilance.
- Global AI development is inevitable, necessitating international cooperation and oversight.
Summary
- Dario Amodei reflects on managing stress and rapid AI development, comparing it to relativistic time dilation.
- He emphasizes rational decision-making in high-pressure environments and the importance of maintaining calm.
- Amodei discusses Anthropic's growth trajectory and their focus on safe, responsible AI deployment.
- He shares insights about his upbringing in San Francisco and how the local culture encouraged nonconformism and innovation.
- The interview covers Amodei's past collaboration with Open Philanthropy and his departure from OpenAI due to trust and value misalignments.
- He addresses the competitive and ethical dynamics between Anthropic and OpenAI, highlighting the importance of public trust and safety.
- Amodei comments on the disorganization at international AI summits and the challenges of collaboration among AI leaders.
- The conversation touches on Anthropic's enterprise focus, including successful products like Claude Code.
- He discusses national security concerns, including Anthropic's stance on AI military applications and the Pentagon's labeling of the company as a supply chain risk.
- Amodei stresses the inevitability of AI technology development globally and the need for principled, cautious deployment.
Chapters
- 00:00Managing Stress and Rapid AI Growth
- 02:36Exponential Growth and Company Scale
- 05:13Influence of San Francisco and Personal Background
- 08:05Collaboration with Open Philanthropy and Early Debates
- 10:25Departure from OpenAI: Trust and Values
- 12:05AI Summit Experiences and Industry Dynamics
- 20:41Enterprise Focus and Product Success
- 32:31AI and National Security Challenges
Full Transcript — Download SRT & Markdown
Speaker A
Well, let's just say I'm, you know, I'm learning the art of finding ways to relax and sleep through moments of unusual pressure.
Speaker A
It is all moving so fast. How does it feel on the inside? It's this feeling of like the exponential, like, you know, suppose you were to accelerate away from Earth on a spaceship at relativistic speed.
Speaker A
The way special relativity works is, you know, you go to sleep and you wake up and two days have gone by on Earth.
Speaker A
And so you have to deal with two days in one day. And then you go to sleep.
Speaker A
And then because you've continued to accelerate, three days have gone by on Earth, and then the next day and four days have gone by.
Speaker A
And that's a little bit what it feels like. I mean, do you go to bed constantly paranoid about what you'll wake up to?
Speaker A
There are enough clear and present issues that we have to deal with that I'm constantly dealing with those, while thinking about how we can prepare.
Speaker A
But, you know, I don't think paranoia or worrying about what you'll wake up to is productive.
Speaker A
You know, I've looked at people in history who've dealt with these very high-pressure situations and, you know, you need to learn to respond rationally and not put dangers out of proportion to each other.
Speaker A
This yo-yoing between, I'm not worried and oh my God, we need to panic today.
Speaker A
I think that's a hallmark of immature decision making. And the actual mature decision making is we can't ignore this.
Speaker A
We can't be complacent. In fact, it's getting to be a bigger and bigger risk.
Speaker A
But, you know, we have to respond rationally. You know, like a surgeon would deal with an operation or, you know, like a military officer would deal with a military operation or, you know, someone making decisions that affect a lot
Speaker A
of people has to make those decisions rationally and they have to understand the risk, but they have to maintain a basic sense of calm.
Speaker A
So my son yesterday was like, can I use your Claude Cowork account? And I was like, absolutely not. I need my tokens.
Speaker A
We're seeing more and more of them, even in the consumer space. We wanted to be more of an enterprise company.
Speaker A
But, you know, even consumer without us putting that much effort is starting to go fast.
Speaker A
You are at the center of the AI universe right now. What does that feel like?
Speaker A
The interesting thing is that the experience I've had for my whole career and certainly the whole time at Anthropic is that there's this kind of smooth exponential and the experience, the smooth exponential is, nothing's happening, nothing's happening, nothing's happening.
Speaker A
A little things happen and then zoom, it goes crazy. That's the experience of the world.
Speaker A
That's the experience of the scale of the company compared to the other companies and compared to the world.
Speaker A
So, you know, I was watching this graph for a while and I said, oh yeah, we'll probably become the AI company with the most revenue and the most valuation sometime around this time.
Speaker A
And indeed, indeed it has happened. So in one sense, I'm not surprised because there's just a smooth line on the graph.
Speaker A
But of course in another sense, when things actually happen, you just see so much more detail and color to it and it definitely is surprising.
Speaker A
And what we're just keeping in mind, all the things we usually keep in mind, which are, which are just, you know, how do we train good models?
Speaker A
How do we put them in good products? How do we make sure that everything's safe?
Speaker A
How do we help people but also manage the societal risks around the technology? It's all the same questions, just kind of under a bigger microscope, as it were.
Speaker A
What were you like as a kid growing up in San Francisco? I know your dad was a leather craftsman.
Speaker A
Your mom worked in libraries. How did that shape you? You know, the whole, you know, like first internet revolution was happening around me and I had absolutely no interest in it.
Speaker A
I was just interested in like doing math and like scrawling things, I was interested in understanding the universe.
Speaker A
I was interested in science fiction. Like that was kind of the general milieu.
Speaker A
I think I just felt a lot of curiosity about the world. You grew up in the town where that is the center of technology and right now it's the center of AI, you know?
Speaker A
Is there anything about this place, this city here that informed your worldview? Yeah, I mean, I think the general spirit of kind of nonconformism and individualism and it's okay to be crazy.
Speaker A
I think a good deal of that probably did rub off on me.
Speaker A
You know, you hear these stories about, you know, you go to countries in Europe or even other parts of this country where it's just kind of discouraged or considered weird to like
Speaker A
think about things in some different way, right? Or have some set of crazy ideas.
Speaker A
And there's a lot of things I'm actually very critical about with Silicon Valley.
Speaker A
But one thing that I think is good about it is this encouragement of like, you know, it doesn't matter if all the experts are against you, it doesn't matter if you have a coherent vision and a coherent worldview of the world,
Speaker A
you should go and pursue it. Maybe it just won't work at all. But if it does, there's this kind of long-tailedness to it where there are certain places you can search certain veins of work
Speaker A
where you might find a huge goldmine there. I think that spirit is very important.
Speaker A
You, Daniela, your sister and her husband, Holden Karnofsky, lived in a group house together back in 2016.
Speaker A
What were you debating back then? That was, I think the time when Open Philanthropy Project was first starting up, which Holden was the lead of.
Speaker A
And I was at that time like a biological scientist. So I was helping them with some of the stuff they were doing around kind of developing world health or biological research.
Speaker A
So I kind of advised on that stuff and, you know, what were the areas that were promising? What were the areas that were less promising?
Speaker A
Your decision to leave OpenAI has become Silicon Valley lore.
Speaker A
What really happened? Like, beyond the narrative, what were the issues? What did you disagree on?
Speaker A
Look, I'm going to say it very simply. You know, there are many difficult issues that you face when you're building powerful technology that Anthropic faces every day where we don't know whether we're making the right
Speaker A
decision or the wrong decision. So there are many valid disagreements to be had on safety.
Speaker A
We certainly had some of those disagreements with them, but people that alone is not sufficient to leave.
Speaker A
People here have had disagreements with me, people here have disagreements with each other. But when you feel that you can't trust someone, when you feel that their values are not what they say they are, when you feel that they're not honest, when you feel
Speaker A
that they're not in it for the reasons that they say, when you see disturbing patterns of behavior and dishonesty, that makes it very hard to continue to work with a company, to continue to trust the company.
Speaker A
And look, at the end of the day, why argue with someone when you don't have the same vision and you don't trust them?
Speaker A
Like the way to resolve it is you go off and do your thing.
Speaker A
They go off and do their thing. And I am completely at peace with the idea that we're doing thin...
Speaker A
We'll see who wins in the market and we'll see who wins in the court of public opinion.
Speaker A
I think those things speak louder than any drama about why, who left, what, you know, we're, we're providing an example of how to deploy this technology, you know, in what we think is a responsible way.
Speaker A
If they disagree, they should make that argument. And you know that, I think that's really all there is to say about it.
Speaker A
There was a moment at India's AI summit where you and Sam Altman appeared to refuse to hold hands on stage. What happened there?
Speaker A
What happened is that the summit was extremely disorganized. We all came up at the last minute and they like changed the order in which we were standing, and then like, they took a picture of us and then they ordered us all to like, hold hands.
Speaker A
You know, if you've ever been to one of these summit, I am not saying anything bad about India in particular, but like all of these kind of international type summits that have like heads of state are like super disorganized.
Speaker A
Okay? But everyone else held hands. Come on. I, I look, I don't know, I don't know what to tell you.
Speaker A
Okay. There was like, you know, Narendra Modi up there suddenly telling everyone to, like suddenly telling everyone to hold hands.
Speaker A
All right, all right, well, okay, look, Sam and Elon are suing each other. You don't like Sam. I, it seems if the people building the most important technology in the world can't hold hands on stage, how can we trust you'll cooperate on existential risk?
Speaker A
So here's, here's what I will tell you. There is a wide variance in the quality and the trustworthiness of the people building this technology.
Speaker A
I think this meme that, you know, different, that no one trusts each other, I don't think it's right.
Speaker A
You know, I've known Demis Hassabis who builds the Gemini models. They're a competitor to Claude models.
Speaker A
I've known him for 15 years. We've worked together on like, you know, a number of issues.
Speaker A
We buy compute from Google, we swap safety ideas all the time. So, you know, my my view of this is that one, there are some players who are more trustworthy than others.
Speaker A
And you know, I think there are players outside Anthropic who, you know, who, who I trust, who I see as trustworthy.
Speaker A
What I think needs to happen is that the trustworthy actors need to need to get together and, and put the untrustworthy actors in a position where they kind of have to adopt the same standards.
Speaker A
With a lot of experience, I've learned that there are some folks who don't do the right thing on their own, but if there's a majority of the industry that's doing the right thing, then I think the rest of the industry is, is kind of, they're left in a position
Speaker A
where there's not much they can do that, that, that then come along. There's like the positive version of it where you inspire other people, that's like Demis and me inspiring each other.
Speaker A
You know, he does Alpha Fold. We're trying to do something in bio as well, right?
Speaker A
We do interpretability research, they start an interpretability research. It's not even competition, it's just, it's just, you know, each company does something cool and the other company's like, that's cool.
Speaker A
We'd like to, you know, do that too and see if there's something new within that we can do.
Speaker A
So that's the kind of, you know, the carrot side of the race to the top.
Speaker A
Then there's the stick side or the implicit stick where you're like, okay, these guys are doing the right thing.
Speaker A
Those guys will look bad if they don't do the right thing. And often we see behaviors where they kind of grudgingly do the right thing while trying to pretend they're doing something different.
Speaker A
And there's something bad or sinister about us that is to be expected. But I think that's the way we get the industry together, and that's the way we get the industry to cooperate.
Speaker A
Now, early on, others focused on fun, splashy consumer apps. You made a bet on coding and enterprise and Claude Code is a hit.
Speaker A
Claude Cowork is a hit. Why did you make that bet? Was it a values decision or a business decision?
Speaker A
When we started Anthropic, the thing that the, the base thing that mattered, the thing that always matters is we wanna, we want to do this, right?
Speaker A
But then you have to ask yourself, okay, in order to fund the very expensive, you know, creation of these models, it, it needs to be a company, it needs to have a business model.
Speaker A
Does the business model get in the way of the values? There's always this question, but I think one of the things I learned is, you know, just from being at other companies and watching other companies is, look, if you pick a business model that fundamentally conflicts
Speaker A
with your values, you're gonna have a hard time, right? Either you betray your own values or you become irrelevant.
Speaker A
You know, you kind of end up in a catch 22 situation. And there are ways out, there are ways to dodge, but it's just, it's just a hard situation.
Speaker A
It's far better to pick a business model that is compatible with your values. And so when we thought about it, we said, look, you know, we've seen the world of social media, the consumer world, it, it really seems to, you know, encourage engagement,
Speaker A
even addiction, you know, the slop we've seen with AI video models, it's like, what's going on?
Speaker A
It wants to maximize the number of minutes that you're, you're paying attention to, because that's the advertising revenue driven incentive.
Speaker A
Whereas if we look at enterprise, look, I mean, you know, we want to make these models useful to people.
Speaker A
If I think of all the positive things you can do with AI, right? I warn a lot about the negative things, but ultimately we think the positive things will outweigh the negative things.
Speaker A
Many of those are basically fall under the banner of enterprise. You know, we want to use AI to, you know, cure diseases that we couldn't cure before, right?
Speaker A
Well, that's working with biotech, it's working with pharma, it's working with academic research groups.
Speaker A
All of those are enterprises, right? We want to use AI to like, you know, to make energy cheaper and more efficient.
Speaker A
That's, that's all enterprise. You know, we want to use AI to help with education.
Speaker A
Most of that is enterprise. You know, we want to use AI to, you know, to address, you know, health and developing world. Well there are nonprofits.
Speaker A
But those are basically enterprises. We want to increase economic growth. That that is basically enterprise as well.
Speaker A
And then I think there's another factor, which is that enterprises care a lot about trust in long-term relationship, right?
Speaker A
Consumer can have this, you know, almost this gimmicky aspect to it, right? Where with enterprise, it's like, what matters is you build a relationship where, you know, you work with, you work with a company for many years, you know, you deliver on what you say they deliver on
Speaker A
what they say, and they basically trust you. And so it's very synergistic with our goal of, you know, deploying these models in a positive and safe way.
Speaker A
And so I think it serves us well to have this business model that largely aligns with our values.
Speaker A
Not that there aren't conflicts sometimes. Not that there aren't hard choices we have to make, but I think the number of such choices, it's much lower than it would be otherwise.
Speaker A
A developer can switch from Claude to ChatGPT or Gemini in an afternoon. Is it really possible to have a long-term lead in this industry?
Speaker A
And, you know, how long would it take a serious competitor to replicate what you've built?
Speaker A
Model quality is the most important thing. Like we're, we're very far ahead right now on model quality.
Speaker A
There is some amount of inertia, but I've never relied on that, right? I've never relied on like the, you know, the, Anthropic has never relied on like, oh, this is sticky and people won't switch.
Speaker A
I think you wanna have a better model. You want to have a better product.
Speaker A
And you know, we, we see the growth rates haven't inflected at all. If anything, they've gone up at least, at least at the time of taping this interview.
Speaker A
So, you know, I think I, I I tend to think that is the most important thing.
Speaker A
Soon after Claude Cowork was released, $285 billion in market value vanished overnight, traders called it the SaaSpocalypse.
Speaker A
If AI continues improving at this pace, how much of traditional software gets replaced and how fast?
Speaker A
Yeah. So, you know, this is, this is one of these questions, it's kind of very hard to predict in advance, right?
Speaker A
If you could predict it perfectly in advance, then people would, and they'd make a huge amount of money on the market and they'd always be right.
Speaker A
So, you know, no one, no one knows exactly what's going to happen. But I would note a few things, right?
Speaker A
All of these traditional software companies have a number of moats. I think what's gonna happen is some of these moats are gonna go away, but others are going to stay around, right?
Speaker A
The ability to quickly write software, I, I definitely think that's going away, right? If, if your moat is, we wrote this complex software that no one else can write, like good luck, you're not gonna be able to defend that.
Speaker A
But I think folks have customer relationships. Folks have know how of how, you know, of how the field works.
Speaker A
Folks have unique domain knowledge. So I think my advice to all of these folks is, obviously, you know, don't be complacent, don't ignore it.
Speaker A
Make a list of all your moats and be very aware that some of them are going to go away, while others are going to become relatively more important.
Speaker A
Because there are limiting factors and there may also, there may also be new moats.
Speaker A
And I think those that deftly respond, that, you know, lean into the list of moats that are still present as well as the new ones will do well.
Speaker A
I think those that are complacent, that kind of, you know, just delude themselves at what worked in the past will, will continue to work.
Speaker A
They're, they're not gonna have a good time. So that is, that is the advice I would give.
Speaker A
And you know, I I, I, I think at the end of the day, I would guess, I mean, it depends what you call SaaS and what you don't call SaaS, but like, I would guess that the software industry gets larger, not smaller.
Speaker A
Although there, there will be some big losers. Explain that. I just think the pie is getting bigger, right?
Speaker A
Like, I think, I think with ai, like the pie is getting bigger, the existing incumbents may be smaller in relative terms.
Speaker A
Some of them may, may go down in value, some of them may even may even go out of business if they don't adapt in the right way.
Speaker A
But I, I, I, you know, I think you, I think you see this often when growth is really fast, right?
Speaker A
If the, you know, if the, if, if, if what's possible with AI grows by 10x, it's very easy for an existing incumbent industry to go up by 1.5x, right?
Speaker A
Just, just, you know, not as much as the whole big pie is growing. So I think that may happen that that's not to say we won't have such some big losers.
Speaker A
I think those who don't adapt, who put their heads in the sand, who don't kind of see what's coming, who don't identify the moats they have, they're gonna have a really hard time.
Speaker A
Your biggest backers are companies like Amazon and Google and Microsoft and Nvidia. These are companies that all have their own agendas.
Speaker A
They are partners in rivals. You have huge commercial milestones tied to funding, who's really calling the shots.
Speaker A
There have been a number of cases where we've really spoken our minds about what we think, you know, I've been very outspoken about the need for export controls on ships to China, right?
Speaker A
I, i, I say this because I think it would be really bad for America, for, you know, the state of democracy in the world for, you know, China to be ahead in AI capabilities.
Speaker A
And, you know, it's, it's like some of the chip makers obviously don't agree with that view, but it hasn't stopped me from saying it.
Speaker A
I'm saying it again now, even after we've signed more partnerships, what they know is that we always work with them.
Speaker A
We've been good partners, you know, we can work together. I'm sure they wish we didn't say these things, but these things are what I believe.
Speaker A
What are you gonna do? You know, they're, they're at the end of the day, they want the, you know, they, they benefit from these deals as much as we do.
Speaker A
You know, look, we're all adults here. We can work together on one thing while disagreeing about another thing.
Speaker A
Bloomberg's reported that you're at valuations that are higher than OpenAI. We're talking nearly a trillion dollars for a five-year-old startup.
Speaker A
How do you make sense of that number and why do you need that much money if you, you know, you're more disciplined on compute, you have a faster path to profit, The compute is ramping up very quickly, right?
Speaker A
So it, it can both be the case that the fundamentals of the business look good, but in, you know, in a year you'll have three times as much compute as, you know, three times or four times, or, I'm not gonna give exact numbers,
Speaker A
but like these compute ramps are very fast. And we have every expectation that the revenue, you know, ramp will meet and exceed those.
Speaker A
But raising money is, is kind of the buffer against this cone of uncertainty. So it's a totally rational thing to do.
Speaker A
It's, it's a very small dilution to the business, and it logically is not at all the same thing.
Speaker A
In fact, it's compatible with the opposite as you know, that there's anything wrong with the fundamentals of the business.
Speaker A
There have been reports of server strain, reliability issues, people complaining about running out of tokens.
Speaker A
You've said other companies are yolo-ing on infrastructure. Do you actually have what you need or are you playing catch up?
Speaker A
So one of these things about compute is there's a marketing compute, right? So, you know, my view is that over a period of time, even longer than a couple months, like, you know, we can get large amounts of compute.
Speaker A
One, one thing that's worth saying here is, you know, I don't think we bought too little compute by any reasonable standard.
Speaker A
So, you know, we were planning for a 10X a year growth in compute, 10X a year is what we expect.
Speaker A
That isn't what we've seen over the first quarter of 2026. We saw a greater than three x growth in revenue quarterly, just in not annualized three x in the quarter, which of course, three to the fourth power is 80x
Speaker A
over the course of the year. We didn't plan for 80x annualized growth. It would not have been rational to plan for 80x annualized growth, because that means if you only get 10x, you know that you, you have eight times less.
Speaker A
So we're, we're, we're in a locally extreme, you know, explosion of compute that's not gonna continue.
Speaker A
If that continued, you know, you just get to revenue by the end of the year, you get to revenue numbers that no company on earth, I don't think that's gonna happen.
Speaker A
It just, it just can't. But you can have these short periods where it's like, oh my God, like, you know, this is faster growth than we ever, ever possibly anticipated.
Speaker A
But I don't know, you saw the compute deals with Google, you saw the compute deals with Amazon.
Speaker A
You know, there are more that we kind of can and will do, like, you know, the market's liquid.
Speaker A
Like if, if, if, you know, if you're able to use compute really well and there's the demand, you'll get your compute.
Speaker A
It might might just take a month or two. Does it feel good to surpass your arch rival?
Speaker A
Look, I, we have a lot of difficult challenges in front of us, and there's this race to the top idea that we're trying to pull other companies along with us.
Speaker A
And I think we've seen that. We have pulled them along with us. Sometimes they don't admit that that's what they're doing.
Speaker A
Sometimes they copy us while they're attacking us. But, but this pull is very valuable.
Speaker A
And so I think the value of being the preeminent company, both commercially and in terms of models, you know, it's, it's not about beating rivals for the sake of beating rivals.
Speaker A
It's, it's about having the ability to pull the ecosystem along with us. And we hope that we can do more of that in the future.
Speaker A
But winning has to feel just a little bit good. I mean, look, we're always trying to succeed, right?
Speaker A
Like, we're always trying to, you know, we're not, we're not trying to fail here, right?
Speaker A
Like, I'm not someone who believes we should shut this technology down. We shouldn't build it like, you know, we, we, we, we, you know, we, we exist within a free enterprise system and, and, you know, there's, there's nothing,
Speaker A
there's nothing wrong with this. We just have to mitigate the risks of the models, right?
Speaker A
And, and so it's always been the balance between the two. Now for most of Anthropic's history, you were the underdog.
Speaker A
I imagine it's easier to take the moral high ground when you have nothing to lose at this scale.
Speaker A
How hard is it to stay true to your values? What I would say is that, you know, I've put a lot of time into thinking about how, how that's the case.
Speaker A
You know, as, as, as companies scale. You know, I've been paranoid at every scale, at every scale of the company.
Speaker A
There's some new challenge, there's some new way the company can lose. Either its, its kind of will to win just commercially or kind of the core of its values.
Speaker A
I'm, I'm worried about both because I see them as synergistic. I actually see the fact that we've been able to make such good models as the thing that, that allows us to assert our values in a way that works
Speaker A
as the company grows, as it gets bigger. There are lots of pitfalls here. There are lots of ways to go wrong.
Speaker A
Not because me or the co-founders or the company's leaders values change, but because the composition of the company changes very fast.
Speaker A
So I spend probably half of my time just talking to the company about the culture of anthropic and how the culture works, right?
Speaker A
When you're growing this fast, you're hiring a bunch of people from big tech companies.
Speaker A
If you don't tell them how Anthropic operates, they'll simply recapitulate the only thing they know, which is how to operate at the companies that they came from.
Speaker A
And so this is a constant struggle and a constant challenge. And, you know, it's like, you know, me and Daniela's, maybe number one top priority is, is figuring out how to preserve this because we recognize that this is the core of
Speaker A
who we are in the long run. Your product velocity is insane. You're shipping so much so fast. How are you doing it?
Speaker A
I would say two things. The first is, you know, we have a unified company.
Speaker A
We have a unified culture. You know, I think we've gotten, you know, grown larger while still being incredibly efficient.
Speaker A
Everyone's still being on the same page, like just the cultural and organizational unity. I would say that's the biggest factor.
Speaker A
And I would say the second biggest factor is Claude itself. That we're now using Claude to help, you know, develop our models and, you know, make them more efficient and quickly develop products.
Speaker A
There's all kinds of new practices you have to develop. You know, we're, we're still new, new, we're still new at it, but you know, it's producing, it's producing a lot of acceleration and increasingly producing reliable acceleration.
Speaker A
And so those are the two factors I would point to. Will you tell me the most wild thing you've seen AI do?
Speaker A
I think some of the wildest stuff I've seen is around biology and medicine. I've seen a number of cases, including Daniela actually, where Claude diagnosed a medical problem that, you know, a bunch of fancy doctors had missed.
Speaker A
And on the biology side, like the models are starting to get surprisingly good at like, you know, you know, tasks like drug design or, you know, computational chemistry or things like, and I'm just like, wow, you know, as someone
Speaker A
who used to be a biologist, I look at it and I'm like, wow, that's hard.
Speaker A
Like, you need a lot of training to do that. And like Claude is getting good at it.
Speaker A
And that's one area where I think we're gonna get a hell of a lot of benefit.
Speaker A
Like that's the positive for AI. We're gonna get these huge, enormous benefits. Life is gonna get better.
Speaker A
The quality of human experience is gonna get better. A century of scientific progress, A a century of scientific progress and a century of progress.
Speaker A
And what it's like to be human. Like a go back to 1900, think of all the problems we had in 1900s.
Speaker A
All the reasons people died prematurely, all the problems they had to suffer, all the material deprivation that we don't have to deal with today.
Speaker A
Then think of another hundred years of that. I really believe this century of scientific and medical progress, if we can get through this, and I, I think we will.
Speaker A
I'm increasingly optimistic. We're gonna have a much, much better world. I know how much you love writing.
Speaker A
You're known for your essays. Do you use Claude to help write? I do. I have not gotten to the point where I actually allow text directly written by Claude in, in, because I, i, i, I just have such a specific style
Speaker A
that I'm, I'm a little picky about it, but I basically use Claude to like, you know, to help me brainstorm, to help me think through the themes to help me kind of, oh, you know, what are some references I could use for this?
Speaker A
So it it, it kind of plays a supportive role. I don't know how far we are from Claude being able to write better than me.
Speaker A
We're not quite there yet. But, but you know, I think, I think certainly it's coming.
Speaker A
I love writing too, and I, you know, I feel like writing it helps you struggle through ideas.
Speaker A
There is a lot of critical thinking involved in that. Do we lose that if we let Claude do it for us?
Speaker A
I, I, I'm a little worried about that. And in fact, that's half the reason I write myself.
Speaker A
It certainly is for external audiences. Many people read what I write, but it, it, it is just as much to clarify my own thinking so that I kind of know what to do next and to create a common reference point across me and others.
Speaker A
I think we're still grappling with the question of how exactly do we use AI in a way that kind of preserves those benefits.
Speaker A
I think the thing I'm doing now does that where I use Claude for research and I use Claude for kind of, you know, how do I help organize my own thoughts.
Speaker A
I think if we just used it end to end, like write an essay about the risks of ai, first of all, it wouldn't write the things that I think, but also I would, I would exactly lose that benefit.
Speaker A
There's some way, as the models get better, I think probably to, to use them directly much more directly in the writing and yet still preserve those benefits.
Speaker A
But I think it's gonna be a subtle thing. It won't be all one thing.
Speaker A
It won't, we'll have to kind of figure it out over time. I think we could have this very unusual combination of very fast GDP growth and high unemployment, or at least underemployment or, you know, low wage job.
Speaker A
Lot of low wage jobs, high inequality. You've been really direct about job loss. AI could eliminate half of all entry level white collar jobs in the next one to five years.
Speaker A
That was a year ago. AI has moved incredibly fast. Is it still 50% or is it higher?
Speaker A
I've always said, and you know, if you go back to those original clips, they always get like, you know, cut out of context and like the three seconds.
Speaker A
But like, you know, the, the real statement was always, I don't know what's gonna happen, but this is an order of magnitude for how crazy things could be.
Speaker A
Also, I always talk about all the things we can do in response to this, right?
Speaker A
I've talked about token tax and working with enterprises to adjust people, and I'm a little skeptical of retraining programs, but like, we should throw them in the mix macroeconomic policy, even from the beginning.
Speaker A
I always talked about solutions, but you know, somehow there's this tendency in the human psychology to clip the three seconds of like, doom is coming.
Speaker A
So my message is just definitely not doom is coming. My message is like, this is something, you know, that we should see coming, that we're worried about and that we need to actually respond to positively.
Speaker A
You know, I don't know exactly, but I'm, I'm still pretty concerned. I'm still the same order of concern.
Speaker A
You know, we are seeing right now that AI is making people more productive. But that's the usual hump.
Speaker A
If you go back, you know, to the kind of industrial, you know, I wrote about this in Adolescence of Technology.
Speaker A
You automate 90% of the job, great, people are 10 times more productive in the other 10%.
Speaker A
'cause they're 10 times more leveraged. But eventually it gets close to a hundred percent.
Speaker A
Now the sequel to that is, well then you have to find something else for them to do.
Speaker A
I, I don't know about the long run, I'm truly uncertain about that. But I do think there are types of adaptation.
Speaker A
Like one thing I'll talk about is, you know, software engineers within Anthropic. We're, we're going through this transition right now where, you know, right now AI makes the software engineers more productive, even though AI writes all the code or almost all the code.
Speaker A
But still it makes people more productive. But we're already starting to see the beginning of like, you know, there may be some people that it's, it's not making more productive that it's better for the AI to just, to just do the thing.
Speaker A
So that's one side of it. The other side of it though is what do we need more demand for?
Speaker A
You know, there's something we call a forward deployed engineer or in like applied AI solutions architect where their job is a mix of technical work and talking to customers.
Speaker A
There's a lot of demand for that because there's a lot of customers and we're growing very quickly.
Speaker A
Now, does every person who is in the pure software engineering quite work for the this other?
Speaker A
No, you know, it's not perfect. It's not one-to-one. That gives you a flavor of, there's gonna be a hell of a lot of disruption, but things will also adjust. Which wins out?
Speaker A
I don't know. But the reason it's important to warn about it is that that's how we can respond.
Speaker A
That's how we can make policy, right? Both within Anthropic and macroeconomically for the whole world.
Speaker A
We wanna put out carefully considered thoughts. We don't wanna say things that people don't believe we'll actually do.
Speaker A
We don't wanna say things that are half baked. We want to think carefully about what should actually be done about these, these problems.
Speaker A
You put out this chart showing potential job disruption like sales, finance, you know, which jobs go away, who gets replaced and what new jobs are created.
Speaker A
So no one knows for sure, because you know, the economy's unpredictable, right? It's the same as the stock market, right?
Speaker A
They're these kind of decentralized processes that you, you don't really know ahead of time what are the pieces of the job that people are still gonna be able to do.
Speaker A
But what I would say broadly is that, you know, anywhere that you have, you know, these kind of entry level white collar, you know, whether it's banking, whether it's finance, whether it's, you know, there's, there's, you know, there's, there's gonna be a lot
Speaker A
of potential for AI to first make people more productive. But you know, then, then, then there's gonna be, you know, then, then there's gonna be a wholesale AI can do the job and then we're gonna have to think about, well, you know,
Speaker A
what is it that people can do? And I think we need to plan about that ahead of time.
Speaker A
We're already doing it. When we talk to enterprise customers, we see choices that they face.
Speaker A
They face the choice of, you know, should I save costs? Which often means hiring less people, basically do the same thing with less resources, or should we do more things with the same amount of resources?
Speaker A
And we always, when we can try to push them to doing more with the same amount of resources, because basically that means like, hire the same number of people or maybe even more people, but just do, do kind of, kind of do new things.
Speaker A
Pushing them towards the positive sum. The thing that that we have going for us here is the pie is gonna expand a lot.
Speaker A
And so, because the pie is gonna expand a lot, there are probably going to be places where people can go.
Speaker A
It's just a matter of finding them fast enough. It's the size of the disruption. It's, it's gonna be big.
Speaker A
And that's what I'm warning people about. But we kind of, we have to solve that matching problem.
Speaker A
And So play this out for me a little bit. You know, you wake up in five years, what does this country look like?
Speaker A
What are those people doing? Yeah, so 'cause if there's that much unemployment, is that not how revolutions start?
Speaker A
Yeah, no, this is the outcome we wanna prevent. This is absolutely the outcome we want to prevent.
Speaker A
You know, I think, I think there's, I think there's a few places, none of them are guaranteed.
Speaker A
We're not sure, but there's the physical world, right? Like things that are in the physical world, yes, there's a robotics revolution as well, but it's a lot slower than what's happening in AI.
Speaker A
People always talk about building data centers, but like when processing information of any type becomes a lot easier, maybe the restriction is gonna be things in the physical world.
Speaker A
And so we need a lot of more people to make, build, manufacture things in the physical world.
Speaker A
Anything that's human centered, I think that's gonna be a big deal, right? I hear all these stories about AI found something that my, like my doctor couldn't find and I feel, but like, but there's a, people really wanna talk to other humans,
Speaker A
particularly over kind of important things, right? Maybe AI can do better customer service, but nevertheless people, or at least some people wanna talk to humans.
Speaker A
So these kind of human relationship driven jobs, like, I think those are gonna be important, right?
Speaker A
And I think there'll be some effort by the humans to kind of direct the ais right?
Speaker A
At, at some level it has to be in line with someone's values and someone's intentions and, and so I, I think there's gonna be some role there, although I don't know how thin versus how thick it will be.
Speaker A
And I think it's very hard to say. There has been a lot of pushback, and I know you've said you're trying to warn people, but that, you know, you're, you know, Jensen Huang said you're conflating tasks with jobs.
Speaker A
Other folks have said this, you know, it's sort of doom marketing that benefits Anthropic.
Speaker A
So, so I wanna be really clear and push back hard against this, the whole picture of there are risks to job loss and here are some ideas.
Speaker A
I mean, we haven't fully fleshed out the ideas because I want to get them right, but Anthropic has come up with lots of ideas.
Speaker A
We've had economic grants, we have the economic index. I talk about the, the possible ways to address these risks from tax and macroeconomic policy to what the new jobs are in the adolescence of technology.
Speaker A
I lay out, you know, I have like five pages where I lay out the difference between tasks and jobs.
Speaker A
Why this time is different than other times. A list of six different things we can do from private philanthropy to government action.
Speaker A
I talk about the problems, I talk the solutions, but social media, which I detest, which I detest as a category, people have these three second clips from, you know, from a year ago.
Speaker A
They don't actually read the essays or they prey on the idea that social media, I've, I've written much more carefully about these things where, where I talk about the risks, the idea that this is cheap marketing is itself cheap marketing.
Speaker A
This is, this is laziness, this is failure to engage with serious intellectual work. And, and I think that is part of the problem.
Speaker A
Again, I think it's, it's part of the disease of Silicon Valley. It's been caught up in this social media world of, of, of three seconds.
Speaker A
And so people only respond to it or they think they only have to respond to it again.
Speaker A
I think it's very dangerous and we failed to have a mature conversation. Instead, people just lazily see this like three second clip and, and they're like, oh, this is what Dario was saying.
Speaker A
It's, it's so stupid. It's so unserious. And whenever someone says something like that, I take them less seriously.
Speaker A
One of the leading AI companies in the world is deeply embedded in many different aspects of US national security across military operations, The standoff between Anthropic and the Pentagon over AI military safeguards is ramping up.
Speaker A
You've had a longstanding anti-war stance starting all the way back to your days at Caltech, and yet you were one of the first AI companies to sign a contract with the Department of Defense to operate on classified networks
Speaker A
that the US uses to fight wars. Explain that. Yeah, so, you know, what I would say is, is look, I mean the world, the world changes.
Speaker A
Like, you know, my my view of this technology, you know, when I see Russia invading Ukraine, when I see the risk of China invading Taiwan, it worries me that we have a kind of resurgent authoritarian block that they're very aggressive
Speaker A
and that we need to defend ourselves. That is something that I, you know, have believed for a while now continue to believe.
Speaker A
And, and that's why across both administrations, you know, you know, across, you know, I may not agree with every policy of I, of either administration, but you know, that's why we've generally been supportive of this.
Speaker A
We don't want a world where China and Russia can build, you know, can analyze all the intelligence with AI, can, you know, can, can use AI for, you know, for attacking Taiwan and Ukraine.
Speaker A
And, and we can't defend them. So that's why we worked with them. We certainly don't do it for the, the money.
Speaker A
It's a huge pain. You know, even, even even putting aside the, the lawfare, it's just a huge pain to get up on government networks for not that much money.
Speaker A
So we, we did it because we cared about it, but similarly because we're doing it because we cared about it, there need to be limitations on the use of the technology and the formulation that I used in Adolescence of Technology.
Speaker A
We should use this technology in every way except the, the ways that undermine our own values, right?
Speaker A
And our red lines of mass surveillance and fully autonomous weapons. Those are things that I believe undermine our values.
Speaker A
It's not worth democracies winning if democracies do those things. And, and so that's the, that's the balance that I, that I see, and that's the stand that we took and it, it explains both why we were the first to work
Speaker A
with Department of War and why there were some things we wouldn't do when, when others were willing to do those things.
Speaker A
I, I think you need to pick a stand and stand your ground. This idea of, you know, companies that seesaw from, we won't do anything with the government to suddenly we're doing absolutely everything with the government.
Speaker A
I don't, I don't get it. You should pick your principles and stick with them.
Speaker A
You've been working with Palantir since 2024. That's right. You know, their technology is used by ICE, police departments, in Gaza.
Speaker A
Is Claude being used for surveillance in other ways? We don't work with ICE either through, either through Palantir or anyone else.
Speaker A
We don't work with CBP, I don't believe we work in Gaza. You know, our, our, we're we're very careful about, you know, scoping our engagements to things that we believe in.
Speaker A
So, you know, you drew your, your red lines, the president banned you from the federal government.
Speaker A
The Pentagon labeled you a supply chain risk. OpenAI jumped in and signed the contract that you wouldn't, what does winning this fight actually look like?
Speaker A
You know, I don't think there's any winning. This fight for a private company like this isn't a fight.
Speaker A
Anthropic is, is trying to win or thinks about winning or losing. This is more a, I won't even call it a fight.
Speaker A
This is more a debate about what the proper use of AI by the government is.
Speaker A
And AI is an emerging new technology. We don't understand the ways in which it's reliable or unreliable.
Speaker A
We don't understand the ways in which it promotes our values or undermines our values.
Speaker A
And so one of the things that I thought was important was to establish a precedent on some of the, some of the use cases we think are good, which frankly is most of them, and some of the use cases
Speaker A
that we're concerned about. And as I've said, we've already seen, you know, you can only do so much with a contract, right?
Speaker A
As we've seen someone else can sign a contract that doesn't respect your, your same red lines.
Speaker A
But what it has done is raised awareness for the issue, and then we have serious bipartisan efforts in Congress attempting to ban some of the things that we're concerned about and attempting to set guardrails.
Speaker A
Again, I don't want to talk about this as a fight, but that's kind of winning the effort to get our country to think more carefully about what is appropriate use of this technology.
Speaker A
That's. Anthropic is run by an ideological lunatic who shouldn't have a, sole, that's, But. That's not my.
Speaker A
Question. My question is AI decision making over what we do, Do you mind being called an ideological lunatic or a bunch of left wing nut jobs?
Speaker A
You know, I've been called worse things than that all the time. You know, people, people can call me or, you know, people can call me or anthropic whatever they want.
Speaker A
The two things that matter are, we're successful as a company and, you know, we stand up for our values.
Speaker A
Like, I actually, in some ways my life is really easy because when those are your, you know, those are the two things you're trying to do.
Speaker A
It's, it's, it's really simple, right? Like it, you know, you just, you always know where you stand.
Speaker A
A US official has said with the help of LLMs, the US military has gone from being able to hit a thousand targets a day to 5,000 targets a day.
Speaker A
That means Claude can help kill more people more quickly. Are you comfortable with that?
Speaker A
I think there's two things here, right? There is, there is the ability of the United States, you know, to be more effective militarily.
Speaker A
I, i I am supportive of that ability. I think having that ability be stronger doesn't cause wars.
Speaker A
It deters wars. Like, you know, I basically, you you're asking like, you know, do you believe in this country, right?
Speaker A
Do you want this country to be a more powerful actor rather than a less powerful actor on the world stage?
Speaker A
I do, I'm a patriot. There's a separate question, which is, you know, are there particular policies that the US government is engaged in that I might support or not support?
Speaker A
Obviously I support some of them and I don't support others of them. It's not up to me. If we provide a technology, you know, the DOW made this point and we actually agree with them.
Speaker A
If we provide a technology, it's not up to us to say, you can do this military operation and you can't do that military operation.
Speaker A
Now, I might privately believe that this military operation makes sense and that military operation is a bad idea, but we're not gonna deny the technology.
Speaker A
You, you have to leave policy in the hands of the military decision makers. What you can do is to assert some high level boundaries that, you know, for, for us, prevent the use cases that seem inconsistent with, with our values,
Speaker A
with our country's values, and promote the use cases that we think, you know, we think, we think encourage our values.
Speaker A
So that's how we think about it. Bloomberg has reported that Claude is being used by the US military in the war in Iran to do AI assisted targeting via platform made by Palantir, Maven Smart System in February.
Speaker A
A US missile reportedly hit a girl school in Iran killing more than 150 people.
Speaker A
Most of them children. Did Claude play a role in that strike? We look, we don't have access to, you know, we, we don't know exactly how, you know, these models were used.
Speaker A
You know, obviously like, you know, these things that, that, you know, mistakes that happen in warfare are really, really terrible.
Speaker A
Like, this is a really terrible thing to happen. If that doesn't make clear why we have to, you know, stand up for use cases that, you know, we don't, we don't support, like, you know, we, we, we were willing
Speaker A
to risk the future of our company to like limit how, you know, these models are used and you know, what, what you're talking about is a use case that doesn't even violate our red lines.
Speaker A
We're worried that there will be a hundred times as much, you know, with, with use cases that, that, that do violate our red lines.
Speaker A
Now I, you know, I I I, you know, again, again, I would say I think overall the use of these, the, the use of these models is appropriate.
Speaker A
I think it's good on net, you know, but military decision makers make terrible mistakes even when, even, even at the best of times.
Speaker A
And I, I don't know if we're in the best of times. Like there's several things we can talk about.
Speaker A
We can talk about making red lines that, you know, prevent uses of the models that are more likely to lead to those problems, right?
Speaker A
If we had allowed, you know, fully at high, if we had just given, in which almost every other company now has to fully autonomous weapons, right?
Speaker A
This is like a human, what what we've seen here is Claude assists, but a human makes the final call.
Speaker A
So a human made that final call, not Claude. Imagine if you had a world in which, not Claude, because we haven't allowed it, but someone else's AI model.
Speaker A
The AI model just makes the decision and the human never sees it. That's what we were standing up for.
Speaker A
That's what we were fighting against. I would, I would also say, you know, there's a separate thing here.
Speaker A
Again, I don't think procurement is the right way to do it, but like, you know, we, we, we need to make sure that, you know, it's, it's a matter of interest to the American people, not to me as a supplier
Speaker A
of the technology, but to the American people that are military decision makers don't make these mistakes that they operate reliably, that, you know, they choose wisely what to do.
Speaker A
Again. You know, that's, that's of concern to me as a, as a, you know, as a citizen, as a supplier of the technology.
Speaker A
Like, you know, the government uses Microsoft Excel a lot. You know, if I said micro, you can use Excel for, you know, this military operation, but not the, you can't, you can't realistically do that.
Speaker A
But hopefully that gives you a sense of how we think about it. This school had a website.
Speaker A
You could have found it in a Google search. Like shouldn't Claude have spotted that, shouldn't AI or whatever technology they used have spotted that.
Speaker A
And does it speak to a scarier issue about using technology as a shortcut in war.
Speaker A
Look, I look what I'm, what I'm, you know what I'm gonna say is, you know, and, and I, you know, I don't, I don't know, this relies on, you know, maybe classified knowledge that I don't have.
Speaker A
But you know, the principle that, that we have established, and I think the principle that was obeyed here is a human makes the human makes the final decision.
Speaker A
I don't know what role Claude or any other AI had, but like, if, if this isn't an illustration why that principle is so important, I don't know what's, is Is AI warfare more likely to stop World War III, a war between the US and China?
Speaker A
Or is it more likely to make it happen? I would say on balance it is more likely to stop it.
Speaker A
But if we have no limits on how it's used, then I think, you know, it could be more likely to cause it.
Speaker A
You know, you've seen Doctor Strangelove, right? The premise of it was like you have a doomsday device that automatically fires nuclear weapons when it thinks nuclear weapons are being fired at it.
Speaker A
What could go wrong? Right. Again, i I I get to this lethal, you know, fully autonomous weapons thing.
Speaker A
I think the way conflicts happen is that, you know, the, the two sides jump at each other.
Speaker A
They misunderstand each other. And when we don't have proper oversight of this technology, I think those kinds of accidents are more likely to happen.
Speaker A
Now, I think if AI is used in an appropriate way, in, in, in, not even warfare, but think of just, just intelligence collection, you know, let, let's say we're able to, you know, predict an invasion of Taiwan or a new movement in Ukraine.
Speaker A
Like, you know, our adversaries will think twice about, you know, about conducting some kind of invasion or military operation if we know everything that they're doing.
Speaker A
And so I think superior intelligence really can de deter conflict here. Superior ability to respond can deter conflict.
Speaker A
I continue to be a believer in these things. Anthropic's making headlines almost on a weekly basis.
Speaker A
Yes. Most notably now around mythos. Of course, This is the latest and greatest Anthropic model, and it is capable of going through all the links of the cyber kill chain and doing so autonomously.
Speaker A
You said mythos was too powerful to release to the public. What surprised you most about it?
Speaker A
I think the thing that surprised me most about it was the models had been climbing in their ability to find vulnerabilities and importantly turn those vulnerabilities into exploits, which people only talk about the vulnerabilities.
Speaker A
They don't often talk about turning the vulnerabilities into exploits, which it, it was quite good at.
Speaker A
So the things that surprised me are we saw this huge jump. It was a particularly large jump and without us really prompting them at all, some of the early companies that we gave this to said things like, this is a super weapon.
Speaker A
You should have to own a gun license to use it. Please don't release this.
Speaker A
Like, the, the demand to do this was coming from the companies we gave it to who were finding so many critical vulnerabilities and exploitability around these critical vulnerabilities that, you know, they, they were basically asking us not to,
Speaker A
not to not to release it. Now to be clear, 'cause things always get distorted in the world of social media, the goal isn't to keep this locked up forever.
Speaker A
We're kind of gradually trying to open this up to a wider and wider set of people and eventually we believe that we should release mythos to to, you know, to a general audience, but with kind of strong cyber safeguards.
Speaker A
Now, a concern is today's cyber safeguards, which we did release on Opus 4.7, which is a good cyber model, but a substantially weaker one.
Speaker A
These can be jailbroken and we're a little concerned about some of the other companies who think this is a sufficient defense because yeah, it works sometimes, but you know, we all know that these classifiers can be jailbroken or gone around
Speaker A
and our own testing as well as frankly our assessment of the models that other, the defenses that other companies have put in place suggests that these defenses are not strong yet.
Speaker A
And, and that's what we're waiting for, getting the defenses to the point where we really have confidence in them.
Speaker A
There was a lot of pushback on it. You know, you have researchers saying they were able to replicate it using, you know, cheaper open source models.
Speaker A
Some folks say OpenAI, you know, has these capabilities already, you know, what do you say to folks who say this is a grand PR play? Yeah, so the, the claim that could be replicated with open source models, that's just incredibly false.
Speaker A
So the idea is mythos looks across the whole cold base and finds something. Some guy went on Twitter and said, well, if you point an open source model at exactly the line of code that mythos finds, then it finds the same issue.
Speaker A
That, that, that isn't the, that isn't the prompt, that isn't the question, right? Like, that is not, that is not the same thing.
Speaker A
The ultimate test of this is like, we go to companies, we go to open source repos.
Speaker A
We found 271 new vulnerabilities in Firefox. We found many thousands within the private, you know, companies who haven't fixed them yet or can't disclose them yet.
Speaker A
Like no one found those 271 vulnerabilities with the previous model. So like the actual workflow of what actually works in practice as opposed to, you know, okay, I find the exact line that mythos found, you know, I found the needle in the haystack.
Speaker A
Something else can now pick up the needle. But what about the folks who say, this was just good marketing.
Speaker A
You know, we have suffered enormously commercially from not releasing this model. This model has incredibly accelerated research within Anthropic and production and next models.
Speaker A
It would do the same in the outside world if we were to release it.
Speaker A
This has hurt us enormously commercially. If this helps defenders, it also helps attackers. Can we defend anything anymore?
Speaker A
What I would say is that the reason that we're giving Mythos to defenders before we give it to attackers is to patch all the bugs.
Speaker A
I don't know, as the models get better, there may be more and more bugs to be found, but there's only so many, they're finite, right?
Speaker A
It's like you have this surface and there's only so many holes in it. You, you patch all the holes and the surface becomes very hard to attack, as well as the code itself is written with the powerful models.
Speaker A
So it's, it's then becomes very hard to find flaws in or break into. So I think on the other side of this, hopefully six months or a year from now, we have a much more secure internet ecosystem than we had in the past.
Speaker A
We're trying to get to that world and we're doing the best we can to open up mythos to new cyber defenders.
Speaker A
We've been talking to the government, we're very respectful of their recommendations. They're slowing the pace at which we open it up because they're worried about counterintelligence risk.
Speaker A
I think that's sensible. I think all serious people here understand that there's real trade offs here.
Speaker A
We see a lot of sniping from people on Twitter and from, you know, from other AI companies.
Speaker A
You look at what they're saying and the inconsistency with what they're doing. It's not, they're not serious people.
Speaker A
They're not seriously engaging with the, the serious trade-offs that, that, that, that we have here.
Speaker A
Look, I have customers calling me up every day saying, I want access to Mythos.
Speaker A
I have countries call calling me up saying, I want access to Mythos and I have the US government and my security team saying, no, wait a minute.
Speaker A
There's risk to it. What? You know, I'm not saying one side or the other is right?
Speaker A
I think it's somewhere in between. Both sides have valid points, but there's a real challenge here and we need to face it together as a society, not accuse things of being cheap marketing, not use cheap marketing to try
Speaker A
and counter position, which some of the other companies are doing it. It just, it just all shows an an incredible lack of gravitas and maturity.
Speaker A
We need to all face these mo this moment together. Have you had to make trade-offs already that you're not entirely comfortable with?
Speaker A
Throughout the entire history of Anthropic has been trade-offs, right? The entire history of Anthropic, right?
Speaker A
Where, you know, in, in, in, in, in, in, in, in some ideal world, you would, you know, you would, you would prefer to, before you release the first chatbot, you know, you could, you could spend years studying, you know, every possible thing that could go wrong with it.
Speaker A
Now, we did delay, we did delay the initial release of Claude, but you know, we did it for a few months.
Speaker A
So what I'm saying is everything is a trade off. You know, the, the extreme ends of the spectrum are, are, are completely insane.
Speaker A
And so everything is a trade off. What I would say is that now that we're in, you know, what I would describe as a commercially leading position, I, I am actually, and Daniela are actually doing all we can
Speaker A
to move the dial even further towards, towards being careful. That's what the Mythos release was, was about, right?
Speaker A
It's very hard to do something like that if you're not the leading player. And so I think you're gonna see more things, more things like that.
Speaker A
You know, there's this argument, why wouldn't the government take you over? Why would they let a private company control technology that's so powerful?
Speaker A
So I actually think that's a very, that's a very serious question and I share those concerns.
Speaker A
I don't think the government should outright take us over, but I would put it this way.
Speaker A
I would say, just to back up and describe the situation, every previous powerful technology we've seen in history was either built by the government or originated with the government.
Speaker A
So nuclear weapons, obviously, you know, initially built by the government and pretty much built by the government after that.
Speaker A
But even like the internet, GPS, cell phones, all the R&D was, you know, was done in the labs and the federal labs and the universities.
Speaker A
AI is the first technology that's been built in the private sector and where government has not really had a serious role and is coming in late to the game.
Speaker A
I think that's actually a dangerous and unstable situation. It is not the situation I would've chosen.
Speaker A
There's not really an alternative, like, you know, this technology is possible to build, our adversaries are building, it has economic value, like it's, it's going to get built.
Speaker A
The, the issue is the government not doing it, not the private sector doing it.
Speaker A
I think we need to think about checks and balances on power. So I think there need to be checks and balances on the power of the AI companies, right?
Speaker A
We have this thing, the long term benefit trust. What that is, is it's a set of, basically it's a body that can appoint the majority of the board members and remove the majority of the board members.
Speaker A
So it basic, it essentially, if you thread it through, has the power to fire me.
Speaker A
And what we're looking at is we're introducing some elements, you know, nowhere near all the elements, but we're introducing a a little bit of the elements of like public governance, right?
Speaker A
Where, where it's like, you know, you're accountable to someone who just doesn't, he doesn't just have stock stock in the company.
Speaker A
So that's, that's very important and that structure is gonna continue no matter what happens to the company. That's on the AI...
Speaker A
And we encourage other companies to have similar structures on the government side, I think we need checks and balances.
Speaker A
You know, there are, there are efforts in Congress that have been announced to enact those red lines, right?
Speaker A
So I really think the, you know, the legislative branch and the judicial branch need to exert themselves because this technology, I'm scared of companies having it, but I'm also scared of government having it.
Speaker A
And then the companies need to provide checks on government, and the government needs to provide checks on companies.
Speaker A
You know, we need basic regulation of the technology. You know, I think we need to start doing pre-release testing, required pre-release testing, testing and auditing of the models.
Speaker A
You know, it, it's very funny to me how there's a particular group of people in the tech world in Silicon Valley who started, you know, they, they, they started with a position of like even having transparency around this technology,
Speaker A
even export control, you know, this is all, you know, just totally, it'll apocalyptically destroy our potential to create the technology.
Speaker A
It'll kill innovation. And then as soon as they see the first real danger, which I've been expecting all along, there's all this talk of like nationalization and the government should just seize it.
Speaker A
Come on folks here, you're, yo you're yo-yoing from like the most extreme anti-regulatory, you know, you know, if you, if you look at us the wrong way, you're destroying the industry to, you know, this completely communist, the government should grab it all.
Speaker A
We, we need a more, we need a more sensible, moderate approach. That's the one we've been favoring all along because we've, we've understood the power of this technology.
Speaker A
We're not panicking, we're not denying it. We see the smooth exponential and we're responding to it appropriately.
Speaker A
So how was your visit back to the White House? You know, we always try to work together with whoever we can in government.
Speaker A
You know, I, I I said we have this simple approach, like we have a set of principles, we like follow those principles and we hope that folks on the other side are reasonable.
Speaker A
And you know, honestly, the government has taken Mythos very seriously. Like we've had good conversations with Treasury Secretary Bessent with Chief of Staff, Susie Wiles.
Speaker A
I think they really understand, you know, the nature of the risks here. Mythos has, I think, you know, helped them to feel much more concretely where these risks are.
Speaker A
So, you know, again, as with any administration, there are parts who we get along with very well and who understand it.
Speaker A
And you know, there are other parts that are harder to get along with. I think that's normal.
Speaker A
That would be the case in any, in any administration, and we just try to navigate it as best we can.
Speaker A
You worked at Baidu earlier in your career, big Chinese tech company. You worked at the Silicon Valley outpost of it, and you've been clear on your views on China.
Speaker A
Strong open source models are coming out of China and you have US companies building on them for free. Is that a threat?
Speaker A
So, you know, one of the things we've seen with this technology is that there's really a premium to how intelligent the models are.
Speaker A
We very, very rarely see that people would prefer to use models with lower intelligence.
Speaker A
Now to be clear, there's a thriving ecosystem. There are lots of challenges and problems that are much easier than, you know, the ones we need frontier models for.
Speaker A
But again, it's an exponential, right? Like it's possible that like these far from frontier models have economic value comparable to what we saw in 2023 and 2024.
Speaker A
But again, we have this 10X a year growth and, and, and so what, what we find is that what's on the frontier is always much, much larger than what is, what is away from the frontier.
Speaker A
I think this is something that people who are used to building products in the previous era don't quite understand, right?
Speaker A
As someone who's come in who, you know, hadn't run a company before, who's like, you know, has never thought about the previous product era particularly to test the social media era, I feel like an outsider to that world.
Speaker A
And I, I, I feel that people's instincts are wrong. They have all these kind of product heuristics and I think the 10x per year model exponential really breaks that like intelligence is just such a huge factor that it outweighs everything else.
Speaker A
And so we're just seeing over and over again that, you know, the value is found on the frontier.
Speaker A
Now what I do worry about with some of these laggard models is the risks of them where we have Mythos class cyber capabilities.
Speaker A
12 months from now, we'll have much better cyber capabilities, but the Mythos class cyber capabilities may just be available for, for anyone to, to download. Now, hopefully we'll have patched everything before then.
Speaker A
I don't think there's anything we can do to stop it, but I, I think it's a serious concern.
Speaker A
Did what you saw by do shape your views on China? Not really, no. I worked there for, I worked there for a year.
Speaker A
You know, I think I probably learned more about like speech, speech recognition and, and you know, all, all, all all of that.
Speaker A
Maybe the only thing that concerned me was, you know, part of how we got all the speech recognition data was, you know, they're like, they said ominously, oh, we don't care about privacy in China.
Speaker A
So we have all this, this speech recognition data. But I think, I think aside from my worries, here are geopolitical, you know, I think the things that most worried me about what happened in China are, you know, what we, what we saw happen to the Uyghurs,
Speaker A
what we saw with suppression of criticism even in the US, with what happened with Hong Kong, right?
Speaker A
The fact that the CCP could reach into the US business network and, you know, and suppress criticism, that's an authoritarian state and, and a high tech authoritarian state.
Speaker A
And when I see how that combines with AI you, you really get a, a dystopia here like 1984 or worse.
Speaker A
And, and my focus is on trying to prevent that. And I think we have an opportunity to prevent that.
Speaker A
I think we have an opportunity for AI to be a pro-democracy technology, you know, that kind of makes people freer that delivers on the promise of equal justice for all.
Speaker A
Or it could go the other way. And, and which way it goes depends on the actions of the AI companies.
Speaker A
It depends on the actions of the government, depends on the actions of all of us.
Speaker A
And so I see us as having a responsibility here. There's a moment that people in your field talk about where AI gets good enough to improve itself and then the improved version improves itself and so on.
Speaker A
Some of your researchers think that that moment is close. How far away is it?
Speaker A
I don't think it's a moment in time. I think it's a continuous process. We're already seeing it in some ways where the AI is able to suggest architectures for the next AI.
Speaker A
You know, I would say a year ago, we were seeing 10 to 15% kind of increase in total factor productivity due to AI.
Speaker A
Like that's probably up to 20 or 30% now might, you know, might, it might be doubling like as with all things we're on the exponential, there's no moment where AI improves itself or runs out of control or becomes unsafe.
Speaker A
What we have is an accelerating exponential and at each point on the exponential we have to assess is, is this a time to slow down?
Speaker A
Is this a time to, you know, put more controls on on this technology? I think more and more of that is gonna be required.
Speaker A
But I, you know, I think the Rosetta Stone to all of this is the smooth exponential.
Speaker A
Again, I think there's an object lesson in the people who were against all AI regulation and then they saw one thing and they wanted to nationalize.
Speaker A
I think there's an object lesson in the people who dismissed the power of AI and then said, oh my God, it's improving itself.
Speaker A
It's running outta control. We have to shut it all down. Yo-yoing between those extreme reactions is incredibly unhelpful as a response to this technology.
Speaker A
The right response, the wise response is to say, we're not gonna panic. Our countermeasures will smoothly ratchet up with the power of the technology.
Speaker A
If you see someone having this kind of crazy yo-yo reaction, that's a sign that they were caught by surprise and that they're not serious.
Speaker A
I understand one of your favorite books is The Making of the Atomic Bomb. That is correct. Do you see parallels between yourself and Oppenheimer?
Speaker A
You know, the figure I most identified with was Leo Szilard, who, you know, the one who first basically had the idea that there could be a kind of chain reaction.
Speaker A
Look, my view is we're we're not gonna get through this with like larger than life personalities or like figures who try and be at the center of everything, right?
Speaker A
There needs to be a balance of power here, right? There's a lot of powerful actors who have interests here, and the only way it's gonna end well for everyone is if there is some, there's basically checks and balances everywhere.
Speaker A
So in some ways I actually see Oppenheimer as a failure case as what should not happen.
Speaker A
You've said there's roughly a 10 to 25% chance of civilizational collapse. That is not insignificant.
Speaker A
Is there a scenario where it's something that Anthropic built, that caused that? I mean, I certainly hope not.
Speaker A
My view is that, you know, the, the actions that we have taken lower that probability rather than increasing it, right?
Speaker A
That probability comes from the, the, you know, the very straightforward recipe of the technology, the existence of many countries in the world, the existence of many companies within an economy and new ones created.
Speaker A
If the void is isn't filled, like that's a dilemma that we're in. We are trying to act to lower that probability.
Speaker A
I think we lower it a lot more than we raise it. But, you know, the inherent property of this technology is that it's unpredictable.
Speaker A
And so, you know, we try to build something and test it a lot before it's released.
Speaker A
And then the models that are released today are not dangerous, or at least not, you know, really I think dangerous outside of cyber.
Speaker A
And then we try and iterate and learn from that. So there's like a zillion defense mechanisms. You know, half of what we do within the company is try and, you know, reduce the risk as much as we can,
Speaker A
but, you know, it's, it's never gonna be zero. I guess what I would say is, you know, suppose there are a bunch of like, you know, airline companies out there and you're like, well, I'm gonna make an airline company that's safer.
Speaker A
It can both be the case that, you know, your airline company is 10 times safer than all the other airline companies.
Speaker A
But you know, if, if someone comes and asks you, like, can you guarantee that your airplane will never crash?
Speaker A
I mean, how could you, how could you possibly, But if there was a 25% chance of an airplane crashing, you wouldn't get on that plane.
Speaker A
That's right. 25% is too high. We're trying to make that probability much, much lower. That is the goal.
Speaker A
You are building something incredibly powerful and stand to gain enormously from it. Why should we trust you?
Speaker A
My view of this is actually when any company starts out and, and particularly, you know, what we've seen with the behavior of, of just Silicon Valley as an entity.
Speaker A
It's, it's thinking over the last couple years. I think starting from a position of distrust, you know, if you don't know anything about me, if you know anything about Anthropic is pretty rational.
Speaker A
I think Silicon Valley has lost a lot of the world's trust and kind of has to re-earn it.
Speaker A
And the message, you know, we're trying to send is or actually different and, and that has to be earned in things that we actually do.
Speaker A
You can agree or disagree, but we stood up for our values. The thing with, you know, Mythos, like it's, it's really hampered us commercially not to put this very powerful model out.
Speaker A
And there were a bunch of smaller things before it, you know, we, we, we put our money where our mouth is on, you know, China, we cut off access to, to models.
Speaker A
We didn't have to do that. No one told us to do that. You know, that cost us several hundred million dollars back when several hundred million dollars was a big, was a significant fraction of our revenue.
Speaker A
You know, the, the delay of Claude two, like we have a long history of it.
Speaker A
We aren't perfect, we make mistakes. But you know, what I would ask is for people to look at the overall history and say, if you add up that overall history, what is the hypothesis about us that is most consistent with that overall history?
Speaker A
People have to decide for themselves, but I think the hypothesis that's consistent is we are genuinely trying to do the right thing.
Speaker A
We're imperfect organizations are, you know, always dysfunctional. We're always trying to, you know, fix them and make them work better.
Speaker A
Many foot faults, many things that go wrong, but at basis we, we have a honest and earnest picture of how to do the right thing and we're trying to execute on that picture.
Speaker A
We will see you on the other side of the exponential then Hopefully. You always wanted to be a Hollywood star, right?
Speaker A
I, I, that's one surprising thing that I didn't understand about the CEO job is how often you have to wear makeup.
Speaker A
That was not on my bingo card, Just a little powder.
Topics:Dario AmodeiAnthropicAI safetyAI developmentOpenAISilicon ValleyAI ethicsClaude CodeAI enterpriseAI national security
