Speaker A
Why are we working on AI in the first place? I'm just going to arbitrarily pick Jared.
Anthropic co-founders discuss their AI journey, safety focus, and innovations like constitutional AI in building safer, scalable language models.
Key Takeaways
- Long-term collaboration and shared vision among founders fueled Anthropic’s creation.
- Scaling language models and safety research are deeply interconnected.
- Concrete, practical framing of AI safety helped gain broader acceptance in the field.
- Constitutional AI represents a novel, promising method to align AI behavior with human values.
- Simplicity and principled approaches remain central to effective AI safety solutions.
Summary
- The co-founders share their personal motivations for working on AI, including transitioning from other fields and early collaborations.
- They recount their long-standing professional relationships formed at Google Brain and OpenAI, spanning over a decade.
- The discussion highlights the importance of scaling laws and language models like GPT-2 and GPT-3 in advancing AI capabilities.
- Safety is emphasized as a core motivation, particularly ensuring AI systems understand human values and can communicate effectively.
- The concept of Reinforcement Learning from Human Feedback (RLHF) is explained as a key technique intertwined with model scaling.
- They describe the 'Concrete Problems in AI Safety' paper as a foundational effort to ground AI safety research in practical machine learning.
- The paper also served as a consensus-building political project to legitimize AI safety concerns within the research community.
- The founders reflect on the early skepticism and eventual acceptance of safety-focused approaches in AI development.
- Constitutional AI is introduced as an innovative approach where AI behavior is guided by a written constitution, leveraging AI’s ability to follow multiple-choice style prompts.
- They emphasize the power of simple, principled methods in AI and the ongoing commitment to safety and scalability at Anthropic.
Full Transcript — Download SRT & Markdown
Speaker A
Why are you doing AI at all?
Speaker B
I mean, I was working on physics for a long time and I got bored.
Speaker B
And I wanted to hang out with more of my friends, so.
Speaker C
I thought Dario pitched you on it.
Speaker D
I don't think I, I don't think I explicitly pitched you at any point. I just kind of like showed you results of like AI models and then and and was trying to make the point that like they're very general and like they don't only apply to one thing and then like just at some point after I showed you enough of them, you were like, oh yeah, that seems like it's right.
Speaker C
How long had you been a professor before like when you started?
Speaker B
I think like six years or so.
Speaker B
I think I helped recruit Sam.
Speaker E
I talked to you and you were like, I think I've created a good bubble here and like my goal is to get Tom to come back.
Speaker A
And did you meet everyone through Google when you were doing the interpretability stuff, Chris?
Speaker F
No, so I guess I actually met a bunch of you when I was 19 and I was visiting the Bay Area for the first time.
Speaker F
So I guess I met uh uh Dario and Jared then and and I guess they were postdocs, which I thought was very cool at the time.
Speaker F
And then I was working at Google Brain and Dario joined and we sat side by side actually for a while.
Speaker F
We had desks beside each other.
Speaker F
And I worked with Tom there as well.
Speaker F
And then of course, I got to work with all of you at Open AI when I when I went there.
Speaker F
So I guess I've known a lot of you for like more than a decade, which is kind of wild.
Speaker A
If I remember correctly, I I I met Dario in 2015 when I went to a conference you were at and I tried to interview you and Google PR said I would have read all of your research papers.
Speaker D
Uh yeah, I think I was writing concrete problems in AI safety when I was at Google.
Speaker D
I think you wrote a story about that paper.
Speaker A
I did.
Speaker E
I remember right right before I started working working with you, like we had a I think you invited me to the office and I had to like come chat and just like tell me everything about AI and you like explained.
Speaker E
I remember afterwards being like, oh, I guess this is I guess this stuff is much more serious than I realized.
Speaker E
And you were like probably explaining the big blob of compute and like parameter counting and how many neurons are in the brain and everything.
Speaker F
I feel like Dario often has sort of effect on people, this is much more serious than I realized.
Speaker D
I'm the bringer of happy tidings.
Speaker A
But but I remember when we were at when we were at Open AI, there was for for scaling law stuff and just making things bigger and it started to feel like it was working.
Speaker A
And then it kind of kept on eerily working on a bunch of different projects, which I which I think is how we all ended up working closely together because it was first GPT2 and then scaling laws and GPT3 and we ended up.
Speaker D
We're at the blob of people that were making things work.
Speaker B
Yeah. I think we were also excited about safety because at that era there was sort of this idea that AI would become very powerful but like potentially not understand human values or not even be able to communicate with us.
Speaker B
And so I think we were all like pretty excited about language models as a way to kind of guarantee that AI systems would have to understand kind of implicit knowledge that.
Speaker D
And and RL for from human feedback on top of language models, uh which was the whole the whole reason for scaling these models up was that, you know, we couldn't do we couldn't the models weren't smart enough to do RLHF on top of.
Speaker D
So that's the kind of intertwinement of safety and scaling of the models that we, you know, still believe in today.
Speaker F
Yeah. I think there was also an element of like the scaling work was was done as part of the safety team that, you know, Dario started at Open AI because we thought that forecasting AI trends was was important to be able to have have us take them seriously and take safety seriously as a problem.
Speaker A
Correct.
Speaker A
Yeah, I mean we took I remember being in some airports in England sampling from GPT2 and using it to write fake news articles and and slacking Dario and being like, oh, this this stuff actually works and might have like huge policy applications.
Speaker A
I think Dario said something like, yes.
Speaker G
Yes.
Speaker A
In his typical way.
Speaker A
But then we worked on that a bunch as well as for release stuff, which was kind of wild.
Speaker H
Yeah, I remember the release stuff.
Speaker H
I think that was when we first started working together.
Speaker A
Yeah.
Speaker H
That was a fun time.
Speaker A
The GPT2 launch.
Speaker A
Yeah. But I think it was good for us because we did we did a kind of slightly strange safety-oriented thing all together and then we ended up doing Anthropic, which is a much larger, slightly strange safety-oriented thing all together.
Speaker F
Well, so I guess just like going back to the concrete problems because I I remember so I joined Open AI like 2016, like one of the first 20 employees or whatever with with you, Dario.
Speaker F
And I remember at that time the like concrete problems in AI safety seemed like it was like the first like mainstream like AI safety.
Speaker D
Yes.
Speaker F
Paper. I guess I I don't really know if I ever asked you what the story was for like how that came about.
Speaker D
Um Chris knows the story because he was he was involved in it. I think, you know, we were both at Google.
Speaker D
I forget what other project I was working on, but like with many things, it was my attempt to kind of procrastinate from whatever other project I was working on that I now completely completely forgotten what it was, but uh uh I think I think it was like Chris and I decided to write down what are some open problems in terms of AI safety and also AI safety are usually talked about in this very kind of abstruse abstract way.
Speaker D
Can we kind of ground it in the ML that was going on at the time?
Speaker D
I mean, now there's been like, you know, six, seven years of work in that vein, but it was almost a strange idea at the time.
Speaker F
Yeah, I think I think it was a way in which it was almost a like a kind of political project where at the time, um, a lot of people didn't take safety seriously.
Speaker F
So I think that there was sort of this goal to collate a list of problems that sort of people agreed were reasonable, already often already existed in literature and then get a bunch of of people across different institutions who are credible to be authors.
Speaker F
And like I remember I had this like whole long period where I just talked to like 20 different researchers at Brain to build support for publishing the paper.
Speaker F
Like in some ways if you look at it in terms of the problems and a lot of the things that emphasized, I think it, you know, hasn't held up that well in that it's, you know, I think it's not really the the right problems.
Speaker F
But I think if you sort of see it instead as a consensus building exercise that there's something here that is real and that is worth taking seriously, then it was a pretty important moment.
Speaker A
I mean, you you end up in this really weird sci-fi world where I remember at the start of Anthropic, we were talking about constitutional AI. And I think Jared said, oh, we're just going to write like a constitution for a language model and that'll change all of its behavior.
Speaker A
And I I remember that sounded like incredibly crazy at the time.
Speaker A
But why did you guys think that was going to work? Because I remember that was one of the first early like big research ideas we had at the company.
Speaker B
Yeah, I mean, I think Dario and I had talked about it for a while.
Speaker B
I guess, uh, I think simple things just work really, really well in AI. And so like, I think the first versions of that were like quite complicated, but then we kind of like whittled away until like just use the fact that AI systems are good at solving multiple choice exams and like give them a prompt that tells them like what they're looking for and and that was kind of kind of a lot of what we needed.
Speaker B
Um, and then we were able to just write down these principles.
Speaker D
I mean, it goes back to like the big blob of compute or the bitter lesson or the scaling hypothesis.
Speaker D
If you can identify, you know, something that you can give the AI data for and that's kind of a clear target, you'll get it to do it.
Speaker D
So like, here's this, here's this set of instructions, here's this set of principles.
Speaker D
AI language models can like read that set of principles and they can like compare it to the behavior that they themselves are engaging in. And so like, you've got your training target there.
Speaker D
So once you know that, I think my view and Jared's view is, there's a way to get it to work.
Speaker D
You just, you just have to fiddle with enough of the details.
Speaker B
Yeah. I think it was always weird for me, especially in these early eras because like I was in physics and then coming from physics.
Speaker B
And I think now we we forget about this because everyone's excited about AI, but like I remember talking to Dario about concrete problems and other things and I just got the sense that AI researchers were very, very kind of psychologically damaged by the AI winter where they were they just kind of felt like having like really ambitious ideas or ambitious visions was like very disallowed.
Speaker B
And that's kind of how I imagine it was in terms of talking about safety.
Speaker B
In order to care about safety, you have to believe that AI systems could actually be really powerful and really useful.
Speaker B
And I think that like there was kind of a prohibition against being ambitious. And I think one of the benefits is that physicists are very arrogant and so they're constantly doing really ambitious things and talking about things in terms of grand schemes and so, uh, yeah.
Speaker D
I mean, I think that's I I I think that's definitely true.
Speaker D
Like I remember in 2014, it was like, there were just like, I don't know, there were just like some things you couldn't say, right?
Speaker D
But but I actually think it is kind of an extension of problems that exist across academia, other other than maybe theoretical physics, where they've kind of evolved into very risk-averse institutions for a number of reasons.
Speaker D
And even the industrial parts of AI had kind of transplanted or forklifted that mentality. And it took a long time.
Speaker D
I think it took until like 2022 to get out of that mentality.
Speaker F
There's there's a weird thing about like what does it mean to be conservative and respectful, where you might think, um, like one one version you could have is that what it means to be conservative is to take the risks or the potential harms of what you're doing really seriously and worry about that.
Speaker F
But another another kind of conservatism or caution and I I think we were sort of in a regime that was very controlled by that one.
Speaker F
I mean, you you see it historically, right? Like if you look at the like early discussions in 1939 between, you know, people involved in nuclear physics about what the nuclear bombs were sort of a serious concern.
Speaker F
You see exactly the same thing with like Fermi resisting, um, these ideas because it just seemed kind of like a crazy thing.
Speaker F
And and other people like Zeller taking, um, taking the ideas seriously because they were worried about the risks.
Speaker D
Perhaps the deepest lesson that that I've learned in the last 10 years and probably all of you have learned some form of it as well, is there can be this kind of seeming consensus, these things that kind of everyone knows that that, I don't know, seem sort of wise, seem like they're common sense, but really they're just they're just kind of hurting behavior masquerading as maturity and sophistication.
Speaker D
And when when you've seen the consensus can change overnight. And when you've seen it happen a number of times, you you you suspected, but you didn't really bet on it.
Speaker D
And you're like, oh man, I kind of thought this, but what do I know?
Speaker D
How how, you know, how can I be right and all these people are wrong?
Speaker D
You see that a few times, then you just start saying, nope, this is the bet we're going to make.
Speaker D
I don't know for sure if we're right, but like just just ignore all this other stuff, see it happen and I don't know, even if you're right 50% of the time, being right 50% of the time contributes so much, right?
Speaker D
You're you're you're adding so much that is not being added by anyone else.
Speaker A
Yeah. It feels like that's where we are today with some safety stuff. There's like a consensus view that a lot of this safety stuff is unusual or doesn't naturally fall out of the technology.
Speaker A
And then at Anthropic, we do all of this research where weird safety misalignment problems fall out as a natural dividend of the tech we're building.
Speaker A
So it feels like we're in that counter-consensus view right now.
Speaker H
But I feel like that has been shifting over the past even just like 18.
Speaker A
We've been helping to shift that.
Speaker H
We've definitely been helping.
Speaker A
I know.
Speaker A
I mean.
Speaker A
We've been doing research.
Speaker H
Constant force.
Speaker H
Yeah. But I also think just like world sentiment around AI has shifted really dramatically. And, you know, it's more common in the user research that we do to hear just customers, regular people say, I'm really worried about what the impact of AI on the world more broadly is going to be.
Speaker H
And sometimes that means, you know, jobs or bias or toxicity, but it also sometimes means like, is this just going to mess up the world, right?
Speaker H
Is how is this like going to contribute to fundamentally changing how humans work together, operate, which is I wouldn't have predicted that actually, you know?
Speaker E
Yeah. I mean, I guess two things that at least connect to what you were saying earlier. I mean, one is, um, I feel like people frequently join Anthropic because they're sort of scientifically really curious about AI.
Speaker E
And then kind of get convinced by AI progress to sort of share the vision of the need not just to advance the technology, but to understand it more deeply and to make sure that it's it's safe.
Speaker E
And I feel like it's actually just sort of exciting to have people that you're working with like kind of more and more united in their vision for both what AI development looks like and the sort of sense of responsibility associated with it.
Speaker E
And I feel like that's been happening a lot due to a lot of advances that have happened in the last year, like like what Tom talked about.
Speaker E
Another is that, I mean, going back really to concrete problems, I feel like we've done a lot of work on AI safety up until this point.
Speaker E
A lot of it's really important.
Speaker E
But I think we're now with with some recent developments, really getting a glimmer of what kinds of risks might literally come about from systems that are very, very advanced, so that we can investigate and study them directly with interpretability, with other kinds of safety mechanisms and really understand like what the risks from very advanced AI, uh, might look like.
Speaker E
Um, and I think that that's something that's really going to allow us to sort of further the mission, um, in a in a like really deeply scientific, empirical way.
Speaker E
And so I'm excited about sort of the next six months of how we use our understanding of what can go wrong with advanced systems to characterize that and figure out how to avoid those pitfalls.
Speaker H
Perfect.
Speaker A
Finn.
Speaker H
Good timing.
Speaker A
Good timing.
Speaker H
We got to do this.
Speaker A
This is the only time we ever get.
Speaker H
We got to do this.
Speaker A
I do want to have.
Topics:AnthropicAI safetylanguage modelsGPT-2GPT-3scaling lawsreinforcement learning from human feedbackconstitutional AImachine learningAI research
