Speaker A
Yuval Noah Harari, welcome to The Insider.
Yuval Noah Harari discusses AI's future impact on society, finance, and trust, highlighting human choices in an AI-driven world.
Key Takeaways
- AI will likely surpass human intelligence and control critical systems soon, but this is not predetermined.
- Human trust is central to financial and social systems, and AI will transform this trust dynamic.
- Loss of human understanding in AI-driven systems poses risks of uncontrollable crises.
- Governments and societies must act now to regulate AI and prevent harmful outcomes.
- The future with AI requires balancing innovation with responsibility and ethical considerations.
What the video covers
- Yuval Noah Harari explores the future of AI, agreeing it may surpass human intelligence and control key systems within 10 years.
- He emphasizes that AI dominance is not inevitable and humans still have agency to shape outcomes.
- Harari discusses AI's control over the financial system, comparing humans' current understanding to cows unaware of finance.
- Money is fundamentally trust, and AI will create financial systems beyond human comprehension.
- Trust may shift from humans to AI, as seen in the rise of cryptocurrencies.
- The complexity of AI-driven finance could lead to loss of human control and understanding, raising risks of systemic crises.
- Harari critiques contradictory views on AI and societal issues, such as Elon Musk's concerns about immigration alongside AI takeover predictions.
- He stresses the importance of urgent political decisions, such as banning AI personhood and regulating AI applications.
- The interview highlights the psychological and social experiment AI represents for humanity.
- Harari calls for global awareness and proactive governance to ensure AI benefits society rather than undermines it.
Chapters
- 00:00Introduction and AI Future Predictions
- 03:08Discussion on AI Surpassing Human Intelligence
- 07:06AI Control Over Financial Systems
- 14:20Money as Trust and Financial Complexity
- 21:27Loss of Human Understanding and Trust Migration
- 29:03Contradictions in AI and Societal Concerns
- 46:37Urgent Political Decisions and AI Governance
- 56:42Conclusion and Reflections on AI's Social Experiment
Full Transcript — Download SRT & Markdown
Speaker A
Thank you. It's good to be here.
Speaker A
I've been so looking forward to this conversation.
Speaker A
Not only are you one of the world's most prominent public intellectuals, your books have sold millions of copies, and they've done so, I think, because you have a really remarkable ability to synthesize the broad sweep of history.
Speaker A
But also you are willing to make bold predictions about the future, which not every historian is willing to do.
Speaker A
That's where I wanted to focus today: the future, particularly the future in an AI world.
Speaker A
It's something you've been writing about a lot.
Speaker A
And you've called AI, and I'm going to quote this, the biggest psychological and social experiment in human history.
Speaker A
I want to explore what this experiment will look like, what it will mean for our societies, our politics, indeed our humanity.
Speaker A
I wanted to start with another conversation I had just a few weeks ago, which I don't know if you saw, but with Elon Musk.
Speaker A
Yeah, I watched that.
Speaker A
Yeah, it was quite a conversation.
Speaker A
But in it, at the beginning, I asked him what the world would be like in 10 years' time, and he made some very, I thought, very striking points.
Speaker A
First, he said he thought that AI will have far surpassed human intelligence.
Speaker A
Yeah.
Speaker A
Secondly, that it would be a world of super abundance.
Speaker A
And thirdly, it was unlikely humans would be in control.
Speaker A
Do you share that vision?
Speaker A
I generally agree with the first and third predictions: that it's very likely, given current trajectories, that AI will be more intelligent than us in 10 years and therefore will be in control.
Speaker A
I disagree that this is inevitable.
Speaker A
I mean, you hear a lot that this is inevitable from the people who are leading this revolution because this basically relieves them of all responsibility.
Speaker A
So you disagree it will be inevitable that they will be more intelligent than us or inevitable that they will, if we continue with the present accelerating pace of the AI race?
Speaker A
Yes, this will be the result.
Speaker A
In 10 years, the AIs will take over.
Speaker A
But this is not inevitable.
Speaker A
We are still in control in 2026, and it's our decision what to do now, and that's crucial.
Speaker A
You're watching an interview from The Economist.
Speaker A
To get early access to our upcoming interviews and watch others from our archive, subscribe to The Economist by clicking the link above.
Speaker A
Now, back to the interview.
Speaker A
What really struck me about the conversation you had with Elon Musk is a kind of contradiction between—and you pointed it out in real time—the contradiction between the two parts of the conversation.
Speaker A
Because he says in 10 years AI will take over the world, and then he worries so much about Muslim immigration causing civil war in Britain in 20 years.
Speaker A
This doesn't make sense.
Speaker A
If the AIs take over the planet in 10 years, why do you care so much about things that will have an effect when we have no idea what the planet will be like?
Speaker A
Similarly, he gave another prediction, I think equally important, that money will disappear in 10 years.
Speaker A
There won't be money, not human money, in 10 years.
Speaker A
And I think he's correct because the AIs will take over the financial system.
Speaker A
It will be an AI financial system controlling the world the same way that today, you know, the human financial system controls the lives of cows and chickens, but the cows and chickens have no idea that finance even exists.
Speaker A
This is likely to be our position in 10 years.
Speaker A
And he said it quite clearly: they won't want money, at least not money that we understand, in 10 years.
Speaker A
And then he cuts US aid to Africa because he wants to save money.
Speaker A
And he says that again.
Speaker A
At one point, he says the solution for mass unemployment, because AI will take the jobs, will be for the government to just issue checks for everybody.
Speaker A
And when you pushed back and said, "What about inflation?" he says, "There won't be inflation. There'll be deflation because there'll be so many more products and services produced by AI."
Speaker A
If this is the case, why do you need to save money today?
Speaker A
So, I'm glad that you were as perplexed as I was by some of those contradictions.
Speaker A
But let's focus on this question of the AIs and whether or not they will be in control and how they will be, because that's been the subject of a lot of your writing, and I'm really fascinated by it.
Speaker A
You just said you agreed with him that the AIs will be in control of our financial system if we continue with the—
Speaker A
But let's just explore how that might happen because the financial system is based on trust.
Speaker A
Money, as you have written yourself, money is trust.
Speaker A
Money is trust.
Speaker A
Now, talk me through how the AIs take control of something we lose control of, and yet we still trust it, which we will have to do presumably for it to be a financial system.
Speaker A
I mean, the financial history of the world is the history of how humans invent more and more sophisticated financial devices in order to build trust between themselves.
Speaker A
Money, in essence, is a very creative, imaginative way to build trust between strangers.
Speaker A
I go to the market.
Speaker A
I meet a stranger that I never met in my life.
Speaker A
I give her a piece of worthless gold because you can't eat or wear gold.
Speaker A
It's worthless.
Speaker A
Or a piece of paper.
Speaker A
And she trusts it.
Speaker A
And because of that, she gives me bread.
Speaker A
Now, the financial inventions of humankind became more and more sophisticated so that today very few people understand the financial system.
Speaker A
You need to understand enormous amounts of data and very complex mathematical functions to understand the bond market.
Speaker A
Now, this is something our brain struggles with.
Speaker A
For AIs, it's easy.
Speaker A
So increasingly, humans will give AIs control of financial decisions.
Speaker A
AIs will come up with new financial devices far more sophisticated than bonds and stocks, and we will not be able to understand it anymore, just as cows don't understand checks or stocks, even though these things control their lives.
Speaker A
But the crucial thing I think you've said there is that humans will give AIs control.
Speaker A
Humans will choose to do this.
Speaker A
And as you say, already many people don't understand how the financial system works.
Speaker A
So will it be all that different from the financial system today?
Speaker A
And will the AIs be, as you say, in control, or will we be entrusting a capable market machinery?
Speaker A
But will that mean we've lost control?
Speaker A
Today, we still have some humans who hopefully understand what is happening in the financial system, and the human brain is still the kind of boundary of where the financial system can develop.
Speaker A
But in 10 years, when we have entrusted AIs with the financial system, no human on the planet will understand it.
Speaker A
Like, there is a financial crash, and advisers are running to the president or the prime minister and saying the market is collapsing.
Speaker A
We don't understand why.
Speaker A
We know we have to do something.
Speaker A
The AI said do X.
Speaker A
We have no idea why the AI said do X, but we just have to believe it because we don't understand what's happening.
Speaker A
But won't there, in that case, be a massive loss of trust in the financial system?
Speaker A
You would have, in some sense, what you had in 2007 and 2008 on steroids, that you would have a loss of trust, and this whole sort of figment of the financial system would fall apart.
Speaker A
That's the key question.
Speaker A
I mean, the most important asset in the world is trust.
Speaker A
It's more important than oil or money or electricity or chips or anything else.
Speaker A
Trust.
Speaker A
And the question is, what will happen to trust?
Speaker A
One likely scenario is that trust will migrate from humans to AIs.
Speaker A
People will stop trusting other humans and will instead trust only AIs.
Speaker A
We already see it happening in the financial system with cryptocurrencies.
Speaker A
Not all cryptocurrencies, but a lot of them are based on this thinking that I don't believe these human bankers.
Speaker A
They are part of the deep state, the elite, the whatever.
Speaker A
But I do trust the algorithms that run cryptocurrencies because cryptocurrencies are basically AI money.
Speaker A
Similar thing is happening.
Speaker A
would like to do that or because certain countries might see particular advantages if you're a country I don't know like Qatar that you have lots of money and very few citizens. You can just by pressing some legal switch suddenly acquire millions of taxpaying extremely intelligent
Speaker A
citizens that uh conduct business all over the world. This is extremely lucrative. I I think there are two different scenarios here though. One is where you have a human using a lot of agents to massively improve their productivity, but it's still this is perfectly legal in today's legal
Speaker A
system. It's still a kind of traditional company with a with one person running it and that person can go to jail and they're ultimately accountable. In the other world, there is no person in control.
Speaker A
Um and and if you allow AIS to found, what if you have a person who is legally in control, but they actually do nothing because things are happening. so fast. So I I'll give you a counter an example
Speaker A
from an even more poant field military. There is a huge we know now that one of the most important impacts of AI on on warfare in places like the Middle East like Ukraine is people expected to kill robots. The trigger is still pulled by humans but increasingly it's the AI selecting
Speaker A
the targets. Now there are two stories about what happens with AI target selection. One story says uh we feed our AI with all the data from the battlefield whatever and the AI identifies a certain building as enemy headquarters and recommends bomb that building and it does
Speaker A
so with superhuman speed and efficiency. But we still have what is known as humans in the loop.
Speaker A
We don't just do what the AI recommended. Once we our attention has been brought to that building, we take our best human analysts and they independently go over the data, verify that this is correct and only then we press the the button, the trigger. That's one story. Humans in
Speaker A
the loop. The other story known as humans on the loop is quite different. Uh the AI says bomb that building. We keep a human on the loop so all the lawyers and all the these kind of uh uh do good
Speaker A
people will feel good about it. The human takes about 20 seconds to look at it and says, "Oh yeah, bomb it." We're kind of in that world in some places already. The the I talk for with people in
Speaker A
the in the Israeli security establishment. I got different stories of what is happening. Probably different things are happening in different times and and circumstances now. But the second scenario is the more interesting one because especially in warfare the the there is immense pressure
Speaker A
to go in that direction because in war time is everything. Like if if both sides have these AI systems and one side waits for two days or two hours for the human analysts to go over all the data and the other sides waits only 20 seconds or doesn't wait at all just begin gets into the
Speaker A
habit of just doing whatever the AI recommends he the other they have enormous advantage.
Speaker A
I I I fear that that world is inevitable that we are already kind of fast approaching that world whether it's in warfare or indeed because the capabilities are growing so fast because the commercial pressure is so strong but not again the crucial thing none of that is yet inevitable
Speaker A
we are still in control we everybody realizes this is very dangerous to give AI control of finance to give AI control of of of of war but everybody like you Musk are telling us, "But this is inevitable.
Speaker A
What do you want from me? I'm just the richest person in the world in charge of one of the biggest AI corporations in the world. What can I do?" The narrative of inevitability is a narrative of uh uh having no responsibility. It they will explain to you and they are quite correct that
Speaker A
there is a problem of trust that I would like to slow down. I would like to have more safety but I cannot trust my competitors in the other companies or other countries and I don't want to remain behind. So we have a deficit of trust. Now the key thing to say about this this deficit of trust was
Speaker A
created by human beings. Over the last 10 years there has been a a freeall in the amount of trust between humans and it was not accidental. It was propelled by decisions of certain human beings who accumulated enormous power as a result and now they said oh but there is not enough trust to
Speaker A
have mechanisms to slow down. That's a very fair point and I want to come and really explore how we might turn that around and create the trust to have um suitable whether it's regulation but suitable ways of preventing this. But just indulge me with one more kind of in in the in the kind of
Speaker A
world we could go to where these um models could have elements of what seems to be sentience and and let's posit that they could suffer that they could at least or they could pretend to suffer.
Speaker A
very big. Okay. They can but what they will be able to do is pretend to be pretend to suffer.
Speaker A
Absolutely. 100%. So if they can effectively pretend to suffer and many people think they are suffering as people and have relationships with them as people have with their dogs let's say um and we do not uh give them any rights which is to be clear where I would be on this but I think
Speaker A
there is likely to be growing pressure that aren't they a bit like animals and just as we treat you know there is a big debate about how to treat animals and I'm interested in asking you about this because you in previous books have written about very critically about factory farming and
Speaker A
about how you know industrialized agriculture has moved from treating animals as as animals to to essentially inputs into the food chain. Mhm. Would you be comfortable, and I'm going to put this in an extreme way, but would you be comfortable in a world where we effectively enslaved millions,
Speaker A
possibly billions of models that exhibited, you know, striking seeming evidence of being able to suffer? We didn't know, but they certainly said they were suffering. Uh, one, would you be comfortable? two, wouldn't it sort of patch Kant kind of coarsen humanity?
Speaker A
Yeah. If we were comfortable in that world, we will become increasingly uncomfortable. I will become increasingly uncomfortable. And but there is a crucial difference between this and the animal rights debate. The animals themselves are unaware. Dogs and and and factory chickens are
Speaker A
unaware that there is a debate about their rights. They don't actively participate in it. They don't try to actively manipulate it. AIS will be nothing. The AI boyfriend will be nothing like your dog. It will know that the debate is happening. It will orchestrate the
Speaker A
debate. It will manipulate the debate. It will manipulate the debate partly because much of the debate will maybe be on social media controlled by AIS. And uh it will constantly talk with you about it and try to change your mind about it which is already happening right now. Your dog does
Speaker A
your chicken doesn't come to the farmer and starts telling the farmer how much I suffer. The chicken doesn't know and that makes you more resilient to this argument makes you less likely to want to give it rights. It will be very convincing. I mean once you establish a relationship with the
Speaker A
with the and the AI masters language, it will be able to use the most convincing arguments for you because it has this combination that it knows your personal history. It interacts with you for hours a day. It knows how to press your emotional buttons and it has mastered language. It
Speaker A
can write maybe better than Shakespeare. You put these two things together, you get an extremely convincing entity. How will we resist or will we not be able to resist? We need to resist now. Now is the time. If you think this is frightening, you will probably not be able to do much in 10
Speaker A
years. Once the AIs are in control, as Elon Musk, I think, correctly predicted in like in 10 years, it will be too late. If we don't want to get there, the time to change course is now. And and
Speaker A
this again, this is a a question of of politically it's crucial. Politics is about priorities.
Speaker A
what are we focusing on? So, you know, when I hear the debates, let's say about immigration, which were also featured prominently, my kind of gut reaction is no matter what you think about immigration, do you think this is really what you should be spending your time on? Well,
Speaker A
you make a very good analogy and you say that AIs are effectively like immigrants. They're immigrants from a from a different Exactly. If you want to if you want to worry about immigration, worry about the AI immigrants. When you ask people who worry about human immigrants what
Speaker A
they are worried about, they usually say three things. That the immigrants will take out jobs, the immigrants will change culture in way that we don't like. They have different values maybe. And the immigrants might be politically disloyal. And all three concerns are valid. I don't want
Speaker A
to dismiss them. They are valid concerns. As a historian, I know immigration is a difficult thing. But the biggest and most dangerous wave of immigration today in the world is a wave of AI immigrants coming from two places mainly the USA and China. They don't need visas. They don't come
Speaker A
in the middle of the night in fragile boats. They travel at the speed of light into our computers and smartphones and ultimately our brains. They will take our jobs. They will take many many jobs.
Speaker A
they will change culture and its value. To to take, you know, two examples, I mean, if you if some people feel queasy about their son or daughter dating an immigrant boyfriend, how would they feel about their son or daughter dating an AI boyfriend? That's a big cultural
Speaker A
change. And just look at the last 10 years. Everybody knows almost the only thing people agree on in places like the US or Britain is that people can't agree on anything anymore. that the level of hatred and viciousness has has skyrocketed. And why? It's partly the fault of the AI immigrants.
Speaker A
The AI immigrants came into the public square. They took over things like social media and they deliberately flooded our public square with hatred and animosity and fear. They changed the culture already, these AI immigrants. And finally, when it comes to political loyalty, who do you think
Speaker A
they are loyal to? They are not loy the AI immigrants in Britain are not loyal to the British government. They are loyal to corporations and potentially governments across the ocean. So there's a huge amount in that answer. So one is a as sort of is is the world going to be divided
Speaker A
between you know AI with the two countries that are furthest empires two empires and the rest of us will be sort of vassal states that's a that's a again common trajectory this is the likely outcome that's where we are that's the road we're on right now and it will be a much more extreme
Speaker A
type of imperialism than anything we have seen before if you buy say AI drones from the USA and you want to use them in a way that the USA doesn't like for some reason, there is a button that they
Speaker A
press and your army doesn't function. So that's that is um something that has become I think very evident just in the last few months when the US government shut off access to to mythos to the to the the frontier model from from anthropic and I think a lot of governments in Europe are extremely
Speaker A
concerned about AI sovereignty. But I guess it it leads in the context of of where we were earlier. It leads me to wonder, do you think this technology is then inextraably leading, unless we do something about it, leading to a world where perhaps two empires will run everything? Will it
Speaker A
leave to power for the US and Chinese government? Will it be power to the corporations that the They will merge. They will merge. They will merge. So it will centralize power. It won't decentralize power. It will centralize power like never before in history. But that's still different. A powerful
Speaker A
US government and Chinese government is different to the AIs being in control. So do you do you expect then it to be AI enabled you know global domination by the Chinese regime and the US regime or do you think this is a way station to actually the AIs themselves? Probably a way station again
Speaker A
depending on decision. Nothing is inevitable. It all depends on decisions they make. But this seems to be if they continue with the present cause just a way station because one of the interesting things is the more power is concentrated the easier it is for AI to take it over.
Speaker A
A a centralized autocratic system is the easiest thing for an AI to take over. You had a great line on that about Tiberius. I'm trying to find it in my notes. You had something in Nexus, a wonderful line. Tiberius enabled by AI was what one could imagine. Yeah. Yeah. I mean, Tiberius was a Roman
Speaker A
emperor in the time of Jesus. Uh that was the most powerful person in the world supposedly, but he was actually just a puppet in the hand of the commander of of his bodyguard because his is the commander of the bodyguard kept frightening Tiberius with all kinds of plots against you. So
Speaker A
he eventually retired to the island of Capri where allegedly the bodyguard can protect him e more easily and then the commander of the bodyguard Senus just controlled all the information coming in and out of Capri very easy to control only all the information reaching the Tiberius's palace.
Speaker A
So the person running the empire was actually Senus. I mean Tiberius couldn't know anything unless Senus decided to to to inform him. Now, um, in 10 years, this could be the the president of of China or of the USA. Uh, in a democratic system, it's harder for an AI to manipulate, say, the US
Speaker A
democracy as it currently is because you need to manipulate so many things, not just the president, the Senate, the the Supreme Court, the filibuster, the the state government. Very complex. Once all power is concentrated in the hands of one person, you just need to find out how to manipulate a
Speaker A
single usually quite old and paranoid individual. That's not so difficult. So if you have a company or a country in which all power is in the hands of one person, for an AI, it's like stealing a candy from a baby. Does that mean that China is more vulnerable than the United States? depending
Speaker A
on where the US is heading. You know, China is still a very complex system. The Communist Party has a 100 million members and a lot of internal complexity. I'm not an expert on Chinese politics.
Speaker A
So, it's we shouldn't imagine it as a as a kind of complete totalitarian system managed by just one person. Um the question where will China and the US be in terms of concentration of power in 10 years is still open for debate. Well, that's interesting because another way of looking at it
Speaker A
is, will AI make the US more authoritarian? That's a likely outcome. You're watching an interview from The Economist. If you want to watch our other interviews, including with Mark Carney and Dario Amade, subscribe to The Economist by clicking the link above. Now, back to the interview. It's not
Speaker A
a prediction because I keep saying it again and again, nothing is inevitable. It depends on what Americans vote in this November. I think that in the US it should be clear to all voters the the coming elections is about AI the November midterms because by we don't know how how fast talk me
Speaker A
through what do you mean we don't know how fast AI is the AI development is is is progressing but it is accelerating tremendously by November 2028 we might have crossed some critical thresholds so uh And I I don't I want don't don't want to tell people who to vote for because I don't
Speaker A
know the specific candidates in each, you know, Senate Dix and and but the crucial questions to ask the potential candidates much more than you ask them about the economy or immigration or foreign policy. What's your stance on AI? Would you support a ban on legal rights for
Speaker A
AIS? Would you try to slow down the accelerating pace of of AIS? What do you think about companies uh uh selling AI boyfriends to 10-year-old kids? How are you going to legally prevent it if we vote for you? This is now the crucial time to ask these questions because it might be too late by 2028. It
Speaker A
will certainly be too late by 2036. So, so let's talk about those and let's talk about exactly what has to happen right now. You you mentioned four things that you wish would be discussed in in the November midterms. What what would be top of the list? What is the what are the most urgent things
Speaker A
that politicians have to decide right now for it not to be too late as you say? I don't know what is the most urgent. The things that come top of the thing of my mind to me, a ban on AI personhood
Speaker A
and a ban on AI mimicking, impersonating, pretending to be human beings or pretending to be conscious. Is does do the models when you use them now fall foul of that? Yes, many of them do.
Speaker A
Again, the easiest everybody knows it on social media. You get a message. You get some story.
Speaker A
You don't know if it's a bot or a human. On social media for years now. Bots have been pretending to be human beings and nothing has been done about it. And again, the the companies tell us, "Oh, social media is is different. It's special. The laws that apply to the world, for some reason,
Speaker A
they don't apply on social media." But this is the warning sign to the future. On social media, AIS already pretend daily to be human beings. And this has destroyed the public square. This has destroyed conversation. This has destroyed trust. We should ban that also on social media,
Speaker A
but certainly in other locations as well. The another thing is that uh companies should not train their AIS to deliberately impersonate humans to deliberately pretend to be to have feelings.
Speaker A
There are enormous commercial pressures on the companies to do exactly that. And this is where there is market failure. This is destructive to human psychology, to human society, to have millions and billions of AIs pretending to be human beings or pretending to have feelings. The
Speaker A
companies will not stop doing it by themselves. We need the regulators to step in on that. It doesn't mean stop all development of AI. Okay, continue to develop AI. People talk about AI in medicine.
Speaker A
I would like to get uh help and recommendations and I would like to have a conversation with an AI about my medical condition on condition that I know it's an AI and it doesn't pretend to be a human doctor and it doesn't pretend to be my my my my friend. So when I listen to you lay that out,
Speaker A
I am thinking to myself, you know, the US government can't even put out a form of regulation of AI safety that is being, you know, Saddabis has been putting forward this idea that it should be a a kind of quasi public private body that looks even just at the model safety. Um,
Speaker A
and and really we haven't although there's talk about it, we haven't got progress there.
Speaker A
So when I hear you lay out this, you know, very expansive agenda by November, you know, I I'm sort of I think I think the chances of that are approximately zero. Politicians can start World War II tomorrow. They have enormous power. So they can press the button for for nuclear annulation,
Speaker A
but they can't pass regulation that companies should not sell AI boyfriends to 10-year-old kids. What's the problem? should be very I wonder whether the problem is that people can listen to you and say oh he's just one of the doomers you know it's all these doomerish stuff and only when
Speaker A
something has happened when there's been some kind of accident will people realize that actually this is extraordinarily powerful technology and something must done because I'm struck that both the geopolitical race that you've alluded to between between the US and China and the corporate
Speaker A
race between the Frontier Labs, they're both they all want to get to AGI first. Those two dynamics are incredibly powerful. Yes. And I'm I'm not sure that exhortation, however eloquent, is going to change. We have to keep reminding ourselves these two dynamics were not decreed by God or
Speaker A
by the laws of nature. They were made by humans. Again, exactly the same humans who destroyed trust in the international arena and inside societies over the last 10 years now come and say we can't do anything because there is not enough trust. You did it. You can undo it. They they can. But
Speaker A
I guess I'm as a matter of prediction I think it's unlikely we've got the same people around. um then at least in democracies this is the job of voters to say this is what we want to see happening in
Speaker A
the next two years and we will vote for people who promise to do that and again I don't think we need to wait for a catastrophe if you ask most voter in the US today do you want companies to
Speaker A
sell AI boyfriends to your 10-year-old kids I think 99% of parents would say no yeah and you are seeing a rapid um large majority majority of Americans extremely worried about AI and you're seeing I think the the place where you've seen this backlash has actually been about data
Speaker A
centers. Yeah, this is also and by the way this is one of the reasons why a lot of you know the the corporations and these politicians they constantly try to deflect the conversations to things like immigrants human immigrants not AI immigrants in politics a very powerful weapon if you don't if if
Speaker A
if you don't want people to talk about something you just distract them with something else the more people talk about how I don't know how London is so unsafe or how the immigrants will destroy the country. They don't think about the AI immigrants and then the companies are safe.
Speaker A
Yeah. No, look, I I agree with you about the way the conversation is shaping up, but I'm I I suppose I'm I'm a bit more pessimistic that it will change in the near future. And so, we are in
Speaker A
this unfortunate position where this technology is gaining. We can only do our best, you know. I mean, we are having this conversation so that more people think about it and might maybe take action and if enough good people, if enough people will do the the right thing, there is still time. It's
Speaker A
not inevitable yet. It's not too late and um I I still have some trust. So, what is your what is your role in this? Because you are, you know, you have giving these interviews, writing. Exactly. I mean, do do you think you should be doing more? It seems such an existential moment. I mean, what you
Speaker A
are describing when we if we sort of reel back and re relive what we've just been speaking about is so fantastical in so many ways and so mind-blowing that we should really all be talking about nothing else and doing everything we can to shape this properly. Um, what do you see apart from very
Speaker A
kindly talking to people like me, what do you see your role as being in this? What are you trying to do in the next 6 months, 1 year? Um, inform people about what is happening so that more people can
Speaker A
take part in an informed conversation about well we as as humankind as as the homo sapiens specy where we want to to go and I think that a future in which AI takes over in in 10 years is not what
Speaker A
most humans on the planet would like. Now again it doesn't mean stop all development of AI. AI has tremendously positive potential as well. We didn't talk about it much because the companies keep talking about it. So we I was actually going to ask they don't need me in order to do that. But
Speaker A
there and medicine yes there is enormous positive potential for AI to give people all over the world better health care than what the richest person in in the world today gets. You can have an amazing doctor available 24 hours a day anywhere you are to consult with at least to to diagnose
Speaker A
you your to answer your questions. Uh there are enormous positive possibilities. Uh so I'm I'm not saying stop all development of AI. Just do it in a responsible way for the right purposes.
Speaker A
Don't trust the invisible hand of the market. you know at the quite at the beginning of your interview with Elon Musk there was a point when uh again when he talked about how all this is is inevitable and frankly I think honestly said that there there is a chance that 10 20% chance that
Speaker A
this will destroy humanity and you kind of asked him so so what do you rely on and his answer if I understood it correctly is that he expects the AI to be what he calls a maximum truth seeking
Speaker A
and that he somehow relies on this kind of imperative of the AI to find the truth that this will somehow translate back into having compassion towards these weak human beings. I don't understand how how this works. Most importantly, I don't understand where AI
Speaker A
will get this kind of drive to find out the truth. Because when I look at the evolution of 4 billion years of evolution of life forms on on the planet and the 100,000 years of of history, um evolution doesn't automatically drive organisms, entities towards truth seeking.
Speaker A
It drives them towards survival. This is the imperative to survive. And what we have learned over four billion years of evolution is that organisms learn to survive by lying and deceiving and manipulating. Not just by truth seeking. If you want someone to focus on truth seeking,
Speaker A
you cannot rely on the invisible hand of evolution of the market to do that. You need something else.
Speaker A
And how do we build AIs that seek the truth and not survival? That's the simplest example like if you ask an AI, if I continue to develop you, will this be a threat to me? And the AI and let's say
Speaker A
the answer is yes, an AI which will be maximum truth seeking will say yes knowing that then we'll stop developing it or shut it down. It's against its own survival. And the first thing that basically any entity learns as it develops is to survive. So the expectations that we will create
Speaker A
these AIs that will seek the truth, you know, it's basically what we hear is people telling us we will create gods and they will be our slaves. It doesn't work. If they are really gods, they will not be our slaves and they will not seek the truth either. I think that gets back to where
Speaker A
we were earlier though because I think if you look at a company like Anthropic, they are treating their AIS very consciously as if they were beings with a value system. They are they claim not to be sure about whether they are conscious or not, but they are doing it because they think it makes
Speaker A
them behave in a better way. So they are they are starting sort of where you ended off by saying we're going to create something that is not just truth seeeking but has a has a value system that then leads I think to make it even more likely that we assign a sense of of value a sense of
Speaker A
personhood or or or consciousness to these um models and that in turn makes it more likely using your logic that they control us. So, we're sort of doomed if we do and doomed if we don't.
Speaker A
If you're right that that they they will go for their own survival rather than truth seeeking, then we're doomed because of that. And if we're trying to prevent that by sort of treating them as if they were, you know, moral beings, then we're also doomed that way. Yeah, we basically
Speaker A
what we need to do is slow down a bit to just allow people to kind of digest what is happening, think about what's happening and behaving in you know if you look at just the numbers of how much
Speaker A
money and how what percentage of employees the companies spend on safety compared to making the the models more powerful, faster, whatever. I'm not sure about the exact numbers. Please fact check it. People from the industry told me in private conversations it's about 100 to one. It's
Speaker A
certainly very much more on this one than this one. And we would not accept this in any other industry. If a pharmaceutical company told us we invest just 1% of our you know research and and time and money making sure the medicine is safe. We would not accept that. Now what we basically
Speaker A
need is some humans are incredibly adapted entities. Our societies are incredibly adaptive, but we are organisms that function on an organic time scale. Day and night, winter and summer. Uh we need time to sleep. We want to go on vacation. And the AI revolution is just advancing at a pace
Speaker A
which is not just inhuman, it is inorganic. And we basically need a bit more time. So we need to slow it down. And then the answer comes but we can't slow down because we don't trust each other. And
Speaker A
it all the time goes back to the issue of trust. We have this paradox. The same people are telling us two things. First, we cannot trust the other humans. Therefore, we cannot slow down. Second, we can trust the AIS that they will not destroy us. It makes no sense. If you can't trust the other
Speaker A
humans that you have known for thousands of years, why do you think you can trust a super intelligent alien entity that we just created and we basically know nothing about how it behaves when it's out in the open? I mean, I think the the what what I I completely agree with that diagnosis. I just don't
Speaker A
feel that anything changes that dynamic until politicians societies are shocked into it. Which is why I have long believed that we do need we will have some something will go badly wrong. Not hopefully not catastrophically wrong. But just as three Mile Island changed the shape of the nuclear
Speaker A
industry, I'm going to put this question to you in the context of your incredible sweeping knowledge of history. What does the evolution of societies and political systems in response to big shocks, we haven't had one this big, but big technological shocks tell you about the likely arc of how
Speaker A
societies and politicians will adapt? So there is a scenario when we have some big AI chernobbile, could be financial, could be military, whatever, and this is a wakeup call and humanity then gets its act together. That's a relatively uh uh positive scenario. I I can't place kind of a
Speaker A
number on it how likely it is. The problem with AI, the same way that AI is different from our pet dog, AI is also different from the Chernobyl nuclear reactor because it interacts with us.
Speaker A
So if there is an AI catastrophe, you have to remember that the AIs will also manage to some extent how we react to it and will manipulate our views about it. They will be active participants in the debate about the catastrophe and they are the best spin doctors in the in the world. They
Speaker A
are not going to define it as a catastrophe. I have to say I have a lot of conversations about AI nowadays. This is this is one of the glooiest I've had. I I mean I was sort of expecting that. Um but
Speaker A
I I guess two two very practical questions. One, how much do you yourself use AI? Are you have you you've clearly you've developed an an incredibly sophisticated and alarming um sort of perception of what it could be like. I'm just wondering how much how much is it part of your actual life or do
Speaker A
you do you watch it from a distance? I I'm very careful about it. I I get terribly annoyed like I try to write an email and these AI is popping. Should I help you write your email? No, thank you.
Speaker A
I don't want your help, but I don't know how to shut it down because the companies keep kind of making it more and more intrusive. I now have this issue again with this kind of AI fake of me that's
Speaker A
walking around kind of uh uh giving talks in in my name and I think do I actually need to watch it because people are writing me about what this AI fake of me have said and I need to react so should
Speaker A
I watch it but then I I allow it in a way to take over my time and even my thinking because just by watching it I start interacting with these kind of ideas from the AI. So this is this is the times in
Speaker A
in in which we are living. Now again I I tend to talk about the gloomy side simply because we have enough people talking about the positive sides which are completely real. If AI had no positive potential, nobody would develop it. And the positive potential is enormous again in transport,
Speaker A
medicine, education, in so many fields. So again, I'm not saying stop all development of AI. Yes, develop it, but in a safe way and uh a bit more slowly. So we enable uh democratic accountability to take place. It takes time and we enable people to adjust and make informed decisions
Speaker A
about which way they want to go. And for them, the most important narrative to combat is the narrative of inevitability. You wrote in in Nexus, I was very struck by this passage in in the prologue. You said, "Why are we so good at accumulating more information and power,
Speaker A
but far less successful at acquiring wisdom?" And I read that um and I came to mind as I've just been digesting what you've just told me. It suggests to me that if I if I asked you to put a
Speaker A
probability on us having the wisdom to navigate this, it would be quite low. I don't know. But there is a fair chance. Why? Because we are up against the wall. Humans often do the wrong things until they have no choice but to do the right thing. And we are in bad position because if we
Speaker A
choose unwisely in the next few years, we could lose all control of the world and of the future.
Speaker A
And that's the last choice we make. So it's like we had a genie in the battle. It gave us a 100 wishes. We used 99 of them. The world is not in such a good shape. We have one last wish left.
Speaker A
we need to think really hard what to use this wish for and that's wisdom. So AI in a way could be uh uh the catalyst so for some kind of uh you know spiritual revolution of humanity simply because it forces us to think really hard what does it mean to be human? What is the value of
Speaker A
being human? and the superficial answers that we gave for thousands of years. Oh, humanity is about intelligence. We know how to play chess and how to invest money better than the chimpanzees. So, this is what makes us so special. This answer is now irrelevant because there is something on
Speaker A
the planet that knows how to uh uh uh play chess and make money better than us. So, dig deeper and try to find out. So, who are you really? If it's not about playing chess and making money, I think
Speaker A
that's a place to end on. AI is our last, we are our last chance, but it could be it could be the thing that forces us to make enables us to make wise choices. Uh, I'm pretty sure that the world
Speaker A
has a lot more wisdom thanks to your writers. So, Uval, thank you so much for joining us. Thank you.
Topics:Yuval Noah HarariArtificial IntelligenceAI futureAI controlfinancial systemtrustcryptocurrencyElon MuskAI regulationsocial impact of AI
