Speaker A
Okay, let's start with a game. Okay? And to win this game, all you have to do is see the reality that's in front of you as it really is, all right?
Neuroscientist Beau Lotto shows through optical illusions that color and brightness perception depend on context, since the eye receives ambiguous light and the brain learns meaning from experience.
Key Takeaways
- Perception is not a direct readout of the physical world; the brain interprets ambiguous sensory input using context and past experience.
- Light intensity and color alone are meaningless without the surrounding context that determines their significance.
- Seeing is a learned, adaptive process shaped by evolution and behavioral meaning.
- The brain can rapidly redefine what counts as 'normal' when environmental context changes.
What the video covers
- Lotto opens with a color-matching game to show that people often misperceive the colors of dots depending on their surroundings.
- He argues that seeing color is one of the brain's simplest functions, yet context shapes it at the most fundamental level.
- The light reaching the eye is ambiguous: the same retinal image can result from countless real-world sources, so sensory information is inherently meaningless on its own.
- Illusions like identical tiles on light and dark surrounds show that the brain interprets brightness based on inferred illumination, shadow, and surface properties.
- Color evolved to help organisms distinguish surfaces and detect predators, which is why it matters for survival rather than being a purely passive reading of light.
- The brain learns to see by finding patterns and associating them with behavioral significance through interaction with the world.
- Experience redefines normality: after exposure to colored light, the brain adjusts its interpretation, so identical images look different.
- The same principles apply beyond color to motion perception, language, and reading, where the brain fills in missing information based on past statistics.
- Lotto concludes that how we perceive reveals not only how we see but who we are as individuals and as a society.
Chapters
Full Transcript — Download SRT & Markdown
Speaker A
So we have two panels here, of colored dots. And one of those dots is the same in the two panels.
Speaker A
So we have two panels here, of colored dots. And one of those dots is the same in the two panels.
Speaker A
So by a show of hands, we'll start with the easiest one. Show of hands: how many people think it's the gray one?
Speaker A
And you have to tell me which one. Now, I narrowed it down to the gray one, the green one, and, say, the orange one.
Speaker A
Pretty even split. Let's find out what the reality is. Here is the orange one.
Speaker A
So by a show of hands, we'll start with the easiest one. Show of hands: how many people think it's the gray one?
Speaker A
All right? (Laughter) So this is pretty amazing, isn't it? Because nearly every living system has evolved the ability to detect light in one way or another.
Speaker A
Really? Okay. How many people think it's the green one? And how many people think it's the orange one?
Speaker A
What I'm going to talk about is not that context is everything, but why context is everything.
Speaker A
Pretty even split. Let's find out what the reality is. Here is the orange one.
Speaker A
But first, we have to ask another question, which is, "What is color for?" And instead of telling you, I'll just show you.
Speaker A
(Laughter) Here is the green one. And here is the gray one. (Laughter) So for all of you who saw that, you're complete realists.
Speaker A
Now, can any of you see the predator that's about to jump out at you?
Speaker A
All right? (Laughter) So this is pretty amazing, isn't it? Because nearly every living system has evolved the ability to detect light in one way or another.
Speaker A
Now let's see the surfaces according to the quality of light that they reflect. And now you see it.
Speaker A
So for us, seeing color is one of the simplest things the brain does. And yet, even at this most fundamental level, context is everything.
Speaker A
But what you've just done is in many respects mathematically impossible. Why? Because, as Berkeley tells us, we have no direct access to our physical world, other than through our senses.
Speaker A
What I'm going to talk about is not that context is everything, but why context is everything.
Speaker A
You vary any one of those parameters, and you'll change the color of the light that falls onto your eye.
Speaker A
Because it's answering that question that tells us not only why we see what we do, but who we are as individuals, and who we are as a society.
Speaker A
Let me show you what I mean. Imagine that this is the back of your eye, okay?
Speaker A
But first, we have to ask another question, which is, "What is color for?" And instead of telling you, I'll just show you.
Speaker A
They are the same, as far as your eye is concerned. And yet they come from completely different sources.
Speaker A
What you see here is a jungle scene, and you see the surfaces according to the amount of light that those surfaces reflect.
Speaker A
The one on the left comes from an orange surface, under direct light, facing to the right, viewed through sort of a bluish medium.
Speaker A
Now, can any of you see the predator that's about to jump out at you?
Speaker A
So how on Earth do we even see? So if you remember anything in this next 18 minutes, remember this: that the light that falls onto your eye, sensory information, is meaningless, because it could mean literally anything.
Speaker A
And if you haven't seen it yet, you're dead, right? (Laughter) Can anyone see it? Anyone? No?
Speaker A
It's what we do with that information that matters. So, how do we see? Well, we see by learning to see.
Speaker A
Now let's see the surfaces according to the quality of light that they reflect. And now you see it.
Speaker A
We're very aware of this in the form of more cognitive attributes, like language. I'm going to give you some letter strings, and I want you to read them out for me, if you can.
Speaker A
So, color enables us to see the similarities and differences between surfaces, according to the full spectrum of light that they reflect.
Speaker A
There's no a priori reason why an "H" has to go between that "W" and "A." But you put one there. Why?
Speaker A
But what you've just done is in many respects mathematically impossible. Why? Because, as Berkeley tells us, we have no direct access to our physical world, other than through our senses.
Speaker A
So you do so again. And yet you don't put a letter after that first "T." Why? Because it wouldn't have been useful in the past.
Speaker A
And the light that falls onto our eyes is determined by multiple things in the world, not only the color of objects, but also the color of their illumination, and the color of the space between us and those objects.
Speaker A
So if I could have the lights down up here. I want you to first notice that those two desert scenes are physically the same.
Speaker A
You vary any one of those parameters, and you'll change the color of the light that falls onto your eye.
Speaker A
And I want you to stare at that dot. Don't look anywhere else. We're going to look at it for about 30 seconds, which is a bit of a killer in an 18-minute talk.
Speaker A
This is a huge problem, because it means that the same image could have an infinite number of possible real-world sources.
Speaker A
Your brain is learning, and it's learning that the right side of its visual field is under red illumination; the left side of its visual field is under green illumination.
Speaker A
Let me show you what I mean. Imagine that this is the back of your eye, okay?
Speaker A
So why don't you do that now? (Laughter) Can I have the lights up again?
Speaker A
And these are two projections from the world. They're identical in every single way. Identical in shape, size, spectral content.
Speaker A
That's your new normal. Okay? So, what does this mean for context? It means I can take two identical squares, put them in light and dark surrounds, and the one on the dark surround looks lighter than on the light surround.
Speaker A
They are the same, as far as your eye is concerned. And yet they come from completely different sources.
Speaker A
So I'll show you what I mean. Here we have that exact same illusion. We have two identical tiles on the left, one in a dark surround, one in a light surround.
Speaker A
The one on the right comes from a yellow surface, in shadow, oriented facing the left, viewed through a pinkish medium.
Speaker A
And see what happens to your perception. Notice that on the left the two tiles look nearly completely opposite: one very white and one very dark, right?
Speaker A
The one on the left comes from an orange surface, under direct light, facing to the right, viewed through sort of a bluish medium.
Speaker A
Why? Because if the tile in that shadow were in fact in shadow, and reflecting the same amount of light to your eye as the one outside the shadow, it would have to be more reflective -- just the laws of physics.
Speaker A
Completely different meanings, giving rise to the exact same retinal information. And yet it's only the retinal information that we get.
Speaker A
If they're under the same light reflecting the same amount of light to your eye, then they must be equally reflective.
Speaker A
So how on Earth do we even see? So if you remember anything in this next 18 minutes, remember this: that the light that falls onto your eye, sensory information, is meaningless, because it could mean literally anything.
Speaker A
This is one I made a few years ago. And you'll notice you see a dark brown tile at the top, and a bright orange tile at the side.
Speaker A
And what's true for sensory information is true for information generally. There's no inherent meaning in information.
Speaker A
Here you see four gray tiles on your left, seven gray tiles on the right.
Speaker A
It's what we do with that information that matters. So, how do we see? Well, we see by learning to see.
Speaker A
And see what happens to your perception. The four blue tiles on the left are gray.
Speaker A
The brain evolved the mechanisms for finding patterns, finding relationships in information, and associating those relationships with a behavioral meaning, a significance, by interacting with the world.
Speaker A
Don't believe me? Let's watch it again. What's true for color is also true for complex perceptions of motion.
Speaker A
We're very aware of this in the form of more cognitive attributes, like language. I'm going to give you some letter strings, and I want you to read them out for me, if you can.
Speaker A
And for all of you, you'll see it probably spinning this direction. Now I want you to keep looking at it.
Speaker A
Audience: "Can you read this?" "You are not reading this." "What are you reading?" Beau Lotto: "What are you reading?" Half the letters are missing, right?
Speaker A
Yes? Raise your hand if you got that. Yes? Keep blinking. Every time you blink, it will switch.
Speaker A
There's no a priori reason why an "H" has to go between that "W" and "A." But you put one there. Why?
Speaker A
So depending on where it looks, it flips between the two possibilities. Are we the only ones that see illusions?
Speaker A
Because in the statistics of your past experience, it would have been useful to do so.
Speaker A
So in my lab we work on bumblebees, because we can completely control their experience, and see how it alters the architecture of their brain.
Speaker A
So you do so again. And yet you don't put a letter after that first "T." Why? Because it wouldn't have been useful in the past.
Speaker A
You can see the queen bee, the large bee in the middle. Those are her daughters, the eggs.
Speaker A
So you don't do it again. So, let me show you how quickly our brains can redefine normality, even at the simplest thing the brain does, which is color.
Speaker A
You see how she has a little number on her? There's another one coming out, she also has a number on her.
Speaker A
So if I could have the lights down up here. I want you to first notice that those two desert scenes are physically the same.
Speaker A
Then you can superglue little numbers on them. (Laughter) And now, in this experiment they get a reward if they go to the blue flowers.
Speaker A
One is simply the flipping of the other. Now I want you to look at that dot between the green and the red.
Speaker A
She's drinking a glass of water that's about that big to you and I, will do that about three times, then fly.
Speaker A
And I want you to stare at that dot. Don't look anywhere else. We're going to look at it for about 30 seconds, which is a bit of a killer in an 18-minute talk.
Speaker A
So they copy each other. They can count to five. They can recognize faces. And here she comes down the ladder.
Speaker A
(Laughter) But I really want you to learn. And I'll tell you -- don't look anywhere else -- I'll tell you what's happening in your head.
Speaker A
(Laughter) Now remember, she's supposed to be going to the blue flowers, but what are these bees doing in the upper right corner?
Speaker A
Your brain is learning, and it's learning that the right side of its visual field is under red illumination; the left side of its visual field is under green illumination.
Speaker A
But those are blue flowers under green light. So they're using the relationships between the colors to solve the puzzle, which is exactly what we do.
Speaker A
That's what it's learning. Okay? Now, when I tell you, I want you to look at the dot between the two desert scenes.
Speaker A
The senses aren't fragile. And if they were, we wouldn't be here. Instead, color tells us something completely different, that the brain didn't actually evolve to see the world the way it is.
Speaker A
So why don't you do that now? (Laughter) Can I have the lights up again?
Speaker A
And how we see is by continually redefining normality. So, how can we take this incredible capacity of plasticity of the brain and get people to experience their world differently?
Speaker A
I take it from your response they don't look the same anymore, right? (Applause) Why? Because your brain is seeing that same information as if the right one is still under red light, and the left one is still under green light.
Speaker A
And they can navigate the world using their ears. Here's David on the right, and he's holding a camera.
Speaker A
That's your new normal. Okay? So, what does this mean for context? It means I can take two identical squares, put them in light and dark surrounds, and the one on the dark surround looks lighter than on the light surround.
Speaker A
That line is broken up into 32 squares. In each square, we calculate the average color.
Speaker A
What's significant is not simply the light and dark surrounds that matter. It's what those light and dark surrounds meant for your behavior in the past.
Speaker A
(Continuous sound) (Sound changes momentarily) (Sound changes momentarily) (Sound changes momentarily) (Sound changes momentarily) (Sound changes momentarily) Beau Lotto: He finds it. Amazing, right?
Speaker A
So I'll show you what I mean. Here we have that exact same illusion. We have two identical tiles on the left, one in a dark surround, one in a light surround.
Speaker A
But we can also do something else. We can also make music with color. So, working with kids, they created images, thinking about what might the images you see sound like if we could listen to them.
Speaker A
And the same thing over on the right. Now, I'll reveal those two scenes, but I'm not going to change anything within those boxes, except their meaning.
Speaker A
And this is what it sounds like. (Electronic representation of orchestral music) So, a six-year-old child. Okay?
Speaker A
And see what happens to your perception. Notice that on the left the two tiles look nearly completely opposite: one very white and one very dark, right?
Speaker A
We're not defined by our central properties, by the bits that make us up. We're defined by our environment and our interaction with that environment, by our ecology.
Speaker A
Whereas on the right, the two tiles look nearly the same. And yet there is still one on a dark surround, and one on a light surround.
Speaker A
Because what I've been trying to do is really celebrate uncertainty. Because I think only through uncertainty is there potential for understanding.
Speaker A
Why? Because if the tile in that shadow were in fact in shadow, and reflecting the same amount of light to your eye as the one outside the shadow, it would have to be more reflective -- just the laws of physics.
Speaker A
So, if we have the lights down. And what we have here -- Can everyone see 25 purple surfaces on your left, and 25, call it yellowish, surfaces on your right?
Speaker A
So you see it that way. Whereas on the right, the information is consistent with those two tiles being under the same light.
Speaker A
Now you can see that changes the light that's coming through there, right? Because now the light is going through a yellowish filter and then a purplish filter.
Speaker A
If they're under the same light reflecting the same amount of light to your eye, then they must be equally reflective.
Speaker A
Now, some of you will have noticed that the consequence is that the light coming through those middle nine on the right, or your left, is exactly the same as the light coming through the middle nine on your right.
Speaker A
So you see it that way. Which means we can bring all this information together to create some incredibly strong illusions.
Speaker A
Do they look the same? No. The question is, "Is that an illusion?" And I'll leave you with that.
Speaker A
This is one I made a few years ago. And you'll notice you see a dark brown tile at the top, and a bright orange tile at the side.
Topics:optical illusionscolor perceptionneurosciencecontextvisual perceptionBeau LottoTEDbrainevolutionsensory information
