Your Eyes Can Change What You Hear

A small experiment about how the eyes and the ears work together — and why that teamwork is different in every language.

---

Try something first. Watch the short video below. The first time, close your eyes and only listen. The second time, watch it again with your eyes open.

The audio never changes. Your ears do not move. But the sound you hear may change — because the second time, your eyes are helping you listen.

🔊 Tap the video and turn your sound on — this only works with audio.

What just happened?

This is a famous trick of the brain called the McGurk effect. When we listen to a person, we do not only use our ears. We also watch their lips, their teeth, their tongue, and their jaw. Our brain then mixes what we hear with what we see, and gives us one single sound.

Most of the time, the eyes and the ears agree, so we never notice. But when they disagree — when the ears hear one thing and the eyes see another — something surprising happens. Very often, the eyes win.

A happy accident in 1976

This effect has a name and even a birthplace. It was discovered by accident in 1976 by two scientists, Harry McGurk and John MacDonald, at the University of Surrey in England. They were studying how babies learn to understand speech.

One day, a technician joined the sound “ba” to a video of a mouth saying “ga.” When the scientists played it back, they both heard a third sound — “da” — which was not in the audio and not in the video. Their brains had quietly built a brand-new sound out of the eyes and the ears together. They wrote about it in the science magazine Nature, in a short paper with a lovely title: “Hearing Lips and Seeing Voices.” Scientists have pointed to it more than 4,800 times since.

Hearing is not just something your ears do. It is a decision your brain makes — using every clue it can find.

The funny cousin: when your brain hears what it expects

Your eyes are not the only thing that can fool your ears. Your expectations can do it too. The British comedian Peter Kay built one of his most famous jokes on exactly this idea. He plays a small piece of a well-known song, then tells the audience the wrong words — and suddenly everyone hears the silly version instead of the real one. And here is the strange part: once you hear it his way, you can’t hear it the right way again.

Peter Kay, The Tour That Didn’t Tour Tour — his classic “misheard lyrics” routine.

The McGurk effect uses your eyes; Peter Kay uses your expectations. But both open the same door in the brain: we do not simply receive sound — we guess it, using everything we already know. A wrongly heard lyric even has its own special name: a mondegreen.

Why this is Part Two

In Part One, we said that the language you speak shapes the way you think. Part Two goes one step deeper, and one step stranger: the language you speak can shape the way you sense the world — even the way you hear.

Researchers tested the McGurk effect with people from many countries. They found the eye–ear connection is not the same strength for everyone.

• A strong connection: speakers of English, Spanish, Italian, German, Dutch, and Turkish tend to feel this illusion strongly. Their eyes pull hard on their ears.
• A weaker connection: speakers of Mandarin Chinese and Japanese often feel it much more weakly. Their ears hold their ground.

Why would this be? Scientists are still studying it, but there are two interesting ideas. First, Chinese and Japanese carry a lot of meaning in tone and pitch — information the ears must catch very carefully — so these listeners may learn to trust their ears more. Second, cultural habits around eye contact and watching faces may also play a part.

There is one more clue that this is built and not born. In English-speaking children, this eye–ear teamwork grows noticeably stronger between the ages of 6 and 8. In Japanese children, it stays much flatter. So a big part of it is quietly shaped by the language and the culture we grow up inside.

What this means if you are learning English

If your first language leans on the ears, English listening can feel harder than it should. English hides a lot of its information on the lips and the face. The good news: you can train your eyes to help.

Face to face, every sound has a shape you can see.

Watch mouths. Choose video, not only audio. In our classes, the teacher faces the students on purpose, so they can see the shape of every sound. Reading the lips is not cheating — it is exactly how fluent listeners already listen, without even knowing it.

One last thought

We like to believe that hearing is simple: a sound goes in, and we know what it is. But two people can sit in the same room, listen to the same voice, and honestly hear two different words — only because they grew up speaking different languages.

Your language is not only in your mouth. It is in your ears — and, it turns out, even in your eyes.