I’m on the back seat of the lower deck of a number 37 bus, outside the red-brick and Portland stone clock tower of Lambeth Town Hall in Brixton, south London.  Although I know exactly where I am, I feel lost. I no longer know whether to trust what my eyes are telling me.

I’ve just been told by a leading vision scientist that I have no real depth perception. 

In other words, I have never seen in three dimensions the way most people do.

The lazy eye I’ve had since early childhood means that my brain largely ignores the signals from my right eye, leaving my world flatter than most people’s. Now I’m wondering what the clock tower looks like to my fellow passengers. What can the woman two seats in front see in the son she has on her lap that I can’t see in mine? Which parts of mountain streams, sea birds and other natural sights am I missing out on? And does this explain why my strike rate on the football pitch is so dismal?

I’d met up with Robert Hess from McGill University, Montreal, Canada, during a trip to London to find out about his research using video games to treat lazy eye. Pilot trials suggest the approach could help adults with the condition – an improvement on established treatments, which are mostly considered to be effective only up to the age of about seven. Even more remarkably, some of the participants in Hess’s and related research have provided extraordinary accounts of what it’s like to begin seeing in three dimensions for the first time.

Hess’s group is one of at least five working on the use of video games to treat lazy eye. It’s hoped that the work could soon offer affected children a less onerous form of treatment, as well as potentially reducing the risk of blindness or severe vision loss for adults with the condition.

But more than this, the knowledge generated by this research is helping scientists understand that the human brain is more open to being rewired in later life than was previously believed. The tantalising prospect is of interventions that would make it easier to reverse problems with cognition caused by things like stroke and brain injuries. As the bus pulls away from the stop, I start to wonder if video gaming could fix my lazy eye and give me my first taste of true 3D vision.

§

The brain develops the circuits that allow us to bring things into sharp focus and see in 3D in the first three to five years of life. However, if the vision in one eye is blocked or undermined in early childhood, as happens in about 3 per cent of people, the result can be amblyopia, more commonly called lazy eye.

Studies have found that people with lazy eye can be significantly slower and make more mistakes when reaching for objects, as well as being at greater long-term risk of becoming blind or suffering significant vision loss. What’s more, this risk is more than twice as likely to affect their good eye, rather than their lazy eye, probably because the good eye is the one they lead with.

There are two main causes of lazy eye. Strabismus, or crossed eye, is when the two eyes don’t look in the same direction. Anisometropia is when one or both eyes are short-sighted, long-sighted or have an astigmatism (an unevenly curved cornea or lens), and the difference between the visual signals they produce is large. In both strabismus and anisometropia, the brain cannot cope with the difference between the two sets of images and so favours one over the other.

Patching the good eye to improve the weaker has been the main treatment for lazy eye for over 200 years. But estimating the success of this approach is difficult because it varies according to the age at which patching is done, how soon it is tried after onset, and whether or not children wear their patches properly. Some estimates suggest it is effective in only between a quarter and a half of cases.

“Patching is hard on the kids as it forces them into a partially sighted world,” says Hess. “They get ribbed, it only has any effect in about half of cases and even in those improvements can wear off after the end of treatment.”

I know about both the poor success rate and other downsides of patching only too well. A family photo album belonging to my parents contains a fading local newspaper cutting from around 1977. It pictures my classmate David and me alongside the Formula One world champion James Hunt at a garage in the then mining town of Dinnington, South Yorkshire. It is – fair to say – a picture of contrasts. Hunt, with his shaggy blond hair and film-star looks, is tanned and smiling broadly. David is wearing ill-fitting NHS glasses and I have a white cloth patch over my stronger left eye.

For the couple of years I had to wear the patch, I hated the ill-defined, fuzzy version of the world my weaker eye provided almost as much as I hated the mockery of my classmates. I often removed the patch when my parents and teachers weren’t looking, and was greatly relieved when, at age eight, the doctors brought my misery to an end by declaring the treatment a failure.

§

Vision is just one of many functions that go through what are called sensitive or critical periods, when the associated brain circuits are unusually open to being moulded by sensory experiences.

Modern ideas about lazy eye and visual plasticity (or flexibility) are rooted in research by neurophysiologists David Hubel and Torsten Wiesel, who in 1981 got a Nobel Prize for their work unpicking the basics of how our brains process visual information. In particular they showed how depriving one eye of vision could lead to lasting visual impairment. When young kittens that had had one eyelid sewn shut for several weeks from birth had it reopened, brain cells responded only to signals from the eye that had remained open. Crucially, normal vision did develop if the covered eye was uncovered and the open eye covered when the kittens were young, but not if this was done later in life.

While Hubel and Wiesel did not extrapolate these findings to humans, others did. Today, we treat lazy eye only up to the age of around seven because it has been widely assumed that beyond then the brain is not plastic enough for it to be effective.

“A lot of people misinterpreted their work as saying there is plasticity only until a fixed age,” says Dennis Levi, a vision scientist from the University of California. “The idea that after around seven you’re basically over the hill in terms of treatment became the clinical understanding.”

The seeds of a new way of thinking about brain plasticity were sown around 20 years ago. During the mid-1990s, Levi found that some forms of acuity, or clarity of vision, could be regained in adults with lazy eye if they repeated specific visual exercises for long periods – an approach known as perceptual learning. “We had some encouraging results, but with a boring repetitive task that had to be done thousands of times for hours and hours, so the question was how do we make it more user-friendly?”

The answer came from one of those serendipitous moments that litter the history of scientific discovery. In 1999, Daphne Bavelier, a neuroscientist at the University of Rochester in New York State, was bemused by the high scores achieved by a researcher in her department and his friends in a computer-based test designed to study the effects of congenital deafness on visual attention. Eventually she linked this to their fondness for the action video games Counter-Strike and Team Fortress Classic.

Bavelier went on to show that video-game players did better than non-gamers on a series of cognitive measures including rapidly processing complex information, estimating object numbers, switching attention between tasks and focusing on important things while ignoring distractions. She also showed than non-gamers who were asked to spend time playing a complex, first-person action game did better on such measures than a group who played a simpler, abstract game. Her finding that gaming could help improve contrast sensitivity – the ability to detect small differences in shades – attracted particular interest because the consensus at the time was that this could not be improved through training.

“Part of what we were doing was questioning just how critical the critical periods in the Hubel and Wiesel model really are,” says Bavelier. “We think there may be ways to reopen them and get the brain to rewire itself to help adults with amblyopia [lazy eye].”

In the UK one group had already started work on a video-game-based treatment. Computer scientists and vision researchers at the University of Nottingham teamed up in 2003 to create the Interactive Binocular Treatment for Amblyopia (I-BiT) system. The idea was to train the brain to take more notice of more interesting, active game content or the central action of video clips presented to the lazy eye at the expense of less interesting, peripheral elements shown to the dominant eye.

Those who have tried modern virtual reality headsets may chuckle at the team’s description of an early prototype as “virtual reality”. It was a large wooden tea chest with a hole in the front through which patients could look at two separate screens within, one for each eye.

Back across the Atlantic, Levi was intrigued by Bavelier’s findings that a range of cognitive functions, including visual ones, could be sharpened through gaming. His own 2011 pilot found that adults with lazy eye who played the first-person shooter Medal of Honor for 40–80 hours with their good eyes patched significantly improved their sharpness of vision.

Despite some apparently promising early results, this monocular approach soon fell out of favour, in large part because of another accidental discovery.

In 2006, Hess discovered something that went against prevailing wisdom. When the signals going to the eyes were altered to take account of the lazy eye’s weakness, people with lazy eye could combine information from both eyes. This suggested that the condition was caused by the brain’s suppression of signals from the weak eye. While trying to measure this suppression in participants who did multiple sessions in his lab, Hess found that suppression actually fell over time. “We’d set out to measure suppression and ended up treating it,” he says.

§

I’m in a curved grey corridor trying to pick up treasure while keeping an eye out for the bots that are out to get me. It’s five months since Hess revealed to me the flatness of my world. Having decided to find out whether this novel approach to fixing lazy eye could work for me, I headed to Levi’s office in the UC Berkeley School of Optometry, near San Francisco.

To play the game that he and Bavelier devised, I peer into a stereoscope, a metal frame containing mirrors that are bouncing two different images from a split-screen monitor into my two eyes separately. The brightness, contrast and alignment of the images have been adjusted to take into account my weak eye. In theory, this means that my brain can now fuse the inputs from both eyes and I should be able to shoot the hovering grey patches that are periodically presented only to my weak eye. In reality, I either fail to see them or just get vague confusing flashes of colour, so the patches keep turning into monsters and killing me. Despite the valiant efforts of a PhD student called Christina to make adjustments, I can’t get the hang of it and give up after about 30 minutes.

Of course the success of many therapies varies between patients. Eric Gillet, 52, tells me how three years ago he saw in 3D for the first time after just a few sessions using Levi’s system. A financial analyst at UC Berkeley, Gillet was walking through the towering redwood, pine and eucalyptus trees that provide the unique atmosphere of the campus, on his way to meet a friend for lunch. Glancing up at an oak tree, he realised something had changed. “Suddenly I could see the space between its branches and the sky. They were no longer flat and stuck together. It was just a few seconds, but it was very powerful.”

Gillet’s lazy eye was diagnosed late, at the age of six, in his native Belgium and he first learned of his lack of 3D vision about a decade ago during a standard eye test. In 2012 he jumped at the chance to become one of 38 adults with lazy eye taking part in a pilot study of Levi’s experimental therapeutic game. This time around, Levi ditched his previous monocular approach (trying to work the weak eye while the strong one was patched) in favour of a binocular system (which degrades images presented to the strong eye in an effort to persuade the brain to take more notice of the signals from the lazy eye).

Unlike me, Gillet was able to play the game and it was after, he says, five to ten sessions into the 40 hours of lab time that he had his 3D oak tree epiphany. Three years on, I ask him what the longer-term effects have been. He holds up his left hand and, smiling broadly, says: “I can see my hand in 3D. My 3D is weak but present most of the time.”

Despite Gillet’s enthusiasm, the trial results were something of a disappointment to Levi. The clarity of vision in the weak eyes of adults who played the game over 40 hours improved by the equivalent of just one line on an optician’s eye test chart – less than the line and a half achieved in his 2011 monocular trial.

§

Less than 100 yards from Gillet’s office, I meet Dillon Thomas. He’s learned to see in 3D thanks to testing an early version of a video game designed for the new generation of virtual reality headsets such as Oculus Rift. Thomas was walking Mordi, his border terrier–husky cross, in San Francisco’s Eureka Valley dog park one day last year when someone inadvertently threw a tennis ball towards his face. “Without thinking or having to look to track it, I knew exactly where it was and how fast it was going,” he says.

“I just moved my head to the side and watched it sail past. I swear to God time slowed down like in The Matrix. When you have depth perception, you don’t have to think about it. It just comes to you.”

As a child Thomas had several rounds of surgery to correct his crossed eyes, the last of which left him with double vision. More recently his effort to use vision therapy, which involves doing visually demanding exercises while using tools such as prisms and filters, did little to help. Then last spring he came across a report about a San Francisco-based developer called James Blaha who had created a game designed to treat lazy eye on the Oculus Rift virtual reality headset.

Last spring Thomas tried Blaha’s early prototype in an office in central San Francisco. “He dimmed the brightness to my left eye and brightened it to my right, and then suddenly what had been a flat image just popped,” Thomas says. “The cube was floating in front of me. He brought it close to my face and although I knew it was a computer graphic I was physically backing off because it felt so visceral. It blew my mind.”

Blaha lent him the kit to practise with for 20 minutes per day for several weeks. Thomas says he now has depth perception within a few feet, depending on how rested he is feeling. On occasion his 3D world has extended to “two or three blocks” and he describes recent visits to forests as “absolutely amazingly beautiful” experiences.

§

Three years ago, Blaha was working as a web developer and making video games as a hobby. Having had a lazy eye since childhood, he was inspired by a TEDx talk by neurobiologist and fellow strabismic Susan Barry, who, at the age of 48, gained 3D depth perception through vision therapy. Her 2009 book, Fixing My Gaze, includes vivid accounts of the new beauty she was now able to experience in forests and snowfalls.

Blaha had already done hundreds of hours of vision therapy and patching as a child, with little success. After reading about the work of Levi, Hess and others, he set about trying to achieve similar results to Barry, only through gaming rather than repetitive visual exercises. Following the release of the first developer version of the Oculus Rift in 2012, he thought that an immersive 3D environment would provide the most effective stimulus to reprogram the brains of people with lazy eye.

One day in November 2013, he realised he was on to something big while testing software he had written to generate a spinning cube. “I was making the signal brighter and brighter to my weak eye and at a certain threshold it popped into 3D,” he says. As the corner span towards him, he physically backed up. “I’d never experienced anything in 3D like that before.” Not long afterwards he began to have true depth perception for his keyboard and other things in real life too.

A few days later he posted a request for $2,000 on the crowdfunding website Indiegogo, promising to use it to create a therapeutic version of the 1970s brick-breaking game Breakout. Within weeks pledges exceeded $20,000. After completing the first version of the game, he moved from his home town of East Lansing, Michigan, to San Francisco to take up an offer of funding, mentoring, training and office space from tech venture capitalists SOSventures in return for equity in his company. Blaha persuaded his high-school friend Manish Gupta to leave his business analytics job at IBM to come and work alongside him.

They recruited 20 people to play the first version of their game for more than three hours in total over four to six weeks. The resulting data suggested that doing so was, over time, reducing the suppression of signals from the lazy eye. They produced a second game designed to maximise the therapeutic effects and by September 2014 had attracted $700,000 from investors. Some eye doctors had warned them that removing suppression could create a risk of double vision. With this in mind, Blaha’s system is being tested in eight US eye clinics and undergoing a trial involving 50 adults with moderate amblyopia at UC San Francisco.

“There’s a huge number of adults who have been told there’s nothing they can do, that could in fact have improved vision,” says Blaha. “I’d love to live in a world where kids no longer wear patches and instead play virtual reality video games for 30 minutes a day. That’s the world we want to make.”

“Except for pirates,” says Gupta, looking sideways at his business partner with a grin. “They should still get to wear patches.”

§

Perhaps, then, what my lazy eye needs is a dose of virtual reality. I go to meet Blaha at his new office in the bustling tech hub of San Francisco’s South of Market (SoMa) district, all exposed steel girders, primary colours and glass.

Blaha and Gupta, the Chief Technology Officer of their company Vivid Vision, are both 27. They sit behind their laptops on one side of a long wooden desk, while I strap on a virtual reality headset. After a series of tests to calibrate the system for my vision, I’m dropped into an all-encompassing new reality in which I must guide a spaceship through an asteroid field. My success in navigating my way through a series of rings presented only to my lazy eye while evading the giant rocks heading towards me means my brain is making use of both eyes. After some 15 to 20 minutes of gameplay, I feel a little unsteady and have a mild headache.

Two hours later, I’m perched on a window seat in a coffee shop pondering an uncomfortably heightened awareness of the blurry right-hand part of my vision, which I usually manage to ignore. It’s an unsettling and somewhat alarming sensation, especially as a few weeks earlier an eye specialist at Moorfields Eye Hospital, London, warned me against trying anything that claims to remove the suppression of my weak eye by my good eye. As I sip my latte, I’m wondering whether what I’m experiencing is a precursor to improved eyesight or a warning of double vision.

§

As the investment that Blaha’s venture has attracted shows, the commercial potential of these technologies has not gone unnoticed. Ubisoft, the multinational video game company responsible for major hits like Assassin’s Creed, announced in March that it had completed development of Dig Rush, a game designed to treat lazy eye that can be played on an Android or iOS tablet. Players wear 3D shutter glasses that present different images to each eye, and they can make progress in the game only if they use both eyes.

Dig Rush was made in conjunction with Amblyotech, a health games startup which acquired the rights to Hess’s technology. Following his surprise finding of a way to reduce suppression in 2006, his group were awarded a US patent for their approach in 2011. In a 2013 pilot involving 18 adults, Hess went on to find that those who played the classic puzzle game Tetris, wearing head-mounted goggles that presented different parts of the game to each eye, improved their weak eyes by about two lines on an optician’s test chart. They also experienced reduced suppression and improved stereo vision. Two clinical trials involving children and adults started last year.

The I-BiT group at Nottingham also showed promising early results. They introduced 3D shutter glasses, developed a new space-based shooter game called Nux and obtained a European patent in 2012. A trial involving 75 children aged four to eight produced “indifferent” results, says group leader Alexander Foss – but, undeterred, they are redesigning their system and hope to start two more trials early next year.

Another UK group is taking a different tack. Researchers at Moorfields Eye Hospital and the UCL Institute of Ophthalmology are using 3D shutter glasses to balance the visual signals the brain receives by blurring images presented to the stronger eye. Seven children with anisometropic lazy eye who did daily one-hour sessions for 60 days improved their clarity of vision by just over two-and-a-half lines on the optician’s chart. A clinical trial involving 100 children is planned.

The results of the small pilot studies carried out so far are promising, and enough research has been published to suggest these teams are on to something. However, some key questions remain unanswered. Crucially, there is no consensus on specifically how playing video games improves visual function.

Hess believes that his 2006 discovery shows that the key is getting rid of the suppression of the weak eye. Others believe it’s more complex than that. In his most recent video game study, Levi found that improvements in participants’ visual clarity were not correlated with either their level of suppression at the start of the study or their reduction in suppression by the end.

Steve Dakin, who developed the Moorfields–UCL system and is now at the University of Auckland, New Zealand, also says his group have seen improvements in visual clarity without reductions in suppression. Because video game study participants often have spectacle correction beforehand, he says, this may be the real cause of some of the reported improvements if their vision hasn’t fully stabilised by the time of the research.

And let’s face it – therapies based on video games are considerably more appealing than wearing a patch. “I’m not sure there’s all that much to choose between video games, watching films, and maybe even patching,” says Dakin. “It may be these new therapies work so well because children accept them.”

Which patients will benefit the most from these new therapies? The promising headline results of the early trials are averages that often hide wide variations in effects between individuals. In some cases, researchers have excluded people with more severe lazy eye. While no cases of double vision have so far been reported as a result of these experimental therapies, the misgivings that some eye specialists have expressed mean careful monitoring is required.

The ongoing larger trials should soon provide clues not only about how these therapies work but also more generally about why the brain is more open to being re-moulded by sensory experiences than previously thought, with potentially exciting implications for the treatment of other lost functions. “I think the fact that there is a residual amount of brain plasticity in the adult is going to mean we are going to be able to do something to help in a lot of acquired conditions that affect the brain, such as stroke and traumatic brain injury,” says Hess.

“Amblyopia has become something of a poster child for brain plasticity research,” says Dakin. "Understanding how this re-mapping works will give us insights into how basic visual processing works. “Of course, once you know where brain plasticity is happening, then you know where to target to induce it.”

§

When I set out on this story, I was intrigued. Could I really get 3D vision from playing video games? In short, no. It seems I have the wrong type of lazy eye. Tests at Moorfields revealed my “good” left eye to be mildly short-sighted (–2.25 diopters) and my lazy right eye to be moderately long-sighted (+5 diopters), making for a significant difference in the images going to my brain. On top of this I had a slight astigmatism, or unevenly curved right cornea, and a micro-squint, and I’d never had spectacle correction for my weaker eye. Put simply, I was a poor candidate for treatment. I reluctantly declined the ten game sessions that Dennis Levi offered me in his lab.

While I was initially unsettled to learn that most people see the world differently to me, it’s something I now view with interest more than worry. I’m pretty confident that I don’t have serious hand–eye or foot–eye coordination problems. It is true that my football shooting skills are pretty poor, but I do better in defence or midfield. I spent what many would regard as far too long juggling as a student, and as a result can juggle five balls (in basic cascade, half-shower and full-shower patterns, in case you were wondering).

In fact, the slight difference between the visual signals generated by each eye is just part of what gives us depth perception. Other cues include size comparisons, shading, assumptions about light sources and the way objects change over time. My brain may not be able to give me the 3D vision that others get, but it does compensate by using these sources of information more.

The heightened awareness of the blurry part of my vision I experienced after briefly using Blaha’s virtual reality system was thankfully gone the next day. The experience of being able to play the game gives me hope that one day I may be able to make progress with his technology or one like it, but I think I’ll wait to see the outcome of a few more trials first.

Although the brief attempts to mend my lazy eye have so far failed, the signs are that video game therapies may prove effective for others in the near future, and could even replace patching. If this happens, generations of young patients will never know how grateful they should be.

The I-BiT group received a Wellcome Trust Translation Award in 2010.

Nic Fleming is a freelance writer and editor, working for outlets including New Scientist, the Guardian, the Economist, Nature, BBC Future and Mosaic. This story first appeared on Mosaic and is republished here under a Creative Commons license.