August 23, 2016
Virtual Reality Los Angeles/via Facebook
A visitor to the Virtual Reality Los Angeles Summer Expo dons a virtual reality headset.
Technology has come a long way since the days of Nintendo's Virtual Boy, the video game console that first introduced a virtual reality experience to households in the mid-'90s. Users complained about Virtual Boy's monochromatic graphics, a feeling of discomfort while playing, and the steep price. Games felt more two-dimensional than immersive, and the system's sales badly flopped.
Two decades after Nintendo's failure, virtual reality has suddenly got its groove back. In 2014, Facebook bought VR startup Oculus for $2 billion, and Google introduced its low-cost Cardboard VR platform. Other companies started to develop their own VR systems, and this year marks the first time that high-end headsets from HTC, Oculus, and Sony are available for purchase by the masses.
The release of VR video games, short films, and live experiences promises to revolutionize the entertainment industry — but whether the technology will live up to the hype remains to be seen. All the headlines touting virtual reality as the next big thing left me curious to learn more about the science: What kinds of tricks do VR developers use to try to fool the brain? What happens when they don't work? And why do some people experience motion sickness when using VR?
I attended the recent Virtual Reality Los Angeles (VRLA) Expo to report on the latest gadgets and apps — while holding onto some skepticism about whether these new headsets would really be immersive enough. In a demo by virtual reality art company Shape Space VR, I found myself navigating a wacky Dr. Seuss-like maze, sailing serenely through towering abstract blobs painted in shimmering colors. The experience felt like floating through an infinite rendition of Isaiah Zagar's Magic Gardens.
I realized in this moment that virtual reality had finally come into its own. All the pieces — the display headset, hand controller for navigation, and well-designed content — worked together to fully transport me to this strange virtual world.
“We're seeing a dramatic improvement in the technology, and part of making VR more realistic and creating that magic actually involves convincing the brain that what it sees is real,” said Hasan Ayaz, an associate research professor of biomedical engineering at Drexel University. “The brain looks at many different cues, and it requires some very detailed artwork to create this virtual reality environment.”
In the case of the trippy 3D maze, the artwork was made by Oscar-winning visual effects supervisor Kevin Mack, the founder of Shape Space VR. Mack, who has worked on such films as "What Dreams May Come," "Big Fish" and "Fight Club," obviously has a knack for creating detailed, fantastical worlds.
'Zen Parade' is virtual reality art by Kevin Mack, the Academy Award-winning visual effects pioneer.
But a good headset remains just as important as quality content in order to create an effective illusion. In the real world, light from an object enters the eye, hitting the retina where it gets detected by photoreceptors called rods and cones. The retina transforms the image into electrical impulses that are sent to the brain. Virtual reality headsets instead place tiny screens in front of each eye to create the same impression on the retina as a real object would, and the results are theoretically indistinguishable.
“That's the trick of VR, it's not something you have to do consciously — you can't not do it,” said Oliver Kreylos, an associate researcher at the University of California, Davis, who spoke at VRLA about how virtual reality works. “There's really no difference between looking at a real object and looking at a virtual representation of the same object. It has exactly the same imprint on the human retina.”
How to solve this dilemma? Put another, physical lens between the screen and the eye so that the glass can converge the light rays part of the way, while the eye's built-in lens does the rest. Hence, all VR headsets from the Google Cardboard to the Oculus Rift come with lenses installed.
“The brain doesn't know which input to process, so it creates a conflict ... the feeling of sickness is the error that the brain creates, in a sense. As we create more consistent information in virtual reality, that problem will go away.” – Hasan Ayaz, associate research professor, Drexel University
Yet, all the tricks VR developers use to fool the brain don't always work because of inherent imperfections found in every system. For the illusion of a virtual world to hold, headsets should track not only users' head motion but also where their eyes are actually looking. Also, the displayed images need to update quickly and without noticeable lag when users move about.
Virtual reality sickness, which feels just like motion sickness, can result from unstable imagery or other glitches. While the actual mechanism that causes this type of discomfort is not fully understood, developers know it often arises with a conflict between incoming visual stimuli and the information going to the vestibular system which senses balance. For example, a user might feel nauseous when flying a plane or driving a race car in VR while sitting still.
“The brain doesn't know which input to process, so it creates a conflict,” said Ayaz. “You can think of it as the blue screen in Windows when it crashes — the feeling of sickness is the error that the brain creates, in a sense. As we create more consistent information in virtual reality, that problem will go away.”
To that effect, Ayaz and his colleagues use optical and electroencephalogram (EEG) techniques to monitor the brain while using virtual or augmented reality with the hopes of improving the technology's immersion factor. In a recently published study, they tested subjects as they navigated a route through campus using either a smartphone or a head-mounted augmented reality setup. The researchers found that mental workload was reduced for those who used the headset, and this group also had better performance on a working memory task presented while on the route.
“It's not only about developing a better display and all that, but it's about understanding how the brain and visual perception works to create consistent information for the optimal experience,” Ayaz said. “I believe that now developers of VR technology are understanding this, and we'll see more biomedical research to improve the user experience overall.”
