What are the types of virtual realities we could create, and how should we go about doing it? ©HOTLITTLEPOTATO
It’s Christmas 2016—prime gift-giving season. As many in the retail industry predicted, millions of people are unwrapping a Vive, Rift, PlayStation VR, or some other virtual reality device. I imagine many people, when faced with this unusual contraption, responded as my grandfather did when I gave him his first experience of virtual reality in 2014: “What am I supposed to do with this?”
Then they did what we all do nowadays when we have a question—they asked Google. According to Google Trends, the use of the search term “VR Content” tripled between December 23 and December 26, 2016. Not coincidentally, so did queries for “VR Porn.” It seems people received their VR goggles, tried out a few stock demos, and then sought out, well, porn.
In the late ’90s, before I founded Stanford’s Virtual Human Interaction Lab, my initial research wasn’t geared toward VR as a consumer product. I worked in a psychology department at UCSB, and we viewed VR as a tool to understand basic brain science, not a gadget that sits next to the TV set. In fact, in my lab at UCSB we likened the VR system to a functional Magnetic Resonance Imaging machine—a ridiculously expensive, bulky technology that needed constant maintenance and could only be administered by trained experts.
Excerpted from Experience On Demand: What Virtual Reality Is, How It Works, and What It Can Doby Jeremy Bailenson. © 2018 by Jeremy Bailenson. Used with permission of the publisher, W. W. Norton & Company, Inc. All rights reserved. ©W. W. NORTON & COMPANY
When I arrived at Stanford, I switched from a psychology department to a communication department studying media use. As my thinking evolved I began to imagine a world in which avatars and virtual reality were everywhere. I imagined a future as William Gibson or Neal Stephenson described it—with VR everywhere. And that raised a host of fascinating, sometimes troubling questions. Could politicians use avatars to rig elections? Could VR make advertisements more persuasive? Could changes in the weight of your avatar change the way you eat in the real world? But even as I explored these knotty issues, I was able to sleep at night just fine. After all, the technology was still confined to those with six-figure budgets and the engineers necessary to keep the systems running.
But in 2010 my thinking began to change. Maybe it was that I was starting a family (my first child was born in 2011) or that I was witnessing the first wave of consumer VR technology flourish when Microsoft created the Kinect. Perhaps I was influenced by new mentors, people like Jaron Lanier—whose vision of VR was synonymous with a hippie-inspired notion of self-transformation—and Philip Rosedale, whose unbridled enthusiasm for a prosocial world of networked avatars was infectious. Or maybe I had just finally drunk the Silicon Valley Kool-Aid. Whatever it was, it now seemed to me that VR might just transform the world for the better—if only we could creatively harness its potential.
What, then, are the types of virtual realities we could create, and how should we go about doing it? People fly to my lab from all over the world to hear answers to those questions. One of the questions I get asked most often by companies trying to get into VR is, “What should we do?” My answer, of course, depends on the context. I have had hundreds of these conversations since 2014, when VR began to go mainstream. Here are three loose guidelines that have emerged and evolved from those conversations:
1. Ask yourself, does this need to be in VR?
I’ve come up with a few rules of thumb for how I think about appropriate VR use cases. First, VR is perfect for things youcouldn’t do in the real world, but not for things you wouldn’tdo in the real world. Flying to the moon like Superman is OK. Participating in virtual mass murder—especially if it is designed to be realistic—is not. This is a big deal. Everything we know about training in VR points to the realization that the medium is incredibly influential on future attitudes and behavior.
Second, don’t waste the medium on the mundane. VR experiences should be engaged in mindfully. Since we are worried about distraction and addiction, we should save VR for truly special applications.
The easiest way to tell whether VR is the right choice for a given experience is to ask: Is it impossible in the real world? If the answer is yes, then VR is a very safe bet. Time travel is not an option outside of Hollywood, so if you want to go back and meet your great-great-great-grandfather or feel what it is like to walk around as a cow or grow a third arm to be more productive in your daily tasks, then you should go with VR.
Using VR to safely experience dangerous behaviors is another good use. A primitive version of VR technology was developed to simulate flight in the late 1920s. Now it’s time to extend this model of military training to firefighters, nurses, and police officers. Crowd control is a routine challenge for law enforcement, but it is impossible to build a real-world simulation to train officers to handle unruly mobs. Imagine how much more prepared officers would feel if they had dozens of virtual practice sessions under their belts. I get phone calls and emails from law enforcement officers fairly regularly. They see this technology as transformative.
When I was growing up, you would hear stories about kids caught smoking cigarettes being made to smoke an entire pack to teach them a lesson—tough love, baby boomer style. This might be an effective way to handle a teachable moment, but it will certainly do damage to a kid’s lungs. In VR you can have the best of both worlds. No, you can’t simulate the pain of inhaling large amounts of toxic smoke, but you can show the longterm effects of smoking on an avatar or take a child on a guided tour of damaged lungs. Or: Illustrate the costs of damaging the environment without chopping down a single tree. Then go on a virtual tour of the devastation. It feels real, and the brain treats it like an experience, but no environmental damage occurs.
VR can also be the right choice when access or cost—in time or money—is an issue. Traveling to the top of Mount Kilimanjaro doesn’t have to be physically or financially impossible or dangerous. Not only can VR afford stunning mountaintop views at a fraction of the expense and exertion, it will also save precious time. I once spent 40 hours—yes, a full work week—traveling back and forth to South Africa to give a 45-minute talk. If I could have used an avatar to give that talk, it would have been like having a 53-week-long year.
And speaking of savings, consider VR’s potential to cut the costs of medical training for surgeons. Cadavers are expensive, and each organ can only get cut once. With VR, there may be cost on the front end to build the simulation, but once it is done, the simulation is like anything digital—you can make a billion copies, send it around the world with a touch of a button—and it lasts forever without degrading.
In short, if a real-world experience is not impossible, dangerous, expensive, or counterproductive, then you don’t want to create it in VR. You might even consider having the experience in the real world.
2. Don’t make people sick
If you decide to create something in VR, not getting people sick should be your primary concern. Good VR feels great. It is fun, engaging, exciting, potentially transformational. But a problem known as simulator sickness—the nasty physical reactions that occur when virtual and real perceptions don’t align—is an absolute deal-breaker. I worry that a few very public instances of this malady could sabotoge the entire VR movement.
When I first started working in the field, we had an incident at UC Santa Barbara. A woman in her 40s participated in an experiment and got a small case of simulator sickness. Back then, bouts of sickness were fairly common: We were only running at an update rate of 30 frames per second (compared to 90 now) and the latency—the lag between head and body movements in the VR world compared to the real world—was very high, producing a disconcerting gap that can confound the senses. After a short time the woman said she felt fine. We said goodbye to her, and she left the lab.
She drove home and parked. But while walking from her car to her doorstep, she got dizzy, fell into a fencepost, and hit her head. My colleagues and I were horrified at the thought that we might have contributed to this woman’s fall. As it turned out, she suffered no serious injury, and there were no legal consequences. But it was a reminder: Avoid simulator sickness at all costs.
Designers should also be careful to build systems where users control when the field of view moves. Moving it automatically can have stomach-turning consequences. At a recent trade show, I watched one of the largest automobile companies give simulator sickness to one CEO after another. They put the executives inside simulated cars and drove them around sharp turns, accelerating and decelerating and putting their vestibular systems through a crash course in disorientation.
Why was VR driving such a strain on the senses? Short answer: Poorly designed virtual reality messes with the systems humans have evolved for physically navigating through space. For hundreds of thousands of years, when a human has moved, three things have happened. First, optic flow changes. This is a fancy way to say that if you walk closer to a rock, the rock gets bigger in your field of view. Second, your vestibular system reacts. For example, sensitive structures in the inner ear vibrate when you move, sending signals to the brain that you are in motion. Third, you get proprioceptive cues from skin and muscles, information that tells your brain where your body is in space.
Driving in VR wreaks havoc on these systems. The drivers at the trade show were seeing the road zip by with the proper optic flow, but they didn’t get the correct vestibular cues because their bodies weren’t actually moving with the car as it whipped around turns. They also didn’t get the proprioceptive cues that would cause them to feel pressure from the seat back on their back muscles and skin as the car suddenly accelerated.
3. Be Safe
Good VR causes people to forget they are still in the physical world, and that can be dangerous. In the process of running VR demos, I have had a 70-year-old man randomly attempt to do a backflip and fall into my arms. Famous journalists have sprinted full speed at physical walls. A Russian businessman almost hit me in the head with a roundhouse kick, and a famous football coach smashed his hands against a podium to swat at a sprinting virtual player.
Ultimately our lab is safe because we always have a vigilant spotter on hand, a person whose job is to watch every move of the user and catch or restrain him if necessary. Of course, the “spotter” solution will not work at scale. I like to joke that most commercial packages don’t come with me. (I am very proud of my spotting skills.) Instead, they come with guidelines, for example, “Please sit down while playing this game,” or scanning systems that often (though not always) warn you about walls. It will only take a few horrifying—and highly publicized—accidents to derail the VR revolution. So here’s my advice to anyone trying VR at home: Whatever energy you plan to put into safety, triple it.
One way to bolster safety efforts is to keep VR simulations short. Think about some of your most memorable life experiences. Did they last for hours or mere minutes? In VR, the dictum “less is more” is especially true. Given that most VR simulations are intense—emotionally engaging, perceptually harrowing, and psychologically compelling—five to 10 minutes is often enough.
NOBODY KNOWS WHAT the future holds for VR. The best we can do is try to understand how it works and what it’s capable of, and then imagine how these factors can meet our needs and desires. Who would have thought that one of the most popular uses of the powerful phones we carry around with us would involve sending text messages and tweets—the type of information that a 19th-century telegraph could handle? Who would have thought that the most elaborate game interface ever invented, the Microsoft Kinect, would not supplant a traditional Xbox gaming controller?
If the internet is any guide for how VR will evolve, most people will not just become VR consumers but VR producers as well, the same way people blog, upload YouTube videos, and tweet. As the technology improves, the range of ways people can express themselves in VR will be bounded only by their imagination. Some of it will be unsavory. While I wholeheartedly agree with the US Supreme Court’s determination that digital simulations are protected free speech, I also believe that just because we have the freedom to build anything we want, doesn’t mean that we should.
We should strive for more than pure sensationalism or escapism in our entertainment. We can make the world a better place—if you’ll forgive me the Silicon Valley cliché—if we respect the unique power of the medium and focus on the prosocial aspects of VR. However it shakes out, it’s a very special time to be a part of this technological revolution. The coming years are going to be a wild ride.