Not so long ago, head-mounted virtual reality (VR) displays were beyond affordability. Now capable systems like the Oculus Rift or the HTC Vive cost under a grand. That’s good news for military trainers looking to cut costs by creating tightly engineered virtual environments that replace live role players, full-scale shoot houses and bring training closer to troops.
“We’re in a pretty tough budget environment,” says Lt. Gen. (Ret) Thomas Baptiste, president and CEO of the Orlando-based non-profit VR trade consortium National Center for Simulation. “In an austerity environment, how do you train [soldiers returning home]? How do you train them at home station?”
The answer lies in consumer gaming technologies, which offer the scale to bring down costs and deliver local training to what the military calls “the point of need,” rather than limited to regional or national training centers.
Virtual reality can safely replicate otherwise expensive and dangerous training scenarios, letting trainers repeat and modify as needed. For complex maintenance tasks, machines can perpetually be taken apart and rebuilt in VR without fear of wearing down real parts. And because trainees can train anyplace they can hook up to a computer – as opposed to jetting off to a life-size simulator – time and travel can be saved.
VR technology enables the Army’s Close Combat Tactical Trainer (CCTT), originally created to provide virtual training support for mechanized infantry, expand into other areas. The Dismounted Soldier Training System, an outgrowth of CCTT, lets individual soldiers train in the same virtual environment used by crews of M1 Abrams tanks and M2 Bradley Fighting Vehicles.
The Army Research Lab Orlando’s Simulation and Training Technology Center (ARL_STTC) researches not only augmented reality for dismounted soldiers, but mixed reality in which virtual elements such as people, targets or ballistics can be mapped into a live field of view. “Can we put a live soldier out in the woods and provide them virtual opposing forces with realistic effects ad accurate ballistic solutions?” asks science and research training specialist Col. Harry Buhl, ARL_STTC deputy director. Virtual elements have to not only look realistic, but behave as if tangible, moving behind objects rather than through them.
The real prize Buhl says is not just to design a mixed reality simulation with one soldier and a few augmented virtual enemies in a live environment, but to “get the larger capability, where we’re dealing with formations of tanks and Bradleys and multiple soldiers. That’s a little farther down the road.”
As a training technology VR is helpful, but still a simulation.
“You learn what’s supposed to happen, but you know it’s not real,” says Dr. Mark Nesselrode, Capt. USN, (Ret.), now a solution architect for training with General Dynamics Information Technology. “It’s not bad training.” But the ability to blend VR and reality could take training to a new level. “If you’re in a virtual environment and it looks like your ship, then when you train in augmented reality, and it is aboard your ship – that’s huge.”
Making all this possible are rapid developments in computer technology, beginning with high-speed, low-power processor chips and continuing with increased network capability and better bandwidth compression algorithms. Most important: The inexpensive, high-resolution LCD displays developed for smart phones and commercial game engines developed to drive consumer video games can do double duty as military training tools.
Commercially-licensed game engines like Unity from Unity Technologies and Epic Games’ Unreal Engine 4 have enabled GDIT to rapidly create a series of shipboard engine-room trainers for the Navy that provide point-of-need scenario training that in the past, would have required expensive life-sized simulators. Those old systems relied on custom software and substantial investments and timelines. The new technology changes the rules and leave the simulation technology development to the experts.
“We don’t see a major need for the Army to be building game engines,” Buhl says.
One promise of the immersive nature of virtual reality training is what experts call presence: the notion the VR experience is so profoundly real, a user feels fully present in its environment.
“You sort of got something if you can suspend disbelief enough to make their heart beat faster, make them sweat,” says Baptiste. “It can’t just be a carnival ride or a Disney experience.”
Michael Abrash (then with game maker Valve), now chief scientist at Oculus, described in a 2014 paper the technical elements needed for creating presence:
- Displays with at least 80 degrees field of view to provide peripheral visual cues for better context and orientation
- Resolution of 1080p (HD) or better for improved clarity and realism
- Low pixel persistence to prevent blurring
- Refresh rates better than 50Hz to both eliminate motion artifacts and improve motion response time to reduce simulation sickness
Though today’s VR systems match those metrics, simulator sickness continues to be a challenge. Improved screen resolution provides better realism and clarity. According to Stephen Hodgson, virtual solutions product developer for training and simulation at Saab Defense and Security USA, there’s been great progress with head-related transfer function (HRTF), which mimics the way humans perceive and locate sounds – letting systems tune sound profiles to individual users. New graphics cards also help. The NVIDIA 1080 graphics cards, released in May, are optimized for virtual reality headsets.
Commercial VR headsets, already not heavy, will get lighter with time. Bandwidth is increasingly capable of supporting multi-user scenarios and wireless connectivity will eventually allow VR users to operate untethered, increase the usefulness and safety of simulations.
But limitations remain. Though avatar generation has improved, there’s a gulf between close and convincing. The so-called “uncanny valley” effect results in users rejecting a computer-generated human that looks somewhat, but not convincingly lifelike.
Resolution and scale create other problems. The Army can take advantage of dynamic terrain programs that alter the performance of a vehicle moving through dirt after it turns to mud in the rain. But while commercial gaming engines might incorporate one square kilometer of terrain, the distances of 90 square kilometers demanded by Army tank training requirements, is too much real estate to render in the highest graphical detail and still display in real time.
Feedback of All Sorts
Haptics – the ability to deliver physical touch feedback to users – presents another hurdle. Haptics can also allow users to see their hands in virtual environments, adding an element of realism. Despite progress, however, experts say tactile feedback is still years behind visual or aural response.
In some cases this means that an old-fashioned mouse and computer screen may provide superior training to VR. “You have very, very detailed maintenance capabilities using a mouse, but when you put the helmet on, and you get into the environment, the fidelity and accuracy goes down significantly,” says Pete Swan, Business Development with VT Mäk, a Cambridge, Massachusetts-based modeling & simulation software firm.
Buhl says bringing haptics to large-scale, collective training environments the Army is building will require significantly more work – enough that it’s not yet clear if the payoff will be equal to the effort.
“Is this a promising path forward or is this an empty well?” Buhl asks. “We don’t know the answer to that at this time.”
Despite technological improvements, Hodgson is skeptical about VR’s ability to pull people out of their own sense of reality and into a virtual space. He notes that the forces of both electromagnetism and gravity subtly affect our real world sense of who and what is close to us. “You spend your entire life building these ideas of what reality is,” he says. “It’s all these little minute things you don’t even think about. As adults, putting yourself completely in this immersive new experience can be very disorienting.”
On the other hand, Hodgson and others agree that younger people are more comfortable with and excited by the interactive, multimedia game-like environments VR can deliver. Their sense of reality already includes a strong digital presence, and they are more comfortable training with this sort of technology.
“The average sailor is 20 years old,” says Nesselrode. “They were all born in 1995 and this is all they understand. This is how they learn. They’re very collaborative. If you want to attract the workforce, if you want to retain a workforce, you have to train them in a form that is not only native to that generation but extremely familiar.”
Real-world training experience is changing expectations and requirements. Customers realize users can and will use their imaginations to fill in “gaps” in the virtual experience.
“In the past, companies would say it has to be 100 percent perfect,” says Swan. Not anymore. VR designers now realize “you can do a lot of training with a lot less realistic graphics.”
That’s because the most important variable in VR is not the head mounted display, the haptic gloves or other gear, but in how best to design for users, says Eileen Smith, Director of E2i Creative Studio at the University of Central Florida’s Institute for Simulation and Training.
“Once we’re working in a computer, nothing happens unless it’s programmed to have the flexibility to let the human play with it,” Smith says. Even VR is ultimately about real people first and foremost, she explains. “That’s what it will always come down to.”