In business, mixed and augmented reality makes a lot more sense than virtual reality.
So far, the people who are really using mixed reality headsets like Microsoft's Hololens are in industry, which is why the integrated hard hat was such a keenly-awaited accessory.
Aviad Almagor is director of the mixed-reality program at land survey and construction company Trimble, which has been working with Microsoft on Hololens.
Almagor worked with an architect in California who was presenting the redesign of the abandoned Packard plant in Detroit at the Venice Biennale design festival in Italy.
"It's a project that's half a mile long, it's got pretty complex geography; it's in several segments, all of which are complex spaces," said Almagor. "With mixed reality, you can move from the tabletop scale view, which helps you understand the relationships between the different segments in the project, to full immersive view at a one to one scale, so you can physically walk through the design and see how a visitor will experience the design. Moving between the two gave them a great perspective of how the design was progressing."
Using HoloLens let architect Greg Lynn try out ideas and immediately see what impact they would have throughout the giant structure, rather than having to wait months to make those decisions.
Being able to experience a space remotely is also useful for working in harsh environments where you don't want to spend a lot of time on site, like open pit mines.
Trimble is working with partners like Caterpillar to put sensors and scanners onto mining equipment to get a detailed 3D view of the mine structure, and the locations of the machinery for monitoring and planning.
"No-one wants to be in the mine. It's far better if you can monitor it from a remote location, that improves safety," said Almagor.
"If you take the 3D geometry and the sensor data and add mixed reality, you get a solution that allows the mining engineer to sit in their office, which might be thousands of miles away from the actual mine and visualise the actual mine almost in real time.
"You can visualise the geometry of the mine at tabletop scale or you can jump to the immersive view and actually be inside the mine. You can see changes in the slopes of the walls as well as the progress of mining activity. You can visualise the machinery almost in real time. You can visualise the blast holes, or see where materials are taken from and where they're stored. The engineering data you require to effectively manage the mine is there in mixed reality."
What Trimble calls a 'materials rainbow' arches from where the material is mined to where it's stored. The mining engineer can also replay the movements of mining equipment to check if operators are following procedures properly, or if there's a more efficient way to do things.
The mining system has similar benefits to those architects are seeing from HoloLens, he says: "It's about visualisation, communications and collaboration -- remotely and co-located."
Almagor also pointed out that building those kind of experiences is getting easier. When Trimble started bringing HoloLens and Sketchup to design and engineering firms, they needed to run multiple pilot programs and offer support services to help them fit it into their workflow.
They will still offer those if required, "but now we have customers purchasing them as a commercial solution," says Almagor. "They buy the software and the HoloLens and they don't need us."
Authoring is getting easier thanks to tools like PTC's ThingWorx Studio MR and the new Simplygon rendering service on Azure. ThingWorx Studio is a web app that lets you build 3D scenes (for HoloLens or mobile AR) by dragging and dropping existing CAD models, connecting them to sensor data and adding gesture and voice controls for interacting with the models. Simplygon can significantly reduce the file size of 3D assets and still keep the fidelity for HoloLens.
Personal and collaborative
KPMG's data and analytics team is using a HoloLens app for data visualisation in the Insights Center in Frankfurt where it presents to groups of corporate executives from its Fortune 500 clients.
The 3D visualisations can show more dimensions of data and make it easier to understand, and the whole group can see them at once. The HoloLens system will come to KPMG Insight Centers in Hong Kong, London, Paris, and New York later this year.
Being able to collaborate is important in consumer scenarios as well, and it's something HoloLens enables rather better than a VR headset, which is more isolating.
"Yes, I can design my kitchen in VR, but I can't interact with the sales associate while I'm doing it. They need the customer to be actively working with them to make design decisions," says Forrester analyst J P Gownder.
That's easier to do when you can see who you're talking to, and not just in a store.
"Anywhere where you have interactivity between multiple workers who are together in a real space, doing 3D visualisation is possible in VR but it's more desirable in mixed reality. There is still something about the social aspect of working together that mixed reality is going to be better because you want to have that social interaction."
The issue with that kind of 3D design system is getting the 3D assets for all the parts and components you want to show; Gownder mentions an online furniture retailer that carries many thousands of different products but only has high quality 3D designs for some 15,000 of them.
Architects and engineers who are already designing projects in 3D in their CAD systems have an advantage here because they can reuse those assets for mixed and virtual reality advertising. Some high-end condo developers selling units off plan are already using 3D walkthroughs.
Other interesting HoloLens apps are coming. CLAIRITY is testing its augmented reality SmartBinocular air traffic control system at Budapest International Airport, and a HoloLens version called HoloTower is in development.
UK police forces are currently looking at Black Marble's TuServ command and control centre, which lets the officers co-ordinating a major incident view information on multiple, virtual screens, according to the company. Officers in these forces already use the mobile TuServ app to capture incident reports and evidence on phones and tablets and that feeds directly into the command and control interface, alongside maps and other relevant information. "HoloLens is expensive, but a command and control room that you only use a few times a year is more experience," points out Rob Hancock, Black Marble's development director.
Where HoloLens truly shines is in situations where you have no other way of seeing something 3D. Professor Mark Griswold of Case Western Reserve University, which is teaching medical students anatomy with HoloLens, says that despite having spent decades studying MRIs of the human brain "I never truly understood their 3D structure until I saw them in HoloLens."
And one geologist at NASA told Gownder that "he learned more in a week with HoloLens than he had in six months with 2D imagery".