Virtual Reality: Moving From 3D To Full Immersion

Category: 
Virtual Reality: Moving From 3D To Full Immersion
February 19, 2019

Rolling Out the Carpet

It has only been half a decade since 3DTV and cinema gold rush that carpeted the floors of the world’s largest technology trade shows like NAB and CES. This was in the days before the VR gold rush: before the drone craze, the cloud hype, the AI boom, the robots in everything, pre-volumetric and lightfield capture, Augmented Reality filters on Instagram and Snapchat, HDR, Alexa in everything, self-driving cars, and government-subsidized TiltBrush spatialized art. Many of these existed long before 3DTV was the biggest buzzword at the trade shows, but I’m talking about commercialized form.

The only commercials I have seen for VR thus far involve an estranged significant other, mouth agape, staring in wonder at something, neglecting their partner, child or other responsibilities. Where is the America Online use case to the public for VR/AR? Too soon? Exactly.

 

The Cart before the Horse

This moment of 3D hope has gone down as a key lesson in the history of media deployment; the rush to get everyone to trade up before there was content – and that spectacularly crashed and burned – is becoming a similar problem with VR and AR deployment. But I want to qualify this: the work that has been done in VR and AR so far is wonderfully creative and artistic. Of course, at first it was demo-y and basic, but it is nice to see how many wonderfully inventive people rushed into the fray and tried many important tests.

 

It isn’t just exclusive to VR and AR that wonderfully creative things are being done. It’s just that those happen to be the zeitgeist. The new 3D wave of the early Aughts, in fact, led to tremendous progress and innovation in the field of digital imaging, visualization, projection and yes, distribution. We were achieving things that we couldn’t have dreamed of a decade prior. Better color and contrast, extraordinarily complex depth and convergence affording us phenomenological images that made the idea of a shark jumping out of the screen laughably basic.

 

Much of the richness of the 3D experience had to do with advances directors like James Cameron made in his approach to Avatar: moving the depth inward vs. outward, drawing us into the worlds vs. forcing us to duck under swinging swords that extruded from the screen. The real-time CG capture/live-action hybrid techniques he was using (and later were used by Spielberg on Ready Player One) were already a form of VR cinema capture and created a perfect on-board ramp for the industry to make one powerful use case of many for immersive technologies.

 

Audiences complained, however. Though they often paid the extra for the 3D upgrade at the theaters, they also often resented it. In the home, the notion that their passive, programmed TV experience required an individual and expensive pair of glasses for anyone in the home, or any guests who may want to watch along, was simply unrealistic – absurd even.

What precisely is on offer here?

 

Bringing It Home

The problem is bridging the gap between creation, discovery, innovation and public adoption. To date, there have been no super wide-appeal apps that have the utility of a news program or the lay-back commonality of Jeopardy! or Wheel of Fortune – the sorts of experiences that singularly justify the purchase of the hardware and are endemic to that hardware…hardware which at 1.0 release should be something you can take off the shelf at WalMart, bring home and plug right in so you can hop into your singularly justifiable purchase.

 

The VR industry pivoted in 2017-2018 to talk about how “Out-Of-Home (OOH)” and “Location-Based Entertainment (LBE)” were the real killer app for VR. This is an important industry, seeing the returns of arcades in parallel to the rise of professional videogaming, but calling it the true market for VR is a cop-out.

 

Where is my Atari 2600 or NES? Where is my Alexa or Phillips Hue?

 

I want my MTV.

image via “liveforlivemusic.com”

 

We have to make certain decisions about what we are attempting to achieve here – development? Professional authoring tools? Home appliances? Mainstream entertainment? All at once?

 

In that case, each vector needs to focus on its aims. If it is R&D, then go crazy exploring the possibilities and defying assumptions. If it is industry tools, B2B, enterprise, then don’t worry your little head about the number of sales at Best Buy. But if you ARE concerned with wide appeal and adoption, then there is a whole lot of stuff that needs to be addressed properly, in its own lane and with total pragmatism.

Familiar Faces in Brand New Spaces

The basic matter of on-boarding people in a simple and painless way: even if the installation process were push-button simple, there would be the matter that no two interfaces are truly consistent. After almost five years, we are seeing some interface commonalities: the rotating reticle for teleportation locomotion is become more common (a good thing), as is the ability to capture in-experience footage and post it to social media and the ability to resume your cloud saves cross-platform. Maybe?

 

From the beginning, we’ve said we need to focus on UX above all. VR won’t be ready until this matter is addressed, but is it too soon to demand this?

 

We cannot rely on old ideas. Here we are inside of the media, and diegetic menus may not manifest as 2D interfaces. Here, where we have vertical in addition to the horizontal field of view and depth can be mapped onto real-world markers – and will eventually prove to be mission critical – we need a new interface grammar. Despite hundreds if not thousands of year of experiential design, we are in Terra Incognita. We must speak to one another about the effects and intuitiveness of these designs and allow each other the permission to make some mistakes along the way. We will all be better for it in the end.

 

People Are People, So How Should It Be?

This year, in our top 10 indie games of the year, we commented on how it felt like the most common thing between all the games was how they managed the user experience: entering and exiting games, saving and restoring gameplay, and offering different modes of play for casual and hardcore players.

 

I started covering games in 2008, so it took a decade for the indie game industry to finally iterate to a reasonably consistent user experience. But first it had to create genres, mash-ups, blockbusters (MinecraftThe Binding of IsaacStardew Valley). VR’s first real breakout – Beat Saber – was a mash-up of Beat Boxer and Light Saber Trainer – using all the powers of Star Force allure with the epidemic of Dance Dance Revolutionand how those relate to rapidly growing eSports wave.

 

Even though it is exceedingly simple to understand – anyone can grasp what to do within seconds – it wasn’t the first attempt. Time was needed to test ideas, to see games come and go. Janet H. Murray, who wrote one of the original great treatises on narrative in contemporary digital media, and more specifically spatialized media – “Hamlet on the HolodecK” – has titled her new one “Inventing the Medium.” That is what is happening, except it is happening across dozens of industries at once in an exponential curve.

 

I recently defended a company being accused by the internet mobs of taking too long to deliver on a crowdfunded complex high-resolution product for under a thousand dollars. I reminded that this had never really been done before and we need to encourage, rather than belittle such actors when they hit a few snags along the way. Constructive input will yield far greater dividends for the supporters who paid in, than if the company goes bankrupt and disappears. This stuff is hard. But it won’t go away – there is no way we can put the genie back in the bottle.

 

Though 3DTV didn’t take off commercially like Best Buy, WalMart and all of its suppliers may have hoped, it ignited many conversations and developments that are only now coming to fuller fruition as VR and AR come into play.

 

Teamwork Makes the Dream Work

Our understanding of interocular distance, parallax, stereoscopy and how we perceive scale all became part of the vernacular. As long as the industries are communicating and occasionally conceding to shared standards, then everyone can continue to contribute to the invention, refinement and evolution of new media. We don’t need homogeneity or uniformity, but some collaboration and concession would be beneficial.

 

Essentially, it is important for us to pause every once in a while as a whole and check in with each other, with ourselves, with the public, and with the manufacturers and developers to understand what is missing – what is creating friction – and address it. With all of these VR consortiums and associations, who is really working on the challenges of standards, practices, protections, safeties, ethics and conventions and deploying them in consideration of the front lines (and the back ones)? The IEEE? Not for consumers. The Consumer Technology Association only just published a glossary of terms for the industry in July 2018. The XRA is trying to bridge the conversation between the big VR risk-takers like Samsung, HTC, Oculus, Sony and Google – but where is Microsoft, the largest software company in the world, whose HMDs represent Acer, Asus, HP and more?

 

The book Writing on Both Sides of the Brain described the “editorial mind” as Caliban and the free-associating “creative” side as Ariel from Shakespeare’s The Tempest. They are shadow and light to each other, but cannot – or should not – inhabit the same space. When we are in free-thinking, experimental phases, we must allow it. When we are in the editorial, didactic phase, then it is not the time to move the goalposts, scribble or change the objective. Right now, it is time for the immersive tech industries to catch up and make good on some of the good faith that its existing audiences and early adopters have given it. Time to tune the products, catch up on deliverables, and create some cohesion in prices and interfaces and installation practices.

 

Untapped Potential Emerges

And then go back to exploring what else the medium can do. I think of how cable disrupted network television, Betamax disrupted cable, OTT disrupted cable, etc. But 3D did not disrupt OTT. It was a different path. VR is not a TV disruptor. The idea is to meet people where they are and offer new estuaries that stem thus.

 

We are discovering what the medium is; we do not yet have an answer to the ultimate questions of what it will be for or how it will settle into place. In 2019, we are seeing absolute leaps and bounds in the nature of media, from 6D.ai‘s real-time volumetric capture of any space via an iPhone to complex AR filters on social media proliferating with every new update to Google’s ARCore and Apple’s ARKit, to full-body intelligent haptic and biometric-reading mesh suits like Teslasuit begging for content.

 

If we think back on Twitter’s early days – or Facebook’s, for that matter – we never could have dreamed it could sway elections or be used as the primary method of communication for the President of the United States of America. Can you imagine what the immersive media will deliver unto us?

 

We have to give the time for the media to discover themselves. But we also have to help them connect to the end users and give it a chance to get there. The best way to do this is to give it a viable user experience so we can understand where we are and how to help it serve humans. We are doing this for the humans still, right?

Disrupt, Interrupt, Coalesce

With all that said, the various vectors will start to circle back and meet up again. Recently it occurred to me that my dusty old Oculus Go, the tetherless consumer darling of the late 2018 VR wave, could play my media through the Plex VR app. The next day it occurred to me that I could watch 3D movies in Plex on a movie screen as big or small as I wanted, with an online friend in tandem, in VR. Whoa. (You can of course also do with your Gear VR – meaning anyone with a Samsung Galaxy S6, 7, 8 or 9 can play, as can anyone with a Daydream/Google Pixel, etc.).

 

This was a huge epiphany. The cycle wasn’t a year, or three, or five. It was a collection of various paradigms, suddenly settling into position and making perfect sense after various iterations, nudges and feedback cycles. The cumulative effect is that as a consumer, I have a more or less out-of-the-box 3D movie theater on a tetherless VR headset to watch blockbusters I can share with my friends. Is anyone even thinking about how to stick a box office in front of this yet? Maybe we should. But what will it look like? Time to start figuring that out so the content creators at every phase of production can make a sustainable living. Come together, right now.

Related articles

VRrOOm Wechat