Multi-Sensory Virtual Reality
There’s something really powerful about smells. They can take you to another time and place and evoke memory in a way that nothing else can. Combining that with Virtual Reality means a really powerful experience.
I made my first smelly film last year. It came about by chance. When I watched an edit of a film we’d made, it didn’t feel like I was really there. I couldn’t get to that ultimate place of presence. It was too cold and the environment was too sterile. This is the problem we, as VR producers face.
If we are creating an experience where you are in direct sunlight on a beach, you need to feel hot.
If you are building a snowman, you need to feel cold.
Being next to food, we need to smell it.
Otherwise, we are only creating half an experience.
So I set out to capture the smells and the heat of the environment that this immersive expeirential film was set in. Some of the reactions are here:
Reactions to the smelly film at Royal Television Society, London, November 2016
I’m talking about this a lot in the next few months but wanted to share four of the important findings that our research has shown.
- The real temperature needs to match the virtual temperature: nearly all experiencers (see below) watching the film responded positively to heat being added to the environment. They all spoke about the “expectation” of the environment being hot. A cold room would not create the right conditions for presence in the environment to be achieved. One respondent said “I felt like I wanted to take my jacket off because I thought I was in a different country so why would I have my jacket on?”. Another said the heat made it feel like “the film really responds to you”.
“The heat once got almost a bit too hot but that’s what would happen in that situation. It added to the experience massively.”
2. Smells are so important: Previous studies, mostly located within CGI environments, have shown the importance of smells within VR. A study by Krueger in 1996 suggested virtual surgical training systems would be limited until odors were present in the environment. Munyan et al in 2016 found that the more senses that were added, the greater the level of presence would be. So we added in a lot of smells. Experiencers wanted more. The Chinese spices, the rotting fabrics, the gone off food, the washing detergent, were not enough. More would have been better!
“When you are walking through the streets, there’s all the different parts of the market. You can imagine you are there and someone sat in a café next you and you can smell what they’re eating.”
3. Multi-sensory work needs to be varied: although the heat fluctuated within the experience, increasing at the critical points, our smells were more constant. We simply didn’t have the sophisticated technology needed to programme the smells at different points, though that it something that we are working on. This is what the experiencers all said that they needed.
4. Adding senses into 360-film helps us in the quest to find presence when interactivity and agency in a film is missing: This research backs up previous studies that show the importance that cross-modal work has within VR. Experiencers felt like they were in the environment and responded as they woudl in the real world. One said: “With my senses being alert I looked into a lot more detail about what was going on. Whether that’s with the second time watching it or because my body was responding in a different way. I was looking at more finer details. I was looking at where the smell could be coming from.”. Adding in multiple senses, gave experiencers a greater feeling of presence, interactivity and agency, despite there actually being none.
“The sensory side of it was unbelievable so that was amazing to have the sensory side of the smell and heat which helped immerse you in it. But the movement, I actually felt like I was moving around in the environment in a more natural way than I did before so I felt like I could almost control where I was rather than just being led by the environment.”
More will beocming on this in teh next few months. Production has started on other multi-sensory films and we continue to use these to research how presence and embodiment within the virtual world can be achieved by adding in layers of immersion.
So, one final thought, which needs a lot more scientific study on:
NB. Experiencers: we like to call those that watch immersive experiential film “experiencers”. Referring to an ‘audience’ implies a sense of mass and using the term ‘viewer’ implies an individuated viewing, both of which feel inadequate to describe the experience of immersive experiential film. Experiencers more actively explore the space in significantly different ways to traditional methods of viewing. Consequently, an ‘experiencer’ is an individual, viewing a shared film space, in an exploratory manner.