Facebook are combining multiple advanced technologies, such as AI and 3D image capture to create better VR interaction.
One of the important features about social experiences in virtual reality (VR). A sense of presence is one of the appealing things about shared VR, with its ability to make you feel like you are with someone who might be thousands of miles away. However, the technology to truly realise this idea is still in early stages.
As the F8 conference, Mike Schroepfer spoke on the technical challenges that need to be overcome in order to create that feeling of presence in VR. In order to create an effective avatar, the system will need to be able to track the motion of a person, generate the graphics and package up that information to send it to other users, who may be in different countries.
Field of vision and depth of focus are currently limited in VR, and both need more advanced technology in order to match requirements for truly realistic avatars. Oculus have been working on many prototypes for this.
Capturing the real world is also important, as it allows for real environments to be created within VR. A captured room was shown at the conference. The room had been captured in 3D, complete with object depth of field and recreated in high fidelity VR.
Hands are hard to capture due to the complexity of them and how much they can vary from person to person and how much they can do. Recreating hands that feel natural to users is a goal for VR, to allow for natural and intuitive interaction.
All this information needs to be captured, and then sent across the world so it can be seen by another person, giving those other users the feeling the other person is truly present with them. Facebook Spaces was mentioned, in which the current version allows a near-photo-realistic avatar based on a single photo, compared to the more primitive ‘blue head’ avatars when it was originally announced.
The company is also working on lip-sync technology which uses AI to process the information to allow an avatar to speak in a way that matches the words a person speaks.
These technologies are still under development, with much advancement still expected in the future to improve the experience of presence within VR.