Touch And Feel Virtual Objects In Mixed Reality

Touch And Feel Virtual Objects In Mixed Reality


This is a work by Pedro Lopes, Sijing You, Alexandra Ion, and Patrick Baudisch at the Hasso Plattner Institut.


The paper "Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation" was published at ACM CHI'18.


Learn more about the work:


We present a mobile system that enhances mixed reality experiences and games with force feedback by means of electrical muscle stimulation (EMS). The benefit of our approach is that it adds physical forces while keeping the users’ hands free to interact unencumbered—not only with virtual objects, but also with physical objects, such as props and appliances.


We demonstrate how this supports three classes of applications along the mixed-reality continuum: (1) entirely virtual objects, such as furniture with EMS friction when pushed or an EMS-based catapult game. (2) Virtual objects augmented via passive props with EMS-constraints, such as a light control panel made tangible by means of a physical cup or a balance-the-marble game with an actuated tray. (3) Augmented appliances with virtual behaviors, such as a physical thermostat dial with EMS-detents or an escape-room that repurposes lamps as levers with detents.


We present a user-study in which participants rated the EMS-feedback as significantly more realistic than a no-EMS baseline. Our project is open source, feel free to try out some of the HoloLens applications (made using Unity3D). All code is available here:

Related articles

VRrOOm Wechat