A new augmented reality system lets physiotherapy students see inside the human body by projecting different layers of muscles and bones over the top of a volunteer “patient.”
The technology, called the Augmented Studio, is designed to enhance the teaching of physiotherapy, in which students currently use their knowledge of anatomy to understand how muscles work beneath the skin of patients they can’t see into. But the Augmented Studio bridges the gap between that theory and practice.
‘Underneath our skin’
By using tracking sensors mounted on a scaffold it projects images of our muscles and skeleton directly onto a volunteer. The images automatically follow the shape and movement of the body, giving students in the studio space an interactive all-round view of how our bodies work. It can even allow them and their teachers to “draw” on the projected image to make information and action more explicit.
The subject makes a T-shape with their arms which allows the tracking sensors to lock on. (Credit: U. Melbourne)
“What we are doing is overlaying virtual models of what we look like underneath our skin and synchronizing that with real human action,” says Thuong Hoang, a research fellow at the Microsoft Research Centre for Social Natural Users Interfaces at the University of Melbourne.
The Augmented Studio was built by Hoang, computer engineer Zaher Joukhadar, and doctoral student Martin Reinoso, who adapted Microsoft’s Kinect body sensing and tracking device as well as “RoomAlive” projection technology; both of which were originally designed for computer gaming.
Once a person steps into the projection space and forms a T-shape with their arms outstretched, the trackers lock on to them and the projected image conforms to their shape and movement.
At the moment the projected “overlay” doesn’t show how our muscles actually move when we contract and relax our muscles. Instead, it tracks the body and movement at the joints. But eventually Hoang wants to add in “animation” that can show the actual movement of muscles as the model moves.
Physiotherapy lecturer David Kelly says the students quickly embraced the technology during pilot sessions in 2016, which are continuing in 2017. He says the combination of live movement and interaction, in which students could actually move and feel the model’s limbs, helps them to grasp the relationship between their learned anatomy and how it works dynamically.
“For first year students it can be really hard to bring together anatomical knowledge with how the body actually works because it can be difficult to visualize. But when they see a real person who they can interact with, while also seeing the muscles and skeleton projected over the top, combined with the ability to draw and write on the body, it all becomes much easier for the students to learn about how the body moves,” says Kelly, from the university’s School of Health Sciences.
The Augmented Studio also provides a more visual and intuitive way of learning that Kelly says will benefit those students who naturally learn more easily by direct visualization, rather than through reading and listening.
“There has always been a group of students that struggle because the limited ways in which we have to teach may not conform to how they learn best,” he says.
Live movement and interaction allows students to better understand anatomical theory. (Credit: U. Melbourne)
Developments in AR, which seeks to use technology to enhance what we can already see, hear, and feel in the real world, are far ahead of chasing GPS-tracked Pokémon. There are viewing devices such as glasses that can overlay what we see with three-dimensional graphics, video, and holograms, and we can generate projections like games that people manipulate by moving our hands.
The big advantage of the Augmented Studio over advances like 3D holograms is that the students can actually touch and move the body, making it a much more interactive experience. They also don’t have to wear headgear, which means it could potentially be used in bigger settings with larger numbers of students.