Photo caption: The surface of a patient's brain as seen through a surgeon's microscope. Augmented-reality overlays outline projected locations of a tumor in purple and of a vein that needs to be avoided in bright blue.
Brain surgery is never going to be easy.
When a surgeon is removing a tumor, even a slight miscalculation in the angle of entry can interfere with important functions of the brain.
But augmented reality — blending digital imagery with the physical world — may help surgeons keep their focus at critical moments during the task.
Neurosurgeon Joshua Bederson recently used augmented-reality technology to remove a three-centimeter-wide brain tumor in a 76-year-old man’s parietal lobe, a part of the brain that handles such information as touch and spatial orientation.
Combining software that builds 3D models of the head and brain, a tracking camera that matches the patient’s facial features to those on the model, and a powerful surgical microscope, Dr. Bederson was able to project a virtual image of the tumor and nearby structures directly onto the microscope’s field of view.
Before using augmented reality, Dr. Bederson could see only the structures directly in front of him through the microscope.
Now, using this technology, he can see farther — below the surface of the brain where his instruments are.
Getting the necessary information during a surgery used to be a slower, more tedious task with many steps.
Dr. Bederson would need to stop what he was doing, look up at the 2D and 3D images on various remote screens, memorize the information there, look back down at the patient's brain and his instruments, and approximate the positions of underlying important structures based on that information.
With augmented reality, he doesn't have to make that switch — known as "attention shift" — nearly as often, reducing the risk of making an error.Think of it as similar to Pokémon Go, except with much more complex imagery and considerably higher stakes.Augmented reality is different from virtual reality, explains Timothy Witham, the director of the Johns Hopkins Neurosurgery Spinal Fusion Laboratory.Virtual-reality systems shut out the outside world, giving users a complete immersive experience.In contrast, augmented-reality systems allow users to see and interact with their surroundings, but with additional data superimposed onto their visual field.
Virtual reality is useful in the planning process, but using it during a surgery would be impractical — and dangerous.Augmented reality, however, holds great potential in the operating room, according to researchers and a growing number of surgeons.However, as with many nascent technologies, there is little solid evidence that the use of augmented-reality technology in neurosurgery improves patient outcomes."At the moment, there are no outcomes-targeted studies," says Antonio Meola, a neurosurgeon at Stanford University Medical Center, who has studied augmented reality in neurosurgery extensively.There are still no prospective studies or randomized control trials, the gold standard for establishing the efficacy of new surgical techniques, he says.
Medical centers have been using augmented reality in a limited capacity in endoscopic surgeries for at least a decade, according to Mehdi Miremadi, a partner at McKinsey & Co. who has worked with several companies developing augmented-reality applications.However, large medical centers started using it in neurosurgery only in the past four to five years, he says, after advances in software that allow for greater precision.Even now, there is a margin of error of one to two millimeters in where the information overlay gets placed, Mount Sinai's Dr. Bederson says.So neurosurgeons have to remain vigilant — even a millimeter's miscalculation could mean the difference between retaining one's eyesight and lifelong double vision, he says.
But, Dr. Bederson says, making use of the information that augmented reality technology offers is an easy choice.
"I've already reached the point now in my own practice where it is so useful that I would not do certain cases without it if I had a choice," he says.