I was getting some very odd looks from strangers on the train. There’s an empty seat next to me, yet I see a procession of people notice the seat, then glance up at my face and then decide to remain standing. But the stares and the pointing by small children was all my own fault. I also knew things were going to get worse in the coming week with rejection from my wife, my kids and suspicion from my patients. Had you seen me you might have behaved the same way too. But to explain why my appearance was suddenly having such a disturbing effect on my daily interactions, and why as a plastic surgeon I’m so excited by the potential of Virtual Reality, we need to confront some uncomfortable truths about judgements we all make about the people around us.
Our Innate Bias
As babies our visual abilities are very limited. Everything beyond 20-30 centimetres is a blur. But babies are born with the ability to recognise the configuration of eyes, nose and mouth as a face. Interestingly, research has shown that when presented with two faces, babies show a preference for a symmetrical face from only a few weeks. In fact even babies a few days old prefer looking at an “attractive face” and numerous studies show that thisinherent bias extends into adulthood.
Babies quickly learn the power of their facial expressions. A stream of strangers peer into their the crib and smile. If the baby reciprocates this is met with an even bigger smile which might break into a chuckle. The baby responds and through positive feedback generates a laugh from the stranger and perhaps physical contact in the form of a tickle. A baby’s smile magically draws attention, smiles and contact from those around in a virtuous cycle of positive feedback. The mechanism of this is the mirror neurone system in the brain which causes most of us to automatically and subconsciously mirror emotions we see. Most psychologists accept that the mechanism of empathy relies on us being able to detect the emotions of others. We see how the best teachers, actors, business people, and politicians are those best able to excite the emotions of others. At the other end of the spectrum of emotional engagement is autism. One of the key features of autism is that this mirror neurone system does not function in the same way and hence impaired recognition of the emotions of others.
Clearly our facial expressions serve an important purpose to communicate what we feel internally and also communicate important information about the environment to others. In fact a smile is a powerful and rapid non-verbal signal that can be recognised at around 125 feet. In my work with patients with facial paralysis, I’m constantly reminded of the things we all take for granted. In terms of evolution the smile was an important signifier to engender trust. One can imagine how important it was for our hunter gatherer ancestors to communicate that a berry was potentially poisonous. The instant grimace of disgust told others to avoid this threat. In fact our fear of contamination, contagion or the unknown subconsciously governs much of our behaviour. In an experiment for a television documentary a journalist used make up to create a large malformation on her face, then timed how long it would take for someone to sit next to her on a busy bus route. She repeated this without the make up- the results? It took over an hour for people to sit next to her with the ‘disfigurement’ compared with a few seconds without the make up.
A moving article in the New York Times by a person born with facial disfigurement called Crouzon’s syndrome highlights the importance of not undermining the idea that looking different IS a disability because of the way you are treated at work, on dates or in public. Whilst face equality is a valid idea (as championed by charities such as Changing Faces) it is important to recognise that we are wired to seek signifiers of health and fitness. Symmetry, and the related concepts of proportion, are Nature’s most powerful signifiers of physical, and genetic health. The reason is simple: achieving symmetry is hard yet it confers extraordinary advantages. Consider it in mechanical terms and imagine how much more difficult locomotion is with asymmetric movements or weight distribution. Millions of years of evolution confirm that with very few exceptions symmetry in the midline plane conferred a survival advantage. But in our post-evolutionary era, we need to recognise our inherent biases so that they can be counteracted.
Accepting our inherent prejudices is difficult. But the idea that perhaps most damages equality (of opportunity) is from well-meaning people who think that they do not discriminate based on appearance. The fallacy of this idea is easily disproved by taking the Implicit Test, an online experiment in subconscious behaviour when presented with supporting or conflicting images and words. If you’re brave enough try it here.
Back on the train, looking at my picture, it’s not too difficulty seeing why I was getting sidelong glances.
It was Facial Palsy Awareness Week [LINK] and one of the surgeons on our board had suggested that for the week male supporters would have half a beard, and women would wear half a face of makeup. The idea was that quizzical looks would initiate conversations about facial difference and introduce the charity. It was a great idea, but it just so happened to coincide with my wife’s’ birthday (she decided that dinner at home was preferable), my self-conscious son’s performance on stage at school (“Dad, do you have to come looking like that?”), and a meeting with colleagues at the Royal College of Surgeons (hence the trip through central London).
I found the week humbling and enlightening and it provided an opportunity to taste a very thin slice of living with facial difference. In the out-patients clinic and in one-to-one interactions I did at least have the opportunity to explain that I hadn’t lost my mind. The self-consciousness I felt when walking through the hospital or down the street yet unable to explain why I looked this way was particularly revealing. Of course, one razor blade later and I was back to myself. If only my surgical blade could provide such an instant solution.
Technology for good
As a plastic surgeon, I constantly see the impact of asymmetry in people’s lives. For my aesthetic surgery patients, it’s a mismatch between how they look and how they feel. For my patients with facial paralysis, the physical, psychological and social implications can be overwhelming. The recentsuicide of an 11 year old girl who survived cancer but was left with partial facial paralysis highlights the impact of the condition.
There is a frustrating aspect of treating patients with facial paralysis that few specialists talk about. We know that early rehabilitation tends to achieve better outcomes. But the very time when patients need to practice their facial expressions (i.e. early in the recovery phase to encourage neural remodelling and make expressions more symmetrical) is the very time when patients want to hide away at home. This is the reason I’ve taken a sabbatical from my NHS post to undertake research and development of a new technology for facial rehabilitation. We are developing a specialised muscle sensing technology to teach machine learning algorithms to identify facial expressions based solely on the muscle activity. This will form part of a biofeedback system. For now we have embedded the technology into a VR headset to create a virtual mirror.
Facial muscle sensors enable expressions to be transmitted into VR
This will enable patients to perform exercises without getting distressed by their reflection when looking into a standard mirror. At the same time they will get accurate real-time feedback on their facial muscle activity. The machine learning algorithms and artificial intelligence that results from this is feeding into our development of sensor-enabled facial expression and emotion sensing glasses. Whereas currently patients have to set aside time every day to practice making symmetrical expressions in a mirror, the aim is for this to enable facial feedback whilst they perform day to day activities. One of the most demotivating aspects of current therapies is that patients attain very little feedback between clinic visits about whether they are making progress. For the ~30% of those at risk of developing permanent facial disability after nerve injury, our hope is that providing early feedback and guidance (via a cloud based telemedicine link to facial therapists) will minimise the development of chronic muscle spasms that are difficult to treat.
What Virtual Reality offers for improving lives
In an ideal world people would not be judged on their appearance, or indeed any characteristic. But it is clear that we give preferential treatment to certain people. Virtual reality offers an opportunity for those with disabilities or disfigurement to be able to interact on a level playing field. Last week I met with a charity called Special Effect that helps those with spinal injuries access technology. They described young people paralysed from the neck down living in social isolation. Whilst VR is currently seen as an isolating medium, the future ability of VR to enable people to escape their physical isolation to be able to (virtually) travel to new places, meet new people without being judged, is very exciting. There are a number of start-ups enabling virtual social interaction such as High Fidelity, AltspaceVR and vTime. They are creating interactive spaces where what you look like, and any physical disabilities are largely irrelevant.
Of course, one missing element is the ability to perform facial expressions, a problem that will soon be solved.
Virtual Reality to create empathy
The power of VR to spark emotional responses is becoming clear. Numerous studies by Skip Rizzo and Jeremy Baillensen in California have shown thepower of virtual reality to engender empathy, and its potential for reducing discrimination is very exciting. I recently had the good fortune to visit journalist and VR film maker Nonny de la Peña and become immersed in two powerful experiences showing a range of uncomfortable situations from domestic violence to the US prison system. The use of VR in education, to reduce discrimination and to almost literally allow one to walk in another person’s shoes is extraordinary.
Emteq, the start-up I’ve co-founded with some of the smartest engineers, developers and scientists around is developing the facial expression and emotion sensing technology. We’re always looking for passionate engineers, researchers and collaborators who are interested in creating a new technology for helping interpersonal interaction. If you have ideas on how facial expression technology could help in your field do feel free to get in touch atwww.emteq.net