Virtual reality now helps you get 'inside the head' of Mongolia's conflicted emperor Kublai Khan - Credit: Leisure and Cultural Services Department/59 Productions
The epic online performance Stones of Venice — directed from Britain using VR and motion-capture technologies in Hong Kong — creates a more intimate connection with audiences for this tale of the ruler’s existential crisis.
Kublai Khan, who ruled the Mongol Empire in the 1200s, is a prominent figure in history known for being the first non-Han emperor to conquer China and unite the country.
His powerful reign has made him a popular figure in literature, including the 1972 novel Invisible Cities, by Italian author Italo Calvino. The story imagines Khan’s candid conversations with Venetian merchant and explorer Marco Polo, who travelled throughout Asia along the ancient Silk Road.
The award-winning British design studio, 59 Productions, has presented a scene from the story to viewers in Hong Kong and around the world. Because of travel restrictions caused by the ongoing Covid-19 pandemic, the performance, which was produced using virtual reality (VR) and motion-capture technologies, has been available to watch online since October 31.
Leo Warner, the veteran creative director and experienced video designer who founded the studio, created the VR experience with Hong Kong-based artist Jeffrey Shaw and two local dancers.
Warner has led his team in designing the opening ceremony of the 2012 London Olympics, watched by more than a billion people.
The show will form part of ReNew Vision, an online platform of the New Vision Arts Festival, which has been organised by the Leisure and Cultural Services Department since 2002.
The scene, called Stones of Venice, is adapted from a conversation between Khan and Polo in the palace courtyard.
Leo Warner, founding director of 59 Productions – the British team behind Stones of Venice – says creating a VR version of a performance based on a novel was ‘an enormous challenge’.
Khan reveals that he experienced an existential crisis over the meaning of ruling a huge empire. While people around him may have called him the greatest ruler of all time, he felt empty because he was unable to see or experience all of it.
Polo, who travelled extensively across Khan’s empire, has been providing information about how his decisions made an impact on the people he ruled.
Through the show, Warner says he wants to convey the message that everyone’s individual story is fundamental to the fabric of the world. “The story of a city or a country only exists in co-existence with every other story,” he says.
Creating the show virtually provided his team with new possibilities to engage with viewers, he says. “The dance is at the heart of our storytelling.”
Warner says the Covid-19 pandemic was ‘a very significant blow’ to his team because it prevented them from travelling to Hong Kong to present the show.
In bringing the virtual experience to Hong Kong, Warner’s team collaborated with Jeffrey Shaw, an expert in digital media technologies and chair professor of School of Creative Media at the City University of Hong Kong.
Shaw, originally from Melbourne, has decades of experience applying motion capture technology in the arts. In 2003, he was awarded the prestigious Australian Research Council Federation Fellowship for his pioneering research into artistic expressions.
Warner says virtual technology helps the audience understand from a first-person perspective why Khan found himself in an existential crisis.
However, the performance will not replace the original show, since it will be shorter in length and is designed to present only the core theme of the story, he says.
Warner says the production aims to put the audience inside Kublai Khan’s head as he faces an existential crisis.
Jeffrey Shaw, an expert in digital media technologies, says motion capture allowed Hong Kong dancers to work with the British production team as they created the show remotely.
Warner says while directing the show via video conferencing was a new experience, the shoot ended up running efficiently.
Shaw says the challenges posed by the Covid-19 pandemic have led media makers to explore new possibilities in creative production.
The performance is carried out by Hong Kong dancers Alice Ma and Joseph Lee. This is an international collaboration – with both of them working with the London-based creative team remotely to curate the performance.
Ma, who appeared in the American Dance Festival in 2007 and has collaborated with various local arts groups, says it has been challenging because they had to register the dance moves accurately over a video call.
Lee, who has performed extensively abroad, including in France, Japan and Singapore, believes VR has opened up new storytelling opportunities. "It's super three-dimensional," he says, which is different from conventional stage performances.
Hong Kong dancers Alice Ma and Joseph Lee say it was challenging to perform in the VR show because they had to learn their dance moves via video call.
Shaw says motion capture – a technology that documents the dancers’ movements in a computer program – has played a central role in creating the VR experience.
To record the moves, dancers must wear a black costume with small markers placed over their bodies. The cameras, with the help of motion-capture software, then track the movement of the markers. Once the moves are saved digitally, Shaw’s team can assign them to virtual characters that the audience will see in the actual VR show.
Shaw says the technology has been widely used in the animation industry because it enables choreographers to create different artistic expressions vividly.
The show was released on October 31 as part of New Vision Arts Festival’s streamlined transitional online platform, ReNew Vision, which showcases five specially commissioned online works, encompassing dance, theatre and music and on-site and virtual elements.
Please visit www.newvision.gov.hk for more details about the performance.