Soccer On Your Tabletop in action
After 3D television failed to fulfil its early promise as the “next big thing” in sports broadcasting, it has been Virtual Reality (VR)’s turn to carry the burden of the industry on its shoulders.
BT Sport has produced live matches and highlights for several events, including the UEFA Champions League Final, while BBC Sport is making a range of VR content available on a dedicated application for the 2018 World Cup in Russia.
Sports organisations are especially interested in how VR can bring remote fans closer to the action by creating an experience similar to that of being in a stadium.
VR sport hasn’t yet hit the mainstream, but it is gaining in popularity thanks to more powerful smartphones and mobile networks. Indeed, the arrival of 5G networks from 2019 onwards will bring faster speeds, higher capacity and ultra-low latency ideally suited for VR.
But a group of researchers are looking at another technology that could change the way we enjoy sporting events – Augmented Reality (AR). Whereas VR presents a fully immersive image, AR adds digital elements to real life scenes, as best demonstrated by Pokémon Go.
Soccer on Your Tabletop
“Soccer on Your Tabletop” is a prototype AR system that converts any football video on YouTube into a 3D hologram that can be projected onto a flat surface like a table. This can be viewed using a compatible headset.
3D renders are already possible through the use of motion capture and other technologies, but these techniques necessitate the use of numerous cameras dotted around a large area like a stadium. The cost and effort involved limits its widespread use and practicality.
In a paper due to be presented at the Computer Vision and Pattern Recognition conference in Salt Lake City, the University of Washington’s Konstantinos Rematas and his colleagues detail how they wanted to see if it was possible to achieve the same feat simply using a 2D video as an input.
The real innovation of the system is how it is able to locate players on the pitch. The researchers used EA Sports’ FIFA video game to train a machine learning model that can estimate the per pixel depth values of any soccer player from a single frame of footage.
You can see various examples of the system at work in the video below, but there are some limitations that the researchers recognise.
They haven’t yet mapped the jumps of the players because it is assumed they are perpetually on the ground and there is no mechanism for tracking the ball, something that is “future work”.
At present, the system only works with recorded video footage, but it could in theory be used to convert a full live match. This would require the developers to not only rectify the limitations they describe in the paper, but they would also need tools to reconstruct the 2D video in real time and for more efficient data compression and streaming.
AR in sport
AR has been around for some time, but the technology and its use cases are still in development. As with high-definition, 4K and VR, sport is seen as an attractive use case to foster interest and encourage adoption. Early attempts have seen sporting organizations use AR for interactive signage and match programs, but there is potential for broadcasting.
One of the main drawbacks of VR’s use in sports media is that something like a soccer match is such a shared experience in which fans feed off the reactions and comments from others. AR could deliver the combination of innovation while maintaining the sense of community.
For example, “Soccer on Your Tabletop” could allow soccer fans to watch a game from different angles or review a contentious penalty decision or a wondergoal from multiple angles. In fact, such an application could interest soccer clubs for their pre and post-match analysis.
A 3D hologram projected onto a table would allow coaches and analysts to communicate insights, such as positional data or a tactical overview, to players in a more visual way. One of the main things I hear from sports organizations adopting technology into their processes is that it must be accessible to the players, otherwise it won’t be effective.
Ultimately, adoption is likely to be constrained by technology. Smartphone-based AR has taken off in a big way in recent years thanks to support from Android and iOS, most notably through the ARKit framework, while the phenomenal success of Pokémon Go introduced many people to the concept for the first time.
It will take time for self-contained AR devices to mature and for networks to advance to the point where the experience is seamless. “Soccer on Your Tabletop” uses the Microsoft HoloLens, which is the most high-profile dedicated device available, but it’s hardly the cheapest and most fashionable piece of electrical equipment on the market.
But manufacturers will refine the technology and designs, while mobile networks will get better. Analysts at CCS Insight predict the market for VR and AR headsets will be worth $10 billion by 2022 and see huge potential for AR smart glasses, especially if someone like Apple gets involved.
It doesn’t take a huge leap of faith to imagine us watching a whole soccer match in AR on our kitchen table with the family in the future.