IN JUNE 2003, the San Francisco company Linden Labs launched a Massively Multiplayer Online Role Playing Game called Second Life. It quickly grew to over a million users, and has become a touchstone for the potential social adoption of virtual worlds. While other such games, like World of Warcraft, have seen far wider adoption, the creators of Second Life insist that it is not a game in the usual sense because it does not have set narratives. Instead, its sole purpose is to create a world in which users can engage. In Second Life, users hang out, build communities, run businesses, and of course, have sex, all through their avatars.
Less than a year after its founding, Second Life got its first newspaper. Started by the philosophy and linguistics professor Peter Ludlow, it began as an experiment in virtual journalism on another platform called The Sims Online, where it was called The Alphaville Herald. Ludlow, or rather his avatar, Urizenus Sklar, was quickly kicked out of Sims for writing editorials against the company that built it, Electronic Arts. In that virtual world, it turned out, free speech protections were limited.
But in Second Life, the renamed Second Life Herald thrived, reporting on both the actions of individuals and corporations within the game, and on the company that built it, which users call “the government.” As the economy of Second Life grew and “real world” companies such as BMW, Major League Baseball, and American Apparel began operating within this virtual space, Reuters opened a virtual bureau and assigned two reporters to cover stories about virtual currencies and the businesses operating in Second Life.
One of these reporters, Adam Pasick, said his reporting in Second Life wasn’t much different from how he would operate in the physical world: “You talk to as many people as you can, you read what you can, you find interesting stories and you chase them down.” Others dipped their toes in virtual journalism, too. Wired produced a travel guide for “visitors” to Second Life, and CNN asked citizen journalists within Second Life to add their news and stories to its iReport site (which includes a story about falling in love with virtual avatars).
In a 2007 CJR article on virtual journalism, Stephen Totilo argues that there were two phases of journalism in Second Life: the initial community members who experimented organically with doing journalism in a world of anonymity, multiple identities, and teleporting; and a second wave of external media companies that became interested when Second Life looked to be a significant economic and social phenomenon.
The growth and impact of Second Life ultimately proved more limited than some of those onlookers expected. While the community grew to over five million users, by 2007, the number of active avatars at any one time was less than 30,000. This could be in part because the virtual space and the avatars that make up Second Life are represented through fairly basic 3D animation. But it also could be that Second Life, like other avatar-based virtual spaces, never intended to replace a user’s “real” life. While many derive meaning from these experiences, worlds like Second Life are not seeking to fully represent physical reality, but rather to serve as a parallel experience that offers an outlet for limited aspects of people’s “first” lives. But a new generation of virtual reality technologies seeks to significantly expand the social and physiological effect of virtual experiences, with profound implications for journalism.
While the promise of virtual reality has been present in research labs, the gaming industry, and science fiction for over 30 years, it’s only now that we have the computational power, screen resolution, and refresh rate to use VR in a small and inexpensive portable headset.
This past year, a wide range of virtual reality headsets has entered the market. VR content is being created by movie studios, gaming companies, and journalists, and the largest technology enterprises in the world are investing significantly in virtual and augmented reality R&D. Users can play video games, sit courtside at a basketball game, and watch news of global events in virtual reality. Industry has taken notice, and the development of hardware, software, and content for virtual reality headsets is poised to become a massive enterprise.
As the Silicon Valley companies building these worlds gain power, tech journalism has proven insufficient to hold them accountable.
Enter Facebook. In March 2014, the tech giant purchased a virtual reality hardware company called Oculus for $2 billion. Facebook’s ambitions in virtual reality are not modest. As founder and CEO Mark Zuckerberg stated in a post announcing the purchase, “One day, we believe this kind of immersive, augmented reality will become a part of daily life for billions of people.”
Three days after the acquisition, Zuckerberg hired the renowned video game developer Michael Abrash to be Oculus’s chief scientist. Abrash spent a good part of his career developing Quake, a hugely successful multiplayer first-person shooter video game. His hire signaled immediately that Facebook saw the future of virtual reality as much more than linear 360 video or discrete stories (the focus of much current virtual reality content, including journalism), but rather as a place in which people will live. If Second Life is an attempt to parallel human experience, Facebook is trying to seamlessly weave together our physical and digital worlds. “Twenty to 30 years from now,” Abrash explained to the journalist Steven Levy, “you’ll have on a pair of glasses, or maybe it will be contacts, and you’ll just be able to be in augmented or virtual reality, as you choose.” What began as a website on which people posted content about their worlds could increasingly become a world in itself.
As Facebook and others begin researching and developing technologies that could augment our lives in significant ways, a new space is opening up for journalism. And unlike early virtual journalism experiments in Second Life, which ultimately mimicked traditional “real world” reporting, journalism inside these new virtual worlds will require an entirely different set of skills and approaches, and will challenge three core journalistic concepts: representation, witnessing, and accountability.
First, virtual reality challenges the ways in which journalists think about representation. At the core of VR’s unique power is a deception–that the user believes she is experiencing something she is not. The goal of journalism in VR, therefore, is to inform the user by blurring the act of journalistic representation. But journalists cannot appropriate the physiological power of virtual reality without also thinking seriously about how leveraging it for journalistic purposes changes the way the world is represented.
Second, virtual reality challenges journalists’ ability to serve as witnesses with agency. It is entirely unclear what tools will be needed to observe events and institutions in a virtual space that is created by a confluence of human intervention and algorithmic control. If the boundaries between observation, participation, audience, and social structure fundamentally break down in virtual worlds, it is uncertain whether virtual reality journalism can be done by a human at all.
Third, as Facebook begins to build a virtual world and signals its ambitions to augment human capabilities, there has never been a greater need for accountability journalism, both within virtual spaces and for the companies building them. These virtual experiences will be designed and increasingly automated to be as addictive as possible. They will be marketed aggressively and widely, and could radically change our lives. The technologies driving them will undoubtedly be used by governments and militaries to seek ever greater control. But as the cluster of Silicon Valley companies building these futures rises to significant and largely unchecked social, political, and economic power, technology journalism has proven insufficient to hold them responsible for their actions.
Whether or not these technology futures emerge, they are being discussed and researched at one of the largest companies in the world, with a user base of over 1.5 billion people and rich data about much of the world’s consumption, movements, knowledge, and networks. How these virtual worlds are designed and created, and how humans will evolve to engage with these new technologies, pose fundamental problems for journalism.
REPRESENTATION AS DECEPTION
Asked when he first realized the power of virtual reality, Abrash describes being in a headset and looking over the edge of a cliff. The simulation was notably artificial; the walls of the cliff were actually lined with an image of a news story about Facebook. “Nothing about it looked realistic,” he explained in the interview with Levy, “except that everything moved the right away, and that was all it took to kick in those low-level systems to say, this is real, you are in an emergency, you’re in danger.”
What he was feeling is the physiological effect that makes virtual reality so powerful. Even though your mind rationally knows you’re not staring over a computer-generated cliff, the feeling is real enough to trick your body into reacting as if you were.
While the precept that the vividness of a representation makes a story more real has always been deeply flawed, it is this very attribute that arguably makes virtual reality a revolutionary step in the otherwise evolutionary progress of delivery mechanisms for journalism. While the introduction of radio, photography, video, and social media each changed how journalism was framed and how societies produced and consumed media, in each of these previous media, the master’s hand was clearer than in virtual reality. With a photo or a video, the audience knows the photographer or videographer is framing the image. There is no pretense that one is seeing everything, and no illusion of actually being there. With social media, one can be immersed in flows of information, but the body does not think it exists inside Twitter.
While the introductions of earlier media surely evoked visceral responses from their unprepared audiences—it is rumored that an early motion picture, L’Arrivée d’un train en gare de La Ciotat, caused a panic in the audience when it was first screened in 1895 in Paris—virtual reality is premised on a far more comprehensive trick. Journalists dabbling in VR are embracing this deceptive quality, amid considerable hype. The New York Times, AP, The Huffington Post, Vice News, USA Today, and the BBC, among many others, have all invested significantly in virtual reality journalism. Most of these initial projects place the user in a highly constructed five to 10 minute “story,” to be viewed either in a VR headset or in 360-degree 2D video. At their best, they represent the news industry creatively experimenting in a new medium that has clear journalistic utility.
In 2014, when I worked at the Tow Center for Digital Journalism, we developed one of the early 360/3D virtual reality journalism projects in partnership with the documentary program Frontline. The project takes the user to West Africa, where the Ebola outbreak began. Our intention was to use the feeling of being there as a tool to tell the story of how the virus spread.
There is a scene in the experience in which the viewer is standing in the village where patient zero (the first known case of this epidemic) died. A young child runs past the camera. The effect is visceral. This scene could have been beautifully written or captured in a still image, but virtual reality adds another layer of pretense: The journalism itself is a kind of lie.
At the core of the virtual reality deception is the concept of empathy. The notion that virtual reality is an “empathy machine” is pervasive in the breathless marketing of the booming VR industry, and it’s one that has excited journalists. The core premise is that feeling that one has experienced something will make a user care more deeply about it.
Empathy is actually a concept about which scholars know a fair amount. Modern social psychology divides it into three components. Empathy can be affective, the emotional response prompted by experiencing an event or action. It can also be cognitive, in that by experiencing something, one can better understand it. Finally, empathy can have a behavioral effect, sparking a conscious decision to take action as a result of both the emotional and cognitive processing of an event.
The scene could have been beautifully written or captured in a still image, but virtual reality adds another layer of pretense: The journalism itself is a kind of lie.
The question is whether these variations of the empathy effect translate to virtual experiences. Does seeing a humanitarian crisis in VR lead to a stronger emotional response, increased comprehension of the crisis, and potential policy action in response? Numerous research projects are studying virtual reality’s potential to trigger all three of these dimensions of empathy. The possibilities for marketers, NGOs, governments, the UN, social platforms, and of course journalism, are clear: If virtual reality can make people care, understand, and act, plenty of people will be lining up to leverage its persuasive utility.
For journalists, however, clothing the act of representation in the physiology of faked experience is a problem. Journalists will need to seriously consider the costs of embracing this degree of deception. While already problematic in the first generation of VR journalism segments currently being produced, this challenge will only expand when journalism moves into more fully realized virtual worlds.
WITNESSING IN THE METAVERSE
Every 10 or 15 years, Mark Zuckerberg has argued, a new computing platform emerges, and with each new iteration, two principal characteristics evolve: Platforms become more natural, immersive, and intuitive, and sharing becomes a richer experience. Mainframes were isolated and could only be used by experts. PCs entered our homes, and their easy interface and multimedia capability made them a part of our lives. Mobile is now everywhere, connecting us intimately to the internet. For Zuckerberg, virtual reality is the next platform iteration: a space we inhabit, rather than a site we visit.
The mission he has given Oculus—to empower everyone to experience anything—sits uncomfortably at the intersection of science fiction and techno-utopianism. The term “metaverse” was coined by author Neal Stephenson in his 1992 novel Snow Crash. This dystopian science fiction describes a convergence of physical and virtual worlds, in which humans are participants in a universe of mixed reality. A decade later in Ready Player One, Ernest Cline similarly depicted a declining industrial society where humans increasingly exist, via headsets and haptic suits, in a virtual world called the OASIS.
Novels like these explore moments when liberal humanist definitions of human agency and free will blur into the structures of computation. In these imagined spaces, the distinction between human action and algorithmic generation breaks down. This results in both a promise of enhanced capabilities and experiences, and also the possibility of control and manipulation.
It is in these imagined realms that the discussion around virtual reality quickly connects to broader questions about humanism and technology. Zuckerberg is clear that he sees a future of human augmentations, or what scholars such as Andy Clark, David Chalmers, and Peter B. Reiner have called “the technologies of the extended mind.” “Our goal is to build systems which are better than people at all the perceptual things that people do,” Zuckerberg has said. “I think we can get there [in] between five and 10 years.”
It can be a challenge to extrapolate these science-fiction futures from the current discourse around virtual reality. Facebook has bought a VR headset company, released two VR cameras, and is integrating 360 video into its News Feed. These steps can appear iterative, but focusing only on these present evolutionary steps risks distracting us from their broader intentions. As we interrogate the discrete VR technologies being developed, we also must consider the longer-term objectives of companies like Facebook, and how and whether journalism can exist in the metaverse.
In February 2016, Facebook created an internal team to work on what the company calls Social VR. The group is led by two video game developers whose mandate is to build a virtual world where one can experience either a virtual representation of the non-digital world (think a virtual classroom or meeting where everyone is beamed into a room together), or a generated environment (think a tennis match in zero gravity against a lifelike, algorithmically generated opponent).
At a demo of Social VR, Facebook Oculus researcher Yaser Sheikh noted that this future will only be possible if designers can perfectly mimic the subtleties of human expression through code. Quoting anthropologist and linguist Edward Sapir, he pointed out that in the physical world, the nuances of human interaction are “an elaborate code that is written nowhere, known to no one, and understood by all.”
Facebook wants to crack this code so that the technology behind the immersive experience essentially disappears. In this vision, our perception of avatars merges with our understanding of what a human looks like and how he or she behaves. Sheikh believes this social presence “will change how we live in this world and with each other.”
Abrash is on it. “This is only a perceptual psychology problem,” he told Bloomberg. “The key is that what you experience is constructed in your brain.” At the moment, affecting this construction technically entails tricking our senses with visuals beamed through headsets and retina projection, sound through earphones, and touch through haptic devices. But both Abrash and Zuckerberg have mused about engineering deeper connectivity directly to the brain as a way to achieve “true fidelity.”
A challenge for journalism is that the lines separating witness, audience, and participant fade in the virtual world. Put another way, you’re both a consumer of media and a creator of it.
The founder of Oculus, Palmer Luckey, is more direct. “The goal is clear: it’s to make VR technology that’s as real as real life, with none of the limitations.”
They are all describing an ambition to achieve a world of increasingly mixed realities that are guided and shaped by unseen technological interventions. A challenge for journalism is that in this world the lines separating the witness, audience, and participant fade. The journalist and audience are at once observers, creators, and participants intertwined with technology. Put another way, when you’re living in the virtual world, you’re both a consumer of media and a creator of it.
Journalism as an act of representation was once clearer on the distinction between observation and creation. While it was generally understood that the journalist observed the world through a filter of her cultural, political, economic, and institutional contexts, technology was seen by many to occupy a more focused role as a tool through which the world could be captured or disseminated.
The rise of the internet, social media, and computational methods has allowed a much wider group of people to serve as observers, and technology has increasingly come to shape not just how, but what we know and communicate. Technology thus becomes an object of journalism to be made visible, and the act of an audience engaging with media becomes a fundamental part of what journalism is. When you consume media, you’re part of creating what makes it meaningful. This is true of print, where a reader’s knowledge and background play a role in how she interprets the news, but it is even more salient in regard to virtual reality.
By this logic, media is not consumed; it is created via participation in the same way a physical event is made up of those who are present. The journalistic act of witnessing becomes inseparable from the collective building of the virtual world, and determined by the same set of algorithmic rules that govern the space.
Postmodern technology critic N. Katherine Hayles goes further, depicting technology and the human as fundamentally intertwined, resulting in “a shift in what it means to be human.” Even more challenging for journalism, she argues that environments, behaviors, and people created or augmented by AI could have a degree of agency that lies outside either human control or our ability to observe. The algorithmic systems governing our engagement in these spaces are in fundamental ways unknowable to humans. Witnessing in these spaces may prove impossible.
ACCOUNTABILITY FOR THE CREATORS
As virtual worlds begin to be imagined, developed, launched, and promoted, it is important to ask what it will mean to hold both the actions facilitated by and within them, as well as the companies creating them, accountable. There are at least three ways in which virtual reality complicates the core accountability function of journalism.
First, it is unclear how journalism can even be done inside virtual worlds. While a Reuters reporter interviewing participants in Second Life showed creativity, and while the recent work in algorithmic ethics and reporting by journalists and scholars such as Cathy O’Neil, Kate Crawford, Nick Diakopoulos, and Julia Angwin has shed light on the ways algorithms are both embedded with biases and can shape social and economic outcomes, this work has involved humans seeking to witness computational systems. As these automated worlds become ever more complex, there will likely come a time when only other automated entities—presumably overseen by automated systems—can understand them.
If this is the case, the human agency that is core to the notion of journalistic witnessing quickly disappears. Can a human journalist legitimately be expected to hold a computational phenomenon that augments, mimics, or creates human experiences accountable? Perhaps it will be necessary to develop journalistic avatars, capable of understanding the behaviors, structures, and biases of the virtual worlds in which they operate.
Perhaps it will be necessary to develop journalistic avatars capable of understanding the behaviors, structures, and biases of the virtual worlds in which they operate.
In this construct of automated systems holding automated systems to account, the ways in which journalistic intentions and methods are coded will predicate their value. How these journalistic systems will be coded, and who, or what, will know how to design them, is a challenging proposition.
This potential makes recent concerns around the influence and biases of the Facebook News Feed algorithm or the power Facebook has over publishers seem quaint. Instead, these contemporary debates should be seen as harbingers of the future toward which Facebook and other large technology companies are speeding, worlds created and determined by a combination of human input and artificial intelligence, and governed by algorithms.
Second, it is critical for journalism to view Facebook the company as part of a significant power shift away from industrial era institutions to a new generation of large, networked actors. Zuckerberg himself gets part of this story right. He views Facebook as a company that seeks to empower individuals rather than control them, and as part of a shift in social organization from industrial-era hierarchical organization to more egalitarian digital networks.
In some ways, this is true. Networked communication has enabled a new form of collective action. However, in the process of connecting individuals, Facebook has itself become a new kind of established power. Facebook controls the distribution of a rapidly growing proportion of news; it holds large amounts of data on a fifth of the world’s population; and its algorithms shape behavior and determine how we learn about global events. We need journalism that holds this new form of societal power to account.
Third, journalism must grapple with the complexity and biases of this emerging corporate and governance ecosystem, and determine the skill sets needed to understand it. If the rise of augmented and virtual worlds fundamentally shapes human experience, then we need to consider the accountability of actions and structures that determine what occurs within them.
Technologies always harbor the complex values and interests of those who design and build them, often with unintended consequences. For example, as new platforms become more ubiquitous and further integrated into our lives, the data used in different components or apps within them can flow throughout the ecosystem, as communications and surveillance scholar Anders Albrechtslund has described. Data about our entertainment habits could bleed into our health apps. Data from home devices could feed into insurance systems. Surveillance data can be used to recreate crime scenes or find criminals.
When this challenge is layered into a virtual world where we could spend more and more of our lives, in a space determined by largely unknown computational systems and incentives all run by a single company, the question of whether journalists will have the access, skill sets, and institutional backing to take on these types of complex structural problems becomes unavoidable.
At a recent Vanity Fair event on virtual reality, Mark Zuckerberg was asked whether Facebook’s future lies in the construction of a metaverse. There was an awkward pause as philosophy and science fiction bumped up against CEO PR. “I think we’ll ease into that.” Zuckerberg smiled, and the audience laughed.
It remains to be seen what versions of virtual reality will be developed, but at the moment, the companies imagining and building this future are doing so largely unchecked by governments, civil society, or the media. What’s more, these evolving technologies increasingly challenge the core historical journalistic values of representation, witnessing, and accountability.
And herein lies both the challenge and opportunity for journalism.
There is no digital tool that I use more, and understand less, than search. It is through the filter of search—a black box of algorithmic decision-making hiding behind copyright protection—that my information world is intermediated. Google search biases what I know, being default on all my devices. Google Scholar influences what academic work I read. Facebook and Twitter searches shape my networks. These tools are extensions of my mind. But what I search online is also used by others. The bread crumbs of my searches are tracked, stored, queried, monetized, and surveilled, all guided by equations serving multiple and often conflicting objectives: efficiency, usefulness, discoverability, profit, and security. These tools are at once the lifeblood of journalism and a key force eroding journalists’ ability to play their proper social role, yet in this contest between utility and vulnerability, there is no practical way of dislocating from the digital world. I choose to remain connected, but also mindful of the need to think critically about the tools we use and the digital world we’re collectively building.