Chilean artist Sebastian Errazuriz vandalized Jeff Koons's virtual "Balloon Dog" last fall using virtual graffiti on his own app as part of a “stance against an imminent AR corporate invasion.” Sebastian Errazuriz
So far, major forays into our augmented world have been pretty harmless. But with technological advancements and unchecked intrusions by private companies, the future could be terrifying.
I’m sitting at New York University (NYU), in a Brooklyn Tandon School of Engineering building with wires sprawling across my body while I stare at a 98-inch TV screen. Glasses with inward-facing cameras are tracking my pupils; plaster wrapped around my finger is measuring my perspiration; cheek pads are monitoring whether or not I’m smiling; another device looks at my heart-rate; and an electroencephalogram (EEG) headset—a contraption with wet squidgy nodes on the end of prongs, similar to a head massager—monitors my brain activity. All while my face is being filmed.
The researchers want to quantifiably measure my response and stress levels to two slightly different virtual environments. On the screen I am looking at a basic scene created in Google SketchUp. Mock-ups of an existing NYU building are identical minus a couple of minor differences: one has wider windows and lighter wall colors. Within the two environments I have to navigate my way up a set of stairs onto a mezzanine level, open a door, switch on a thermostat, and return.
Hardly thrilling (that’s the point), but Semiha Ergan, an assistant professor at NYU, is keen to see if the architectural theory behind such rudimentary design aspects is true and can quantitatively measure how much of a difference there is. Though the researchers couldn’t disclose the architecture firms actively interested in this research, I was told that a few well-known “three-letter” firms were keeping a close eye on things.
As you might have guessed, Ergan’s hypothesis was right. In the supposedly less stressful environment that had wider windows, I perspired less, my heart-rate was lower, and my facial expressions along with the EEG monitoring also indicated lower stress levels. The ramifications of this research pertain not only to virtual reality (VR) but augmented reality (AR) services, in which computer-generated visuals are superimposed on a tech user's view of the real world
“Do people have an agency to save these bits and bites? Yes, of course they do. Does the conservation community have the proper tools in place? No.”
Unfortunately, Ergan’s test didn’t include Balloon Dog, Jeff Koons’s AR-only sculpture in Central Park. The virtual sculpture appeared exclusively on Snapchat screens in New York’s Central Park in late 2017 when users visited the area. Chilean artist Sebastian Errazuriz decided he wanted to vandalize the work using virtual graffiti as part of a “stance against an imminent AR corporate invasion.” After submitting his version to Snapchat without a response, he placed Balloon Dog in the same geo-tagged location as Snapchat’s using his own app, ARNYC.
The response gained traction online—Hyperallergic soon declared it,“the first act of vandalism to occur in the realm of augmented reality.” Errazuriz’s “vandalism” and the attention it received may be a preview of what to expect from augmented additions to the public realm through social media apps.
Inevitably, advertising will join the augmented public realm. It already has in less immersive ways, including televised sports broadcasts that project advertisements onto to the field of play. Likewise, visual recreations of dead artists can now appear on a concert stage.
Advertising online is hyper-targeted thanks to services like Google AdSense, which employs user-tracking cookies to show the most relevant ads for a user as determined by algorithms. This system relies on “impressions”—essentially, each time an advert is shown by Google, either on a Google search or on a advert box operated by Google—something which, as Ergan’s test has shown, could be tracked easily when monitoring eye movement if actuated in a virtual or augmented environment.
But what happens when the echo chamber of social media is married with AR and VR? “The effects could be the real-world amplification of what we're seeing online,” architecture historian Owen Hopkins told CityLab. “It could drastically affect our everyday experience of reality.”
Will people even want to engage or be able to process AR-heavy spaces?
Physical ads have a propensity to gain cultural significance. The iconic yellow arrowhead signs of In-n-Out Burger across the west coast have become allied with the culture surrounding the fast food restaurant itself. In the U.K., Catford’s “Big Cat,” a locally famous piece of mimetic architecture in South East London that advertises the Catford shopping center, evokes so much civic pride that residents campaigned (successfully) to save it when developers proposed its demolition.
But AR advertising lacks phenomenological qualities and is bereft of permanency, all of which can diminish a sense of place.How would both these scenarios have played out in a world with AR advertising? Would a claim for the preservation of a virtual object—an AR version of Long Island City’s 5pointz, for example—have any merit, say if the firm that implemented that AR decided to discontinue it?
Stefan Simon, director of the Institute for the Preservation of Cultural Heritage (IPCH) at Yale University, describes the advent of digital and virtual technology in architecture as the “4th industrial revolution.” Speaking to CityLab, Simon warned that without developing the proper tools to save it, much of our digital heritage will be lost. “Do people have an agency to save these bits and bites? Yes, of course they do,” he argues. “Does the conservation community have the proper tools in place? No.”
Simon references Vint Cerf, an internet pioneer who a few years ago said that future historians looking to study the 21st century will find an "information black hole." This is because, according to Simon, the programs needed to view our digital files will soon become be obsolete. “Obsolescence is a key factor in this process [and] migration of data alone won’t help to mitigate this threat,” he says.
In the more immediate future, it is perhaps the printed and sign-based advertising industry where AR could be most disruptive. Companies can’t place their ads on a wall or billboard without permission, but with AR it is easy to see how advertisers could abuse their free reign. Media theorist Franco Berardi aruges in Precarious Rhapsodies that the rising overload of information in the average person’s daily life has already “produced a saturation of human attention that has reached pathological levels.” Will people even want to engage or be able to process AR-heavy spaces?
HYPER-REALITY from Keiichi Matsuda on Vimeo.
Japanese filmmaker Keiichi Matsuda’s “Hyper-Reality” demonstrated in 2015 how dangerous a dependency on AR could be, with supermarkets devoid of any meaningful physical design, filled instead with blank spaces for AR to overlay and fill. AR is perhaps even more intrusive on the street, bamboozling the unnamed protagonist in the short film with an overload of adverts and information. The city had become subjective; shared experience in public spaces instead eschewed for personalized and augmented perceptions of the environment.
But, with Semiha Ergan’s results applied, AR could in fact lower stress. Some aspects of it in Matsuda’s “Hyper-Reality” are genuinely beneficial—even life-saving, such as the red warning arrows that indicate the path of oncoming vehicles.
Space Popular, an architecture firm in Bangkok, is also exploring the positive possibilities of AR. Run by Lara Lesmes and Fredrik Hellberg, the pair proposed what they call a “multidimensional memorial” for Cubryna Park in Warsaw, Poland. Named Kazymierzowsky Rebound, the project sees a basic steel structure act as a template for AR renditions of former versions of the Kazimierzowski Palace.
The palace dates back to the the 17th century when it was built for King Władysław IV in 1641 and has been significantly transformed five times since. With the steel being allied physically with recovered stone fragments of former palaces, the project, according to Hellberg, “links the incredibly real to the wholly virtual.”
“This is good because perception of what is real and what is not facilitates a better understanding of place,” he says. Hellberg is also concerned with future architects using AR. “Interaction with architecture will soon be dynamic and architects need to be involved in the cognitive process of seeing,” he said. “We will be hiring someone to amplify experiences of spaces through augmentation and most young architects will be designing virtual/augmented enhancements.” Hopkins views this pessimistically, asking rhetorically, “Do architects just become video game designers?”
If we are to take the quantified data from NYU’s research, then AR’s use by architects in some cases merely needs to be algorithmic and—like video games—scripted by computers. “This completely breaks down the architect/client relationship, which is the bedrock of the profession since it became a profession,” Hopkins says.
Like Matsuda and Hopkins, Hellberg is wary of AR’s darker prospects. “Local governments will need regulation for [AR] in the future because it will become intrusive,” Hellberg warns. “We need to create some sort of order. That will be the last phase of this digital revolution.” Errazuriz echoed this sentiment when he told The New York Times last fall that the public should charge Snapchat for its use of “virtual public space.”
Some may argue that Snapchat having AR autonomy isn’t a bad thing. After all, you do choose to use the app. However, there is the possibility of it being bought out, as Koons’s Balloon Dog demonstrated. This doesn’t pertain just to advertising, but privacy as well. Residents in Neo Bankside, across the street from the Switch House viewing platform at the Tate Modern, complain about nosy museum-goers peering into their homes. It’s entirely possible that Snapchat could one day charge those residents to put fake blinds or curtains in their windows when Snapchat users point their lenses at them.