Reality+ : Virtual Worlds and the Problems of Philosophy (9780393635812), page 25
Pokémon Go does not require fancy headsets. The augmentation is all done by a smartphone, with virtual creatures inserted into an on-screen video image from a camera. More sophisticated augmented reality technology involves glasses with the ability to project images into the user’s field of view. This technology yields an especially immersive form of augmented reality. So far, the glasses have been unwieldy, but they’re getting smaller and more powerful. Augmented reality contact lenses are on the horizon.
Within a decade or two, we may all use augmented reality. It could eliminate the need for screens in desktop and mobile computers by projecting a screen or another interface in the space in front of you. It might eventually replace street signs and traffic signals with digital counterparts. It could enable communication with faraway friends as if they’re in the same space with you. It could navigate for you using built-in maps, recognize people for you using automated face recognition, and translate foreign speech for you using language-translation algorithms. It could bring historical locations to life by augmenting them with scenes from the past.
Figure 31 Augmented reality glasses, augmenting the ruins of Plato’s Academy in Athens with images of Plato and Aristotle in the Academy (taken from Raphael’s The School of Athens).
Augmented reality promises to augment our surroundings and our minds simultaneously. As I’ll discuss in chapter 16, it augments our minds by extending our brains with new capabilities for navigation, recognition, and communication that we did not have before. This chapter focuses on the way it augments the physical world.
We can ask the Reality Question about augmented reality: Is augmented reality real? Are the Pokémon Go creatures real, for example? Unsurprisingly, my answer is, for the most part, yes. They have causal powers, and they exist independently of our minds. They may not be real creatures, but they’re real virtual objects, existing as digital objects in a computer and made visible through an augmented reality system.
Another key aspect of the Reality Question: Is augmented reality an illusion? That is, is it the way it seems? This is a harder question. With augmented reality, virtual objects seem to be in the physical space around us, so it’s harder to say (as I did for virtual reality) that they seem to inhabit only a virtual space. There’s also a stronger case that augmented reality involves an illusion—the illusion that virtual objects are present in physical space.
To get to the bottom of this, we need to address the question “Are virtual objects in augmented reality present in physical space?” The answer might seem obviously no, but things aren’t that clear. There may be at least some sense in which Pokémon Go creatures can be said to exist in the physical space around us.
Virtual objects in augmented reality
Suppose that sometime in the future, everyone uses the same augmented reality system; let’s call it Earth+. Earth+ augments the physical environment all over Earth with virtual objects for everybody. The system is surgically implanted, and everyone has it. At certain locations in physical space, everyone sees the same virtual objects: virtual helpers, virtual furniture, virtual buildings.
Users of Earth+ don’t just see and hear the virtual objects in front of them. Thanks to brain-stimulation technology, you can smell and taste virtual objects. You have the sense of eating virtual food and drinking virtual drinks. Thanks to special haptic technology, you can touch and feel virtual objects. You can pick up a virtual rock and feel its weight. Thanks to special bodysuits, you can sit in a virtual chair and encounter resistance when you run into a virtual wall.
Users will typically know whether an object they’re interacting with is virtual or physical—it’s usually obvious. Virtual objects look different from ordinary objects. They typically have special features; for instance, virtual chairs can automatically change size or shape or comfort level. Virtual food stays fresh forever.
Figure 32 A virtual piano in Washington Square Park.
Let’s say there’s a virtual piano in Washington Square Park. You sit and play, and everyone hears the music.
Now, we can ask: Is the virtual piano real? It hits many of the marks on the reality checklist. It has causal powers: It plays music, you can’t walk through it. It is mind-independent: Even if all users have left for the day, it remains in Washington Square Park unless someone chooses to move it.
Is it a real piano? This is tricky. Even in the nonvirtual world, we have digital pianos and other electronic pianos. Are they real pianos? A few might say yes. It’s increasingly common to talk of “acoustic pianos” and “digital pianos” as if both are pianos, though of different types. Still, most people would say that digital pianos aren’t real pianos. A real piano must have strings that vibrate when struck by a hammer, along with other acoustic mechanisms. If digital pianos aren’t real pianos, virtual pianos are probably not real pianos, either. Similarly, we’d probably say that virtual trees in Earth+ aren’t real trees. On the other hand, virtual books are arguably real books. As with virtual reality, some virtual X’s are real X’s in Earth+, and some aren’t.
Finally, the big issue: Is the virtual piano the way it seems? Or does it involve an illusion? Perhaps the biggest challenge concerns space. The virtual piano appears to be in physical space. It seems to be about a meter tall, piano-shaped, and situated in the middle of Washington Square Park. Is it really in that place? Or is there nothing there but air, in physical reality?
Why say the virtual piano is not really in Washington Square Park? One reason is that if someone came along without the Earth+ system, they’d see nothing there. Perhaps it’s already invisible to birds and squirrels. If Martians land in Washington Square Park, they won’t see the virtual piano. And if a group of mavericks never get the Earth+ system installed, the piano won’t be there for them. You might think a virtual piano in Earth+ is like a rainbow: It seems to be there for some people, but it isn’t there in reality.
On the other hand, why say the virtual piano is really in Washington Square Park? Certainly users will talk that way. It will be second nature to treat virtual objects as real objects (“Have you sat on the wonderful virtual sofa at Rockefeller Center?”), even if users distinguish them from physical objects. Furthermore, the virtual piano behaves as if it’s in the park. It looks, feels, and functions like a virtual piano in the park.
A natural resolution of this matter is to make a distinction. The virtual piano is not physically in the park. But it is virtually in the park. An object is physically in a space if it has physical matter that occupies that space. An object is virtually in a space if it functions as if it occupies that space. The virtual piano has no physical matter in Washington Square Park, but it functions as if it occupies the space.
If the virtual piano seems to be physically in Washington Square Park, this is an illusion. The virtual piano is only virtually in the park, not physically in the park. On the other hand, if the virtual piano seems only to be virtually in Washington Square Park, this isn’t an illusion. The virtual piano really is virtually in Washington Square Park.
In the previous chapter, I argued that sophisticated users of VR may see virtual chairs as virtual rather than as physical, and that they may see them as situated in virtual spaces such as Second Life rather than in physical space. If so, VR is not illusory. In Earth+, a sophisticated user may also see a virtual piano as virtual rather than physical. Now, armed with our distinction, we can go a step further. A sophisticated user may see a virtual piano as being virtually in its location in Washington Square Park rather than being physically in that location. If so, there’s no illusion.
I don’t think this conclusion is obvious. It may be hard for users to avoid the sense that the virtual piano is physically in the location in the same way a physical piano could be. Still, a generation raised on Earth+ would presumably learn to treat virtual objects and physical objects in very different ways, and this automatic interpretation would affect their perceptions. The distinction between physical location and virtual location could eventually become second nature; if it does, perception in Earth+ need not involve an illusion.
I conclude that the virtual piano in Earth+ is a real object. In addition, there’s a reasonable case that it’s a real piano and that it’s not illusory. If so, this augmented reality is genuine reality.
From augmented reality to alternative facts?
It’s easy to imagine that in the future, there will be multiple dominant systems of augmented reality. Instead of a single universal reality, there will be Apple Reality, Facebook Reality, and Google Reality. Each corporation will set up its own virtual worlds and augment them with its own virtual objects.
In Facebook Reality, there may be a virtual piano at a certain location in Washington Square Park. In Apple Reality, there may be a virtual sign at the same location. In Google Reality, there may be nothing there at all.
Is the virtual piano in the park? According to Facebook Reality, the virtual piano is in the park. According to Apple Reality and Google Reality, it is not. Which reality is correct? It’s hard to believe that one of the three is objective reality and the others aren’t. Instead, the situation seems symmetrical: Relative to Facebook Reality, the virtual piano is in the park. Relative to Apple Reality and Google Reality, it isn’t. We have three different equally valid systems of reality. Has objective reality gone out the window?
Here we seem to have a sort of relativism. Whether a fact (the virtual piano is in the park) is indeed a fact depends on the system one’s using. You might say that now we have alternative facts.
Alternative facts gained notoriety after President Donald Trump’s inauguration in January 2017, when there was a dispute about the size of the crowd. The White House press secretary Sean Spicer said that many more people rode the DC Metro that day than on the day of Barack Obama’s second inauguration in January 2013. Records show that in fact ridership was much higher in 2013. In an interview, reporter Chuck Todd asked Kellyanne Conway, counselor to the president, why Spicer would utter a “provable falsehood.” Conway responded by saying that Spicer had stated “alternative facts.”
Conway was criticized for her remark. Todd responded that “alternative facts are not facts. They are falsehoods.” Many saw Conway’s statement as suggesting a sort of relativism about truth. Relativism is, roughly, the idea that there are multiple equally valid sets of facts corresponding to different viewpoints. Facts are facts only relative to a viewpoint. Relative to Todd’s viewpoint, ridership was higher in 2013. Relative to the White House’s viewpoint, ridership was higher in 2017.
Relativism is a deeply controversial idea. There are some varieties of relativism that are widely accepted. For example, most of us are relativists about etiquette; what counts as a polite thing to do varies from society to society. A traditional American custom considers it polite to eat meat and vegetables with just a fork after first cutting them. Europeans hold that one should eat them with both knife and fork. Some British and Australian customs even require balancing vegetables on the back of one’s fork. There are no objective facts about which of these customs is right; there are just relative facts about who’s right from each viewpoint. But while relativism about etiquette seems acceptable, many people resist relativism about more concrete matters, such as the laws of physics.
In my view, augmented and virtual reality lead to a sort of relativism, just as differences in etiquette do. But both are harmless forms of relativism that don’t threaten the idea of objective reality.
Much of what we once thought of as absolute has turned out to be relative. One might have thought the time of day was absolute. To our ancestors, it was an objective fact that a certain lunar eclipse happened in the morning. Now we know that the time of day is relative: When it’s morning in Sydney, it’s evening in New York City. One might have thought the strength of gravity is objective. Now we know it’s relative; gravity is much stronger on Earth than it is on the Moon. Correspondingly, weight is relative, too. I weigh much more on Earth than I would on the Moon. One might have thought that shape, mass, and time are objective. According to the special theory of relativity, however, they’re all relative to a reference frame. In a reference frame where an object is moving close to the speed of light, its shape compresses, its mass increases, and time slows down, at least relative to an object that is at rest in that reference frame.
Still, all of these matters are consistent with an underlying level of objective reality. For example, it is an objective fact that it’s 1:00 pm in New York City. It is also an objective fact that a certain rock weighs six pounds on Earth and one pound on the Moon.
How can we reconcile this sort of relativism with objective reality? It’s easy. We simply need to allow that relations are part of reality. It’s 9:00 am in relation to New York and 2:00 pm in London. The object is circular in relation to one reference frame and elliptical in another. You need relations like these to give a full description of reality.
The same goes for multiple realities. You might have thought that whether or not there’s a piano in Washington Square Park is an objective matter. It turns out to be a relational matter. In Apple Reality, there’s a piano in the park. In Google Reality, there isn’t. The same goes for laws of physics: It’s an objective fact that in Apple Reality, the laws are one way. It’s an objective fact that in Facebook Reality, they’re another way.
Importantly, both the Apple Reality system and the Google Reality system are part of objective reality. It’s an objective fact that in Apple Reality there’s a piano in the park. And it’s an objective fact that in Google Reality the laws of quantum mechanics are true. That’s how we reconcile the relativism of virtual worlds with objective reality.
An extreme relativist may say that there’s no level of objective reality at all. Even relational facts—such as the fact that this reality contains a piano, or that I find Bob Dylan’s music beautiful—might be true from my perspective and false from yours. A non-relativist will say there are objective facts about what’s true from my perspective and about what’s true from yours. But the relativist will say that even facts about what’s true from my perspective are only true from a particular perspective! That’s an interesting view, but there’s no good reason to accept it.
Even in a cosmos with multiple virtual and augmented reality systems, there are plenty of objective facts. For a start, there are objective facts about what happens in each reality system. It will be an objective fact that the piano is in the park in Apple Reality.
There are also objective facts about ground-level reality. Nothing in our discussion of virtual worlds within virtual worlds suggests that there is not a base reality at the top of the chain. Even if our own reality is a simulation 42 levels down from base reality, base reality has its own independent existence.
Importantly, there are objective facts about what happens in our ordinary reality. It is an objective fact that a certain number of votes were cast in the 2020 US presidential election. It’s an objective fact that Joe Biden was declared the winner. Of course many facts about ordinary reality will be relativized to time, place, and more. Joe Biden was elected president of the US in 2020 but not in 2016.
The same holds true for virtual worlds. Biden was elected in ordinary reality, while perhaps someone else was elected in Meta Reality, the universe of our simulators, and someone else was elected in Second Life, a virtual world we have constructed. But once we realize that reality is relational in this way, what happened in the United States in 2020 in ordinary reality remains an objective fact.
There can be disagreements about these facts. Trump supporters can think that Trump got more votes (in this reality), while Biden supporters think that Biden got more votes. But there’s an objective fact about which is right. With some luck, we may even be able to find it.
In the Black Mirror episode “Men Against Fire”, soldiers use an augmented reality implant called MASS that makes human mutants appear to be roaches. This system enables a form of genocide wherein the mutants are wiped out by the soldiers. What do we say about reality here? If MASS Reality removes any trace of the mutants in a farmhouse and shows virtual roaches instead, we might say that in MASS Reality, the farmhouse contains virtual roaches and not humans. However, in ordinary reality, the farmhouse still contains humans. A soldier who kills apparent roaches in MASS Reality is destroying virtual roaches there but is also killing humans in ordinary reality. Even if everyone in the world had the MASS implant, humans would be dying. The relativism of multiple realities does not yield an escape from cold, hard facts about ordinary reality.
Near-term augmented reality
For now, Earth+ is science fiction. Existing augmented reality systems are much more mundane. Augmented reality systems mainly allow us to see and perhaps to hear virtual objects, without allowing us to touch, smell, or taste them. Virtual objects don’t impede our movements. No augmented reality system is installed permanently. Users put on their glasses occasionally but more often have them off. And no system is universal. There are a few different systems, each of which is used for short periods by a few users.
Still, some of what we’ve said about Earth+ applies to near-term augmented reality. Suppose I’m redesigning my living room with augmented reality, so as to see a virtual sofa in the corner of the room. Is the sofa real?
As noted, the virtual sofa is a real digital object existing inside the augmented reality device’s computer. It has genuine causal powers—at the very least, it causes you to see it, and it might even cause you to buy a sofa. To a limited extent, it exists independently of our minds. If I take my glasses off but leave the program running, the digital object still exists and is still available to be seen by others, in principle. So it satisfies our first three criteria for reality—existence, causal powers, mind-independence—at least to some extent. But it fails the fifth criterion: It clearly is not a real sofa. I cannot even sit on it.
