3: Augmented Reality: Enhancing The World Around Us
While reality is far richer than the digital experience, it has significant limitations, as the laws of the physical world are confining and unyielding. The author describes the real-world experience of watching a golf tournament in person:
By necessity the viewer can only be in one place at a time, which makes him unable to observe everything that is going on. He can choose to stay at one hole and watch each of the players at that location, or choose to follow a player from place to place. In either case, he must remain behind the barricades, far away from the action, and seek a position within the throngs of other observers from which he can see the action. The live observer misses a lot: the laws of physical reality make it impossible to watch an entire golf tournament, every shot by every player.
(EN: The author skips over a middle step I think is worth considering: television broadcast. The camera itself offers the viewer a better view of the action by it ability to zoom in to get a closer perspective. Broadcasters use numerous cameras, quickly switching to whatever location offers the most action. Commentators can offer additional information that enhance the appreciation or understanding of the viewer. Record the broadcast on DVR and the viewer can skip about in time. But it is still limited to the editing choices of the broadcaster, who decides what information the viewer will see and offers only a partial experience. It's better, but still not as good as what the author next describes.)
The author describes a technology called "FanVision" that pairs a small viewing screen with an earpiece, which connects to a system that give the user the ability to view from any of a number of cameras that are placed all over the course, along with the leaderboard and player scores. His description of the experience is using the device while attending the event to be able to keep up with what is happening in other places he cannot be, or from angles he cannot observe. "For the first time at any tournament, I did not feel like I was missing out elsewhere on the course."
(EN: The author describes using the device while attending, rather than instead of attending, the live event. But it could be used by a person in another location. In the latter instance, it would clearly be a substitute for reality rather than an augmentation to it. Even in the author's example, it seems the user would be switching between the device and his surroundings, not really augmenting his experience as substituting for it, as he alternates attention between the digital and the real. While he watches the game, he is having a real experience, then he switches attention to the device and has a digital experience. They are still distinct experiences, so I'm not sure if this is a good example of augmented reality.)
Enhancing the World around Us
The author suggests that "augmented reality" stretches one of the dimensions of space, time, and matter. For the FanVision device, he suggests that it allows the person to extend into no-matter, escaping the limitations of his human body to observe the game as an omniscient. (EN: but it also allows him to overcome the confines of a physical space, and likely time as well, so it doesn't quite fit the "one parameter" criterion.)
A further definition of the concept of augmented reality is to overlay reality with digital information intended to extend, enhance, edit, or amend the way we experience the real world.
(EN: This is the definition I understand, and find most meaningful - but it's difficult to provide an example of it, because in most instances, it is not an overlay that coincides with our experience of reality, but a device that requires us to take attention away from reality in order to focus on a distinct and separate experience. It lacks synchronicity and true co-incidence. For "true" augmented reality, consider the example of audio programs that are sold by museums, enabling the user to "hear" a narrative about various pieces on display while viewing the display itself. That better fits my understanding of augmented reality - the user hears the information at the same time he sees the object in a real space. There is no attention switching. Arguably, the person sacrifices one sense - hearing - from his real experience in order to gain the augmentation of a better digital component. It's dreadfully low-tech, but more accurate to the notion of augmentation rather than substitution.)
The author considers GPS "the quintessential example" (sic) of augmented reality - a device that is used while driving (or even walking) to provide a map of location and indication of points of interest nearby. (EN: Likely the most familiar example rather than "quintessential" but a better fit, though again, it's switching - though in this instance, the information the device provides is essential to the task being performed and would make little sense viewed on its own, outside of the present context.)
The author also refers to the experimental technology being developed by smart phones - where the camera on the phone depicts what is visible behind it, much like holding up a pane of glass, but then overlays it with icons so the user can access pop-up dialogs with more information about the things in their field of vision. (EN: This, I think, qualifies more as a "quintessential" example - it captures the quintessence of overlaying reality with data, invisible in the real world, that contribute to the users understanding.)
The author speaks to the simultaneous strength and weakness of such a device: there can be thousands of content layers over a given view of reality. Look through the viewfinder at a building, and the device could inform you of what the primary business is inside of it now (a restaurant), a list of he people who are inside, the name of the person who owns the building, historical events that have occurred within the building, information about its architecture, the amount of electricity and water that are being used, and a multitude of other data that would be of interest to different people in different situations. This information could crowd out reality, unless it were filtered for a specific interest or task.
Location-based Offerings
Mobile computing has created a demand for location-based services, which uses GPS, triangulation, or other techniques to identify the location of the device (and the user carrying it) to be able to present information germane to their specific physical location.
An interesting factor is that location-based services are used to enhance or empower the user to act in the physical world - the location and time in which they presently exist. However, there are a few examples of location-based services that are used strictly for entertainment: a "jewel collector" application that enables people to treasure-hunt (something like geocaching, but the cache is digital rather than physical) or the ability in Foursquare (which is primarily to enable people to tell others where they presently are and get insight from other users who have been there) to compete to become "mayor" of a location.
There's also mention of a service that allows a smart phone to act as a pair of binoculars to zoom in on things that are at a distance. The author specifically mentions the coin-operated binoculars that are used in tourist locations, which help the user view landmarks - a company is simulating that on the smart phone, though which the user can not only view a location, but pull up additional information about it.
Another service in use at museums and galleries enables the user to scan a three-dimensional barcode (QR code) to retrieve extended information about an exhibit, which can be a text article about the item, a 3D model they can view from various angles, a related video presentation, etc.
Retuning to the example of sports, there's a technology that competes with the FanVision unit the author mentioned that enables attendees at stadium events to connect to a local network, not only to access different cameras to observe the event from various angles, but to get access to instant replays, team and player statistics, and other information that will enhance their experience of watching the event.
Beyond Fun and Games
Most examples of augmented reality focus on leisure activities - sporting events, tourist destinations, and other activities that, while engaging, seem to lack a certain gravity - but augmented reality can also be used for "serious business."
For example, "Vein Vision" uses an infrared camera to image the location of a patients veins and superimpose them on the skin - which enables a clinician to find the vein on the first stick rather than having to guess, based on their limited visual acuity and knowledge of anatomy, on where the vein might be. Technology is also underway to help surgeons better visualize the location of their tools inside the patients body when doing laparoscopic procedures, improving their outcomes and reducing the chance of serious mistakes. In this sense, augmented reality amends reality by revealing to us things that we cannot see with the naked eye.
Another example is in industrial operations, which are being developed to facilitate assembly-line work by showing workers a diagram of the completed task with instructions for where and how a given part should be attached, allowing someone to be instantly competent at a task that would otherwise take time to learn, and avoid mistakes due to unfamiliarity.
Another example is heads-up displays for people operating vehicles. Military and commercial pilots "have long used" heads-up displays - while the instrument panel is a clutter of gauges, the heads-up display provides access to those that are most relevant at a given time. The same technology is being applied to vehicles, with heads-us displays, proximity detectors, rear-view cameras, and the like.
This is being combined with GPS data to provide a "smart windshield" that shows the user more than merely a view of the road ahead - along with an "augmented reality" view that highlights things such as road signs (a stop sign might be highlighted, or a speed limit sign if the vehicle is exceeding the posted speed). Things that used to be wondrous and futuristic are presently not very far from becoming available to the consumer.
Sensory Prosthetics
One function of technology is to improve our sensory perception. Spectacles give a person the ability to see things better, overcoming their poor vision. A telescope gives a person with normal vision the ability to see things at a great distance, overcoming the limitations of what is considered to be normal or good vision. But technology goes further: it enables us to see into the infrared or ultraviolet spectrums, which doesn't merely correct or enhance what we normally consider to be sight, but gives us a prosthetic eye that gives us perception we do not have by virtue of nature.
(EN: The author gives a flurry of examples, some of which have to do with the present topic, others of which seem either oblique or unrelated, so I'll condense a great deal.)
Various systems can convert between the spoken word and written text, enabling the blind (or illiterate) to hear what is written or the deaf to read what is said aloud.
He also refers to some progress in the area of touch - off-topic, but interesting nonetheless: an experimental technology at the University of Tokyo that is attempting to create touchable holograms that uses ultrasound to simulate tactile sensations, and another at a Montreal college where researchers have developed floor tiles that can simulate the feel of various surfaces (snow, grass, or pebbles underfoot).
Another digression into technologies that are attempting to simulate scent, though admittedly, companies that were devoted to this purpose made little practice: DigiScents went bankrupt trying to develop such a device, NTT experimented with the notion but defunded the project, and even Proctor and Gamble pursued, then abandoned, a technology that "played" discs of scents. He expects this is not the end of it, as scent is highly effective in evoking memory and is a clear gap in the ability to create a convincing virtual environment.
These various technologies are geared to provide to the user sensory capabilities that extend beyond what he is capable of experiencing in reality, giving the limitations of his physical organs. This can be done to restore a sense that is diminished or missing in a disabled person, or to extend the senses of a normally-abled person.
Applying Augmented Reality
On a practical level, a developer who is considering augmented reality must consider what senses that a user would wish to have enhanced, and do so without "perturbing the rest" and consider whether the consumer will bear the inconveniences of technology for the sake of the value it provides. May people would say that they want to improve or enhance their senses, but if it requires medical procedures and implanting devices into their bodies, few would be willing to take that step.
A bulleted list of principles follows:
- Augmented reality is a natural fit for a business with a physical space such as a store, office, or other physical location. Consider what information or experience can be "overlaid" atop the environment or elements within it.
- Consider the degree to which technology overtakes the real environment: whether the digital enhancement is so important (to the customer) that it should become their primary focus
- Where the augmentation is far more important than the real, consider completely virtualizing the experience (or parts of it)
- Leverage the devices people already have - such as smart phones - to deliver your augmented experience (EN: Given that smart phones are not ubiquitous as technology enthusiasts like to think, this should not yet be the only method of delivery.)
- View the augmentation device as a prosthetic. Consider what senses it should be used to enhance.
- Augmented reality works well for tracking - giving the user the ability to locate an item, place, or person that they cannot see. Tracking a package in the possession of a delivery service, locating an item in a store, finding a person, etc.
- Augmented reality works well for education, whether your product is education itself, or where some bit of knowledge would improve the user's ability to get value from a given product or service.
- The notion of education can also be applied to small amounts of knowledge delivered as needed and in real time (a "tip" to the user) as well as to conventional learning (memorizing information for future use)
- Augmented reality can project a vision of a reality that does not exist, but could, showing the user what could be as a result in the change that might be made. (EN: Consider before-and-after comparisons used by cosmetic surgeons)
- Consider whether your user will consider augmented reality to be an intrusion and a nuisance rather than a value - technology can be annoying and distracting.
- Finally, keep in mind that augmented reality is not about the cool things that technology is capable of doing, but about the way in which it provides actual value to the user.