9: Mirrored Virtuality: Absorbing The Real World Into The Virtual
The opening example considers a site that is used to track baseball games. It started out as little more than a live feed of the game's score that updated as the game progressed, but later evolved to the point where it provides a digital representation of the actual game: showing runners on base, the players in the field, even the speed and arc of each pitch. None of this involves live cameras: it's completely modeled based on data.
The author's concept of "mirrored virtuality" involves a representation of the real world via a digital device, particularly experiences that are time-locked to phenomena that are occurring in the real world. (EN: my sense is that this is next of kin to a television camera, or perhaps a sonar system, though the example he uses above takes more sophistication to present less data from reality, substituted by more artificial data to represent the detail that is lost.) He also distinguishes mirrored virtuality from full virtuality in its being tied to physical objects and time in the material world rather than a realm of complete (or even partial) fantasy.
The idea of creating a digital representation of a real space alone does not qualify, nor does a realistic model of a thing with which the user can interact in imaginary ways. Mirrored virtuality is tied to the realm of the real - what is actually happening, when it is actually happening.
Second Earth
A few examples are provided: air traffic controllers use 2D and 3D representation of the position of planes in flight, using a combination of radar and other data to show the precise position of each aircraft in real time. Another firm is working on the same idea for crowd control - creating a digital representation that shows the positions of individuals in a large crowd, as a means to pinpoint troublemakers in a riot situation and distinguish them from bystanders.
Other MV applications could be sties that show traffic on a map (while it doesn't show individual vehicles, it portrays the roads and a general sense of their number) or satellite weather maps that use colored regions to depict precipitation in near-real time. Even a stock-market ticker, which represents deals struck in real time, approaches MV.
A "second earth" concept is also considered: using a mapping system to create a digital representation of a city, along with street cameras that will provide data to represent every object (people and vehicles, primarily) on the streets and sidewalks. It's not a 2D video, but a 3D model that strips away extraneous details to enable analysts to focus on what is important to them.
Predicting the Present
Strictly speaking, there is no present technology that reproduces events in real time, but instead shows users a picture of the recent past because it takes time to gather and model the details. Weather and stock reports may update every ten minutes, and even a video camera takes some amount of time to transmit signal from camera to screen - it may be milliseconds, but it is still a delay.
In an attempt to bring virtuality closer to real time, some experimentation is being done with predictive algorithms that are generally accurate. If we know the rate of speed and direction a vehicle was moving, we can project where it will be presently.
Another example is computer modeling of epidemics, such as "Google Flu Trends" which has been even more accurate than the predictions of organizations such as the CDC. By tapping into a feed of real-time warnings, the system can fairly accurately predict the spread of contagion through the human population and project this data on maps.
A key factor is that these technologies predict the very near future - the further into the future you extend a prediction, the less accurate it will be. Returning to vehicular data, we can be highly accurate in projecting where a vehicle will be in two seconds based on its present velocity and direction; but the prediction of where it will be in two minutes, two hours, or two days is entirely inaccurate.
Real-time Web
Social media such as Twitter, where a large volume of posts occur my large numbers of users, provide a massive amount of data that can be analyzed in near-real time. The site trendsmap.com does this, superimposing the most common words used in twitter posts in various geographical areas.
There is much interest in improving the immediacy of the Web - data that is fresh or new has always been of greater interest than historical or evergreen data, and being more up-to-date is a competitive advantage that the providers of sites and services seek to secure, so it's likely the amount of granular and momentary information available will increase, providing a fire-hose of data to draw upon. As such, the present problem isn't getting data, it's filtering the data to provide meaningful information.
The Quantified Self
Mirrored virtuality reflects the culture of quantification: in the present day, numbers are given a level of reverence that borders on religious faith, and the phenomenon is pervasive. An increasing number of people are involved in obsessively collecting data about themselves, and various sources have attempted to coin a term for the phenomenon: life logging, life tracking, life streaming, etc.
(EN: The author takes this as a good thing, but I find it a bit disturbing, especially given the example of health care: a person who is suffering neither discomfort nor disability can be conned into paying for treatment he likely doesn't need simply because his "numbers don't look good." In that same vein, I've observed quite a few people who are obsessive about numbers pertaining to their health, and they inflict considerable misery upon themselves daily in order to influence the data.)
A few examples are provided:
- Financial services site mint.com aggregates all financial data about a person, down to every transaction, to provide them an analysis of their personal finances to guide their economic choices
- Nike manufactures a shoe that, when linked to a PDA, will give detailed information about the workouts of a wearer: speed, mileage, time, and other factors. Another firm uses a pedometer to track how much you walk in your everyday life, and another tracks the hours you sleep.
- A myriad of wearable devices are used to measure basic statistics: pulse, blood pressure, blood sugar, respiration, etc. and e-mail this information to physicians to constantly monitor patients.
In the past, the only method for measuring such things was to compulsively take notes about every facet of our lives. A person obsessed with their weight would step on a scale each morning and write it down; a person obsessed with finances would keep a ledger of every transaction; a person obsessed with fitness would keep a notebook of their exercise activities; etc.
Computer technology enabled us to input all of this data into a spreadsheet and run charts and graphs to consider our status and monitor our progress. Sensor devices even automate the task of collecting the data. However, the basic activity of measurement and analysis is the same as it ever was.
The point to all of this obsessive behavior is not merely collecting information, but using this information as a means to make changes, to transform rather than merely observe. However, the author concedes that while many firms are catering to compulsion, "few if any of these goods, services, and experiences actually make possible the transformations we desire."
Topsight & Technology
The notion of wearable computing originally involved constantly making the processing power of a computer available to a person, anywhere he happens to be; but with the increased memory capacity and array of sensors, it has in some cases been about tracking a person throughout their daily lives. In this sense, we are digitizing our entire life experience, collecting data and storing it in digital memory. At the present time, we can collect a lot of useless data - or at least, data that seems useless until we can find a way of deriving meaning from it.
The human mind is much the same: we collect a great deal of sense data, and very seldom recognize any pattern to it unless it is dramatic enough (in frequency or impact) to come to our attention. There is much we observe, but do not notice, due to the imperfection of our senses and our memories; and computer technology can overcome both: it remembers everything perfectly and is capable of more complex and subtle analysis.
The author borrows the term "topsight" to describe this omniscient view of a person's life - not merely a big picture that makes assumptions about the details, but a big picture that is composed with omniscience of all the tiny details.
With the right software, you will be able to mine a digital memory archives to identify patterns you might not notice on your own: all the information you perceive can be remembered and stored in a data warehouse. This enables you to remember every person you ever met, ever route you ever walked, every thing you ever consumed, every television show you ever watched, etc. and then perform analysis on all of these fragments to come to a meaningful and insightful analysis.
(EN: "With the right X" is a phrase that pops out at me as being the equivalent of "if you had a magic wand." Though I'll admit it's more plausible that someone, somewhere, someday will invent a technology that will do a given thing, it still seems a bit specious. It's a bit more troubling with technology because software is written by someone - the programmers of software are very much like wizards who gain control over other people by means of their software, and their victims seldom pause to consider that it's not the wisdom of the computer to which they are sacrificing their independence, but the less trustable intelligence and less objective goals of the software designer.)
Considering time: all of these real-time observations become historical data: mirrored virtuality is tied to actual events in actual time; but it is also involved in projecting the future and modeling what will become if the trends it observes are continued, and by doing so suggests an alternate outcome that could be achieved if the behavior related to the data collected is altered.
The elements necessary to the task are:
- Sensors that collect data in real time
- Storage for the collected data so it can later be accessed
- Arrangement of the data in a meaningful format
- Analytics that explain what the data means
- Forecasting and alerts to make the user aware
The notion of the dashboard has become popular of late, a hackneyed metaphor that is useful in some instances (EN: Though far fewer than instances in which it is actually used) to get a sense of performance, moment by moment, in real time.
Applying Mirrored Virtuality
Ideas and tips:
- Mirrored virtuality's greatest power is in its ability to call attention to the things that are unknown - and this includes not only things that are invisible, but also things that are entirely visible, but not noticed or recorded
- MV begins by creating a virtual model of what is historical and real, then uses that model to project what might happen in an imaginary future, and how that future can be changed by altering present behavior.
- MV is most useful when it provides actionable information, not just data for the sake of having data. Collecting information in case it might be useful in future, or in case the user can figure out what it means on his own, has limited appeal.
- In terms of time, MV can help us understand the past, predict the near future (the present), and forecast the far future.
- A lot of historical data is already available via the Internet - both distant history (years or centuries ago) as well as recent history (minutes or seconds ago)
- People are obsessed with numbers, and pandering to this obsession is profitable.
- The private is becoming increasingly public. Older notions about what data a person is willing to post about himself for others to view are overly conservative.
- You may find a way to measured what presently is not measured, or apply different logic to existing measurements, to provide new insight.
- There is good opportunity is smaller and less glamorous roles: it's not necessary to develop a full consimer application, but a component that someone else will use in their application - e.g., you might focus on developing a sensor, and let others figure out how to use the data and incorporate it into a consumer device.