I have no trouble believing that young Bill Clinton smoked marijuana and didn't inhale. I'm the President's age, and I remember one party back in the '60s when...ahem.
Confession may be good for the soul, but confession of innocence can be bad for the image. Let's put it this way: I admit to about as much as the President admits to.
Why do I burden you with this dubious confession? As a disclaimer. This month's column takes as its central metaphor a theme on which I, let us say, am not an expert. Drugs.
I'm going to talk about virtual reality, which, I contend, is a psychedelic technology. What I mean by that is that I suspect that some of the interest in this technology comes from the same motives that cause people to use psychedelic drugs. It's not just my opinion; a lot of people who are intrigued by virtual reality draw the parallel. On the other hand, Steve Aukstakalnis and David Blatner, authors of Silicon Mirage (Peachpit Press, 1992), an excellent book on virtual reality (VR), caution that, whatever people's motives, their expectations will be disappointed if they think VR is a drug. You can always pull the plug, they say.
But I'm getting ahead of myself. First we need a definition, popular though this term may be. Virtual reality means "a computer-generated, interactive, three-dimensional environment in which a person is immersed." VPL founder Jaron Lanier coined the term in 1989.
I'm going to look at how real the technology is, what has been accomplished to date, what the practical applications are, and what technical problems remain unsolved. And is there some reality to these alternate realities, or will those expecting to be transported to some other place necessarily be disappointed?
Virtual-reality systems are interactive, which means that they need to respond to what the user does. So does a spreadsheet, but VR is usually charged to respond to things like head orientation, hand movement in three dimensions, and sometimes body movement or orientation. When you move your head in a VR system, the view is supposed to change, and change realistically. Not only that, the system may be required to provide feedback in other modalities, like sound and tactile sensations. It's not necessary that the virtual reality delivered by such a system feel exactly like "real" reality--virtual-reality systems typically don't worry too much about keeping users from walking through walls, for example--but it must be responsive and internally consistent, or it just won't work. When you consider what it takes to make this kind of feedback consistent and to make it responsive to subtle movements of the user, you can see that VR isn't easy.
Before looking at some of the tools and the successes of VR, let's look a little more deeply at the challenges.
You can always tell when someone is using a virtual-reality system. The goggles are a dead giveaway.
Virtual-reality systems may or may not present their users with auditory or tactile feedback, but they invariably give them a picture. That picture is by definition an explorable 3-D space. The user gets the visual impression of, for example, moving down virtual hallways, around rooms, and sometimes, unsettlingly, right through furniture and other seemingly solid objects. As you move your head or your hand or your entire body, you walk or fly through virtual spaces.
Implementing a system like this requires more than just presenting a succession of 3-D images. Goggles, or some such headgear, are a requirement because the sense of immersion that virtual reality tries to give demands that the image wrap around the user. The monitor that you probably spend too much of your waking hours staring at cuts maybe a 30 degree wedge out of the center of your field of view. A good virtual-reality system will wrap its picture around you to the full extent of your visual field, perhaps 240 degrees horizontally and 120 degrees vertically.
This wraparound view is crucial to giving the sense of immersion that virtual reality demands. Two other things should be obvious about it, too. It demands some sort of wraparound hardware; hence the goggles, helmets, and other headgear. And it makes big demands on processing power. Of course, the virtual-reality designer wants those 3-D images to move at a reasonably fast frame rate and with decent resolution (and color would be nice, and realistic color would be nicer). Now enlarge the field of view by a factor of 32 without lowering the resolution. Just in terms of bits in the image, the large field of view required by virtual reality can be as demanding as realistic color vs. black-and-white.
The human visual system, interestingly enough, has a technique for dealing with this problem. Your visual system keeps in focus only what's directly in your line of sight; everything peripheral is out of focus, fuzzy. Something similar has been proposed for virtual-reality systems: The idea is to display most of the picture in low resolution, using higher resolution for what's directly in the center of the user's field of view.
Another reason for the goggles is to provide 3-D. We speak of 3-D modeling programs, but what these programs really do is represent 3-D objects internally and let us have whatever 2-D views of the objects we like. To really see three dimensions, our two eyes need to get different information, and your monitor screen can't provide different pictures for your two eyes. To deliver the two separate images that our visual systems integrate to produce a sense of depth, it's necessary somehow to channel the images to the individual eyes using techniques like those described in Duvanenko and Robbins's article, "Algorithms for Stereoscopic Imaging" on page 18 of this issue. Hence, again, the goggles. 3-D movies and comic books use differently colored lenses, but this doesn't work if you want to use colored images.
Hearing might seem simpler to virtualize than vision, until you start to think about the complexities of concert-hall acoustics. Objects cast acoustic shadows, but they don't block the sound in the simple way in which objects block light. (Virtual-reality systems are not generally charged with having to model the relativistic bending of light around masses, or the wave nature of light.) In fact, obstructions can alter the character of a sound, adding overtones and echoes. As anyone who remembers quadraphonic sound systems knows, the experts are just now starting to understand what's involved in reproducing sound with a sense of realism.
Virtual-reality systems don't all provide sound feedback, but stereo sound is the basic minimum for those that do. One of the familiar sound phenomena that just doesn't happen without stereo sound is the cocktail-party effect. Standing in the middle of a living room full of people engaged in a dozen different conversations, you can pick a conversation and listen to it, ignoring the others. (An automatic version of this attention phenomenon is the name effect: If anyone anywhere in the room uses your name, you'll probably hear it, whether you were paying attention to their conversation or not.) The cocktail-party effect doesn't happen without stereo hearing; plug one ear and you can't pick a conversation out of the general hubbub.
So you need stereo. But stereo isn't enough; virtual-reality systems that support sound really need highly responsive 3-D sound. The reason is that, if the sound isn't real enough, it will provide cues that are just plain wrong, and conflict with what the visual feedback is saying. We use sound a lot, mostly unconsciously, for orienting ourselves. All of the world around us that isn't in our field of view is ear space. We have some faint consciousness of what's behind us, and where, and it's our ears that give us the data. A virtual-reality system with no sound can work, but one with inadequate sound will at some point break the illusion, providing feedback that seems just plain wrong.
Not breaking the illusion is more than just a matter of providing a seamless interface. In some VR applications, bad sound feedback or a mismatch between sound and visual feedback can have an unsettling effect: nausea. Bad data can make you sick.
Touch is really two sensory systems: mechanoreceptors, the nerves in the skin that respond to contact; and proprioception, which is feedback from our muscles. Although the two systems are sometimes hard to distinguish, it's worth doing so, because they are not equally virtualizable.
Mechanoreception is really hard to virtualize. Virtual-reality systems don't try, generally, and it probably isn't important that they do, although it might be nice to be able to slide your hand across the virtual hardwood floor in the virtual house to see how smooth it is.
Proprioception is different. Robots have been responding to proprioceptive feedback for decades. Proprioceptive feedback is when your hand or a robot vacuum cleaner or a virtual hand encounters resistance. In the case of the hand or the robot, it's real resistance. If you're using a VR system to pull molecules apart with your fingers, it's all artificial, but what you feel should still match what you see. That feel must be constructed and fed to you somehow.
There seem to be some basic limitations to the possible realism of proprioceptive feedback. Even if a VR system could somehow make the visitor to a virtual office feel the solidity of the office walls, how could it stop the user from walking through the wall anyway? You can make a VR system look real, sound real, and even to some extent feel real, but you can't make it be real; you can't make it be composed of solid objects. That's where the illusion breaks down in every VR system I know of today. You can test if it's real by poking it.
So what's the reality of virtual reality?
The tools of VR have improved a lot since Ivan Sutherland made the first head-mounted display back in 1968. It didn't block your "real" vision, but superimposed virtual wire-frame models on reality, and was familiarly known as the "sword of Damocles" because it hung from the ceiling on a heavy arm that provided the mechanics for tracking head position.
Today, the latest versions of the VPL eyephone spread a 422x238 or 720x480 pixel display across a 108-degree field of view, weigh only a couple of pounds, and use magnetic position sensors and fresnel lenses to spread the picture across the field of view. The ARVIS from Concept Vision Systems (Conway, Washington) uses contact lenses and curved screens and fills a 240x120 field of view. The BOOM from Fake Space Labs (Menlo Park, California) frees the user from the helmet; you just rest your head on it when you want to dive into its virtual realities. It uses CRTs to give higher resolution than the other systems mentioned. The Sutherland approach, overlaying the virtual reality transparently onto reality, can be seen in the Terminator movies when Arnold looks at people and gets data about them superimposed on their faces. That display technology exists today, although the database it implies does not.
Aukstakalnis and Blatner say that 3-D interactive sound is farther along than 3-D interactive visuals, but in no area are all the problems solved, even in expensive NASA systems.
The input devices are more impressive to look at. VPL's dataglove uses fiber-optic cables running along the fingers and deduces hand movements from the quantity of light traversing the cables. Light escapes through cuts in the cable as fingers are bent. It is one standard for capturing hand movements; the Data Suit generalizes the concept. The Dextrous Hand Master from Exos (Burlington, Massachusetts) is a higher-precision device that wraps an exoskeleton of mechanical sensors around the hand; ugly, but precise. A less imposing but important tool is the ubiquitous Polhemus tracker, developed by Polhemus Navigation Sciences (Colchester, Vermont), which tells the system where your head is and how it's oriented. It uses two sets of three magnetic coils, which define the 6 degrees of freedom of 3-space motion: positions on the x, y, and z axes as well as roll, pitch, and yaw. Alternatives to magnetic orientation include optical systems (you wear cameras on your head) and image extraction. Image extraction is the "right" answer; remote cameras observe you and software figures out what you're doing. It's also the hardest.
Virtuality has some success stories to tell. There's Sitterson Hall at the University of North Carolina, which now houses the computer-science department. It was designed using architectural walk-throughs, VR models that let the user virtually walk through the building before ground has been broken. Researchers built a treadmill-and-helmet system that lets the user walk naturally to move forward, turn handlebars to turn, and tilt his or her head to orient the view. The walk-through actually allowed the users, including the computer scientists who would be occupying the building, to discover design flaws. The architect balked at one change the computer scientists thought was needed, until he took the virtual tour himself.
NASA researchers have developed systems that can simulate in real time acoustic factors such as reflecting surfaces and various degrees of surface-sound absorption. This sound can then be delivered through headphones as a 3-D sound field, and can be correlated with visual feedback.
VR systems are also available commercially (or will be soon) for things like observing how radiation will pass through a patient's organs, setting up virtual physics labs in schools, and letting astronomers get a sense of the 3-D structure of the universe by flying through a virtual universe. Arcade games that use the technology of VR to wrap the player in a virtual world are already in use, and more are likely to come out this year.
Possibly the most useful application of VR is also the most abstract, the least real. Often analysts struggle with summary statistics and exploratory data-analysis techniques, trying to get a feeling for how a complex, multidimensional database is structured. VR lets them fly through the data, switching dimensions with a wave of the hand. This, it seems to me, is very useful.
So does virtual reality work as a psychedelic drug? Does it take the user to another place, inaccessible to the unaltered consciousness? Does it bring one face-to-face with the howling Tao? Do you care?
The answer is probably no to the howling Tao bit. But does VR take the user elsewhere? That question is probably useless in that form, but here are some alternatives that may be more useful: Does virtual reality have the potential to give us a sense of being somewhere else? Yes. Will there be things we can do in the somewhere elses of virtual reality that we can't do elsewhere? Yes. Is it possible that some of those things will become so important to us that we can't imagine getting along without them? Yes. Is that what the seeker after transcendent experience is looking for? No, probably not. Does that matter? No, probably not. The real question about this stuff is perhaps, How important will it be to me? Time will tell, but some of the applications I've described suggest that some applications of virtual reality may become, or may already be, very important to you.
Another issue is, does virtual reality have the potential to be addictive? If that means psychologically addictive, rather than physiologically, then I think the answer is, of course. Why should virtual reality, skillfully done, be any less addictive than television, the premiere electronic drug?
Copyright © 1993, Dr. Dobb's JournalVirtual Vision
Virtual Hearing
Virtual Touch
Reality