Mary Ann Lieberç Inc.
Challenges of Recreating Reality in Virtual Environments
MILTON P. HUANG, M.D.,' JOSEPH HIMLE, Ph.D.,' KLAUS-PETER BEIER, Dr.-Ing2
and NORMAN E. ALESSI, M.D.'
ABSTRACT
Virtual environments (VE), also known as virtual reality (VR), have been used in exposure treatment of phobias by simulating a situation that the patient fears. Initial studies ofthese treatments have demonstrated. effectiveness of treatment, but have not compared it to the "gold standard" of in vivo exposure. We are in the process of comparing VE exposure treat-ment for acrophobia to in vivo exposure treattreat-ment by replicating an actual in vivo exposure environment in a virtual model. Our design of both experimental system and experimental protocol aims to extract the essential, unique aspects of the VE experience that make
itdif-ferent from traditional treatment, and to increase our understanding of how these relate to the psychological history that people bringto Such encounters. Besides the challenges of pro-tocol design, this process also provides an illustration of the challenges of working with rapidly changing hardware and software standards, as we are attempting touse state-of-the-art equipment and software, such as the CAVE (CAVE Automatic Virtual Environment) and VRML (Virtual Reality Modeling Language).
INTRODUCTION
'PE
USE OF VÌRTUAL ENVIROF4MENTS (VE), also known as virtual reality (VR), is growingrapidly with continuously advancing
tech-nologies and related standards. Many potential applications exist for the use of VE in the treat-ment of psychiatric disorders,1 the most stud-ied being that of the treatment of phobias. In this case, VE has been generally used in expo-sure treatment, by creating a feared environ-ment in vrtual reality, then leading a patient
through that. environment using traditional
techniques of graded exposure. Many case re-ports suggest the effectiveness of VE treatment for various phobias including fear of flying,2 fear of spiders,4 and acrophobia.5 Controlled
'University of Michigan Department of Psychiatry. 2University of Michigan Virtual Reality Laboratory.
studies have been conducted primarily for acrophobia.6'7 These studies have shownthe
ef-fectiveness of VR exposure using comparisons to wait-list controls and cognitive therapy, but have failed to compare it to in vivo exposure, which is the current state-of-the-art treatment for acrophobia. Our work is aimed at explor-ing such a comparison, so that we can better understand how the psychological effects of virtual reality differ from the normal psycho-logical interaction with reality itself. This ex-ploration has lead us to face three primary chal-lenges. The. first is in
the design of an
appropriate experimental system that takes ad-vantage of current technology to permita com-parison between the modality of VE therapy and traditional therapy. The second challenge is to design an experimental protocol for this system that cannot only measure. the effective-ness of this new treatment modality, but also allow insight i$vhat specific factors produce-IECIINISCRE UNVVESng 163 aboratorjum 'icor
Scheepshydmp,j
krchief Mekelweg 2, 2628 CD Deift --- ici Oi ?8687 - Fa 015 - 18183edifferences between it and traditional treat-ment. Our final challenge has been to
imple-ment such these designs in
an environmentwhere computer hardwareand software stan-dards changeon a constantly accelerating ba-sis.
EXPERIMENTAL SYSTEM
As we wish to compare how.people react to virtual environments and how they react to re-ality, our primary, challenge in design of the experimental system has been controlling for the differences between virtualañd in vivo ex-posure. Our two-pronged approach includes creating a virtual model to duplicatethe actual environment as closely as possible and mini-mizing the intrusiveness of the VR equipment. The Anxiety Disorders program of the Univer-sity of Michigan has traditionally treated pa-tients suffering from acrophobia
by eposing
them to progressively greater heights looking out the windows of the East Elevator shaft ofthe University of Michigan
main hospitalbuilding. We createda virtual model of the dif-ferent elements of this experience, including walking in the lobby space, using the elevator, and examining the views out the window at different, floors. The three-dimensional (3D) geometry was developed through the use of geometric modeling software8 and enhanced with various multimedia software.!It includes the building's interior elements (floor, walls, doors, benches, plants, elevator,etc.), as well as the exterior visible from the Window
(court-yard, trees, other buildings,
walkways, citybackground).
Virtual environments often have a
'cartoon-like" feel, because the appearance of geometry is only modeled using colorand reflective qual-ities to reduce the computational load. A tech-nique called "texture, mapping" provides a more realistic appearance, but is computational intensive. Textures are digital images and pho-tographs that are pasted onto the geometry and simulate the complexappearance of real objects in a more naturalway. Using digital cameras, we captured elements of the in vivo environ-ment and transferred these imagesinto the vir-tual model. In addition, libraries of textures
were used for appearance' enhancement (e.g.,
fabric on elevator walls, wall textures, and floor tiling).
Use of state-of-the-art technology helps to minimize differences between VR exposure
and in vivo
exposure. Our model isexperi-enced in a CAVETM (CAVE Automatic Virtual
Environment) as developed by the University of illinois at Chicago and is now commercially sold by Pyramid Systems.The CAVE is a pro-jection-based VR system that surrounds the viewer with four IO' X 10' screens, arranged to form three walls and the floor of a cube (Fig-ure 1). The viewer wears liquid crystal shutter glasses and a six-degrees-of-freedom head-tracking device. As the viewer moves inside the CAVE, a Silicon Graphics Onyx computer
cal-culates the correct perspective projections for each wall, based on head-tracking measure-
-ments that describe the viewerslocation in the virtual model. The images for the left and the right eye are projected in a rapid, alternating' sequence. The shutter glasses alternatelyblock
the left and the right
eye in synchronizationwith the projection sequence, giving the ap-propriate 3D image. The CAVEenvironment is
a superior system when comparedwith the tra-ditionally used head-mounted display (HMD)
that is worn like
a helmet. An H'MD isex-tremely intrusive, uncomfortable to wear, and provides an unnatural restricted field of view
(FOV). In contrast, the
lightweight shutterglasses and the surrounding walls of the CAVE provide a very wide FOV that properly stimu-lates the human peripheral vision and
facili-FIG. 1. 'CAVE projection system.
tates significantly orientationand navigation in the virtual world. The stereo projection on the
floor enables the
viewer to perceive objectabove and below thefloor. Looking down from a Window or approachinga cliff can become a very convincing experience. The viewer can also see real objects in the CAVE like his/her own body or that of others, more closely sim-ulating the effect oftherapist presence in vivó. The use of these technologies allows our
sys-tem to make the
experience of our virtualenvirorunent as close to reality as is currently feasible. To control for the remaining short-comings of the CAVE experience in the
com-parison with in vivo
exposure, we will placecomparable restrictionson the in vivo partici-pants by asking them to wear shutter glasses
and not to touch
physical objects like walls,since subjects in the virtual environment are unable to feelany objects of their surrounding virtual world.
EXPERIMENTAL PROTOCOL
Our experimental protocoil is designed to clarify variables thatseparate the treatment in the CAVE to treatment in the actual environ-ment.10 This requires us to not only
examine the subjects using traditional assessments that are used in the psychiatric literature, but to also use other assessment toOls to attempt to mea-sure other variables thatare not usually evalu-ated. Subjectsare selected as having diagnosis of specific phobia" using the Structured Clin-ical Interview forDSM-IV (SCID),'2 thengiven a battery of questionnajr to assess their acro-phobia.'3-'6 A behavioral approach test in the elevator lobby establishes a uniform measure of their ability to tolerate the in vivo
situation. They are then randomized to a treatment
con-c
dition of either in vivo exposure, CAVEexpo-- e sure, or a control conditionof relaxation
train-3
ing. In each
Condition, a therapist takes then subject through a standardized training for
90 a mm, accompanied by physiologic monitoring e of heart rate, respiration rate, and galvanic skin t response. After completion of exposure train-ing. each subject is returned to the in vivosit- ti
uation for a repeat
behavioral approach test.d
This will allow
a direct comparison of ims
provernent among all conditions,as well as ex-amine the ability of virtual exposure to gener-alize to real life.
Many other variables may show differences between the virtual and in vivo groups. Phys-iologic measurements and subjective measure-mentsofthe impact ofthese exposures will al-low us a sense of theeffect on anxiety. We will also measure demographic variables,pnor ex-posure to different types of technology, and
other measures of the impact of "virtuality."
The literature
contains several attempts tomeasure the extent to which a subjectfeels trûe "presence" in a virtual environlnent.17-19 We will examine how suchvariables influence both measures of anxiety and measùres of treatment
efficacy.
IMPLEMENTATION IN HARDWARE
AND SOFTWARE
Our process of modeling and experimental design has been instructive, teaching us lessons about the interactions of hardware, software, and psychiatric knowledge and the challenges of integrating them together in producing prac-tical virtual reality. The interaction is complex. If we extrapolate from other models of tech-nology, such as that of the Internet,20 we rec-ognize that each of thesethree elements
are
de
-pendent upon the Others, yet they each have their own trajectory of development with hard-ware moving the fastest, software quickly fol-lowing, and psychological models lagging
be-hind;. the difficulties
we have faced in
our modeling process give concrete example of this.Computer hardware changes continuously
as industry competes to create better and
heaper products. Processing speed doubles very 18 months,21 allowingfaster rendering of
D models andmore effective inclusion of tech-iques like texture mapping. Our aim of using state-of-the-art system like the CAVE is an xample of an attempt to take advantage of hese trends. Unfortunately, this rapidity of
de-
-'elopment tends to make each hardware solu-on isolated, not easily integrated with other evices and outstripping the development
of
sys-tern is commercial, each installation requires unique adjustments. In our case, setup of the CAVE has been a long process with frequent testing forrealigning projectors, connecting the computer to the projectors, or calibrating the tracking devices in the 3D space. We are ex-ploring the interferences between the electronic fields of the tracking system and of our physi-ologic monitoring equipment as the CAVEwas not designed for the use of such systems inside. We are also continuing to get additions to the CAVE as other hardware is added to increase. the speed of information flow. Unfortunately, we have already found that such changes have invalidated some of our current software, ne-cessitating rewrites. We àre always attempting to adjust for ongoing changes in hardware.
We have seen in other ways how hardware
development outpaces the development of
supporting
software. The CAVE is
pro-grammed in either C or C+ using specific li-brary calls that control different hardware ele-ments. Higher level libraries (EVL, Performer, Open CL, and others) allow easier program-ming for the CAVE environment. Recently, a standard for the development of virtual envi-ronments has been developed for the World
Wide Web (WWW). This standard, called
VRML (Virtual Reality Modeling Language), is
not restricted to the WWW, but can be utilized for any VR application.. VRML defines a virtual environment by geometry, appearance (e.g., color, textures), and illumination, and in-cludes fünctions for animations, interactions, and behavior scripting. Translators that con-vert a VRML application into a CAVE applica-tion are under development and will, in the fu-ture, make VRML an éxcellent standard for the
definition and exchange of virtual
environ-ments. In the meanwhile, we are coding the in-teractjons of our model in both Performer and VRML, requiring some reduplication of work. The challenges. of keeping up with advanc-ing standards of härdware and software and interconnecting them offer a contrast to deal-ing with a lack of standards in how we orga-ni.ze our understanding of the relevant
psy-chological and social
factors that should influence the creation of a virtual environment. Clearly, the potential umber of such factors isHUANG ET AL
endless. In designing our environment for
treatment of acrophobia, our primary consid-eration has been trying to make the virtual en-vironment as similar to the real enen-vironment as possible. Our choices have been made based on subjective experience, reiteratively entering the virtual environment, "feeling" what seems "right," and then making appropriate changes
in system programming. This design process is
subject to errors from the fact that many
per-ceptions of the environment are unconscious in
their influence and, therefore, ignored. In ad-dition, we intentionally left out some parts of the experience because of the difficulty in ac-curately replicating them. We do not simulate reflections off of the window, windòw tinting, or dust, which all commonly serve as cues that a barrier is present. Thus far, there is little re-search on how important choices like "window tinting" or the presence of reflections are in making virtual reality simulate reality.
CONCLUSIONS
-Our experimental procedure is aimed at de-
-termining where the differences exist between
virtual reality and reality in the context of
acro-phobia treatment. It compares the treatment of subjects in a real location to treatment ina VR
simulation of that real location. Use of mini-mally intrusive technology and appropriate protocol design allow us to make treatment in these two settings as similar as possible, per-mitting us to examine VR itself as an indepen-dent variable. This requires much work as we
have seen in our attempts to use the CAVE and VRML.
The design of this project and the difficulties
of realizing it suggest some of the challenges
we face in using virtual reality in mental health,
as well as important considerations for future work and research. Keeping up with continu-ously changing hardware and software
re-
-quires time and dedication. We need education
in how such learning can be intégrated intoour usual work, as well as how to define our own expertise and interface it with that of
techno-logical experts.24 As mental health experts, we need to focus on the design of experimental
systems and protocols that can be used to in-vestigate questions of interest, independent yet cognizant of the underlying technology used to realize these designs. Perhaps the greatest chal-lenge before us is how we can organize our ex-pertise in a way in which other researchers in virtual reality and computer interface design can access it to incorporate it into their work and research. We need to work on models that define the mental attributes that should be con-sidered in creating virtual environments. Al-though work has been done on the effect of
en-vironmental variables on factors related to
perception, little exists on their impact on other psychological factors and how they relate to our current knowledge of mental illness, per-sonality attributes, and normal mental func-tion.
COmputer technologies now face the mental health field with new challenges and opportu-nifies. We hope that a thoughtful exploration of the design process of mental health applica-tions in virtual reality wjll encourage the ex-perts in this area to work together to further develop ideas about other standards we need. Such work will be essential to encourage re-search and increase our understanding and use of virtual environments in psychiatric and psy-chological applications.
12.
REFERENCES 16.
I. Glantz, K., Durlach, Ni., and Aviles, W.A. (February 1997). Virtual reality (VR) and psychotherapy:
Op-portunities and challenges. Presence: Teleoperators and 17.
Virtual Environments, 6(1), 87-105.
Rothbaum B.O., Hodges, LP., Watson, BA., Kessler, C.D., and Opdyke, D. (1996). Virtual reality exposure 18.
therapy in the treatment of fear of flying: A case
re-port. Behiviour Research and Therapy, 34(5/6), 477-481.
North, M.M., North, S.M., and Coble, J.R. (February 1997). Virtual environments psychotherapy: A case
study of fear of flying disorder. Presçnce: Teleoperators and Virtual Environments, 6(1), 127-132. 19. Carlin, A.S., Hoffman, H.G., Weghorst, S. (February 1997). Virtual reality and tactile augmentation m the treatment of spider phobia: A case study. Bheaviour
Research and Therapy, 35(2), 153-158. 20.
5. Rothbaum, B.O., Hodges, LF., Kooper, R., Opdyke,
D., Williford, J., and North, M.M. (1995 Summer).
Vir-tuai reality graded exposure in the treatment of acro- 21. phobia: A case report. Behavior Therapy, 26(3), 547-554.
. Rothbaum B.O., Hodges, L.F., Kooper R., Opdyke. D.,
Williford, J.. and North. MM. (April 1995). Effective-
-ness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia. Americin
Journal of Psychiatry, 152(4), 626-628.
Lamson, R.J. (1997). Virtual Therapy: Prevention and Treatment of Psychiatric Conditions by Immersion in Vir-tuai Reality Environments. Montreal: Editions de i'E-cole Polytechnique de Montréal.
Mortenson, ME. Geometric modeling (2nd ed.). New York: Wiley, 1997.
Huang, M.P., and Alessi, N. (1996). Tools for
deve!-oping multimedia in psychiatry. In M. Miller, K.
Ha-mond, & M. Hile (eds.), mental health computing. New York Springer-Verlag, pp. 322-341.
Huang, M.P., Himle, J., Beier, K., and Alessi, N.E. (in
press). Comparing virtual and real worlds for
acre-phobia treatment. In Medicine Meets Virtual P..ealüy 6. Amsterdam: lOS Press.
American Psychiathc Association. (1994). Diagnostic and Statistical Manual Mental Disorders (4th ed.).
Washington DC.: APA Press.
First, M.B., Spitrer, RL, Gibbon, M., and Williams,
J.B.W. (1997). Structured Clinical Interview for DSM-IV Axis I Disorders (SCm-I), Clinician Version. Wash-ington, DC: American Psychiatric Press.
Cohen, D.C. (1977). Comparison of self-report and overt-behavioral procedures for assessing
acropho-bia. Behavior Therapy, 8, 17-23.
Marks, IM., and Mathews, A.M. (1979). Brief
stan-dard self-rating for phobic patients. Behaviour Research and Therapy, 17(3), 263-267.
Michelson, L (1986). Treatment consonance and re-
-sponse profiles in agoraphobia: The rôle of individ-ual differences in cognitive, behavioral and
physio-logical treatments. Behaviour Research and Therapy, 24(3), 263-275.
Menzies, R.G., and Clarke, J.C. (May. 1993). The
etiol-ogy of fear of heights and its relationship to severity
and individual response patterns. Behaviour Research and Therapy, 31(4), 355-365.
Sheridan, 1.8. (Spring 1996). Further Musings on the Psychophysi of Presence. Presence: Teleoperatorsand Virtual Envirsnments, 5(2), 241-246.
Welch, RB., Blackmon, T.T., Liu, A., Mellers, B.A.,
and Stark, LW. (Summer 1996). The effects of
picto-rial realism, delay'of visual feedback, and observer
interactivity on the subjective sense of presence. Pres-ence: Teleoperators and Virtual Environments, 5(3),
263-273.
Hendrix, Ç., and Barfield, W. (Summer 1996). Pres-ence within virtual environments as a function of vi-sual display parameters. Presence: Teleoperators and
Virtual Environments, 5(3), 274-289.
Huang, M.P., and Alessi, N.E. (July 1996). The
Inter-net and the future of psychiatry. American Journal of
Psychiatry, 153, 861-869.
Schaller, R.R. (June 1997). Moore's Law Past, present, and future. IEEE Spectrum, 34, 52-59.
Virtual Reality Modeling Language ISO/IEC Draft Inter-national Standard 14772-1 (December 1997).
Wagner, M. (JuLy 7, 1997). Real-world uses for 3-D
Web spec. Computerworld 31, 45-i-.
Huang, M.P., and Alessi, N.E. (in pess). An
infor-matics curriculum for psychiatry. Academic Psychiatry.
Messi, N., Huang, M., and Quinlan, P. (1997). 2005: Information Techology impacts psychiatry. In L.J.
Dickstein, M.B. Riba, and iM. Oldham (Eds.), Amer-lean Psychiatric Press Review of Psychiatry (VoI. 16).
Washington DC: American Psychiatric Press, pp. V169-87.
Address reprint requests to:
Dr. Milton Huang 1500 E. Medical Center Dr., TC 3502/Box 0390 Ann Arbor, MI 48109-0390, USA Email: mhuang@umich.edu
