Space through movement: A method for three-dimensional image presentation

THROUGH

MOVEMENT

C.J. O v e r b e e k e

TRdiss TRdiss M.H. S t r a t m a n n

lipt

SPACE THROUGH MOVEMENT

A niL-ihud for ihree-dimensional image presentation

C.J. Overbeeke

M.H. Stratmann

A mclhud for ihree-dimcnsiunul image prcsonluiion

Gezamenlijk Proefschrift

ter verkrijging van de graad van doctor aan de Technische Universiteit Delft,

op gezag van de Rector Magnificus, prof. dr. J.M. Dirken, in het openbaar te verdedigen ten overstaan van een commissie door het College van Dekanen daartoe aangewezen, op maandag 21 m a a r t 1988 te 14.ÜO uur door CORNELIS JOHANNES OVERBEEKE,

geboren te P u t t e , licenciaat in de psychologie te 15.30 uur door MICHAEL HEINRICHSTRATM ANN,

geboren te Verl, Diplom-Designer.

TR diss TR diss

1617 1618

1 De koppeling tussen bewegingen van een waarnemer en verschuivingen in de

optical array met het fixatiepunt als ankerpunt, is noodzakelijk en voldoende om,

uitgaande van een tweedimensionaal monitorbeeld, betrouwbare afstandsschattingen te maken. dit proefschrift

-2 De koppeling tussen bewegingen van een waarnemer en verschuivingen in de acoustical array met een samenvallend draaipunt, is noodzakelijk en voldoende om betrouwbare richtingsschattingen te maken. dit proefschrift

-3 De gebruikelijke formule voor de monoculaire bewegingsparallax is:

v = «z-ZoKzoi-v,

waarin v, de snelheid is waarmee het hoofd zijwaarts bewogen wordt, Zo het fixatiepunt is en z een tweede punt is en v de snelheid waarmee de projectie van z over het netvliesbeeld beweegt. Deze formule is niet algemeen geldig zoals in de meeste handboeken over perceptie beweerd wordt. Zij geldt alleen voor punten die loodrecht op de bewegingsrichting staan.

4 Het feit dat gestructureerd licht ook informatie bevat over voorwerpen die zich achter de kijker bevinden wordt niet dikwijls onderkend.

5 De theorie van de directe waarneming beweert dat eerst de gedragsbetekenis van een voorwerp wordt waargenomen en dan pas de structurele kenmerken van het voorwerp. Deze opvatting heeft grote consequenties voor het ontwerp- en vormleeronderwijs. Studenten moeten getraind worden in het zien van de relatie tussen de gedragsbetekenis en de structurele kenmerken van een voorwerp.

6 Relativisme en defaitisme zijn twee verschillende begrippen. dit proefschrift

-7 Door te leven als Nederlander in België en als Belg in Nederland, krijgt men een goed inzicht in het begrip ruimte.

8 C'est nul sans bulles betekent niet dat het leven zonder bul niets waard zou zijn.

1 The coupling between movements of an observer and shifts in the optical array with the fixation point as pivot, is necessary and sufficient to make reliable distance estimates on a two-dimensional monitor. - this thesis ■

2 The coupling between movements of an observer and shifts in the acoustical array around a coincident point is necessary and sufficient to make reliable direction estimates. - this thesis ■

3 The common formula for the monocular movement parallax is:

V = ( ( Z - ZoJ/Zu) . V ,

where v, is the velocity at which the head moves sideward, Zo is the fixation point and z another point, v the velocity at which the projection of this point moves over the retina. This formula is not generally valid as is proposed in most handbooks of perception. It is only valid for points on the axis through the fixation point and perpendicular to the direction of movement.

4 The fact t h a t structured light also affords information about objects behind the observer is not often acknowledged.

5 The theory of direct perception asserts that what an object affords is perceived prior to the perception of the structural aspects of t h a t object. This view holds many consequences for design education. Students must be trained in discerning the relation between affordances and structural aspects of an object.

6 Relativism and defeatism are two different concepts. this thesis

-7 By living as a Dutchman in Belgium and as a Belgian in Holland, one gets a good insight in the concept of space.

8 C'est nul sans bulles does not mean t h a t live would be worthless without a doctoral degree certificate.

de verschuivingen in de weergegeven scene bereikt wordt is van ondergeschikt belang. Belangrijk is alleen dat de koppeling bereikt wordt, en wel rond het fixatiepunt. dit proefschrift

-2 Het visuele systeem is niet bestand tegen gedwongen onbeweeglijkheid. dit proefschrift

-3 Stereoscopieen binoculairzien zijn begrippen die strikt onderscheiden dienen te worden. dit proefschrift

-4 De voorgestelde werkwijze is gebaseerd op de bewegingen van een waarnemer en de daaruit resulterende verschuivingen in de weergave van een scene. Daarom lijkt het mij veelbelovend naar een combinatie met het Pulfrich-efFect te streven omdat dit effect, in tegenstelling tot de andere stereoscopische werkwijzen, op beweging in de scene is gebaseerd.

5 Als het lukt de convergentie en de accommodatie van een waarnemer tegelijkertijd met zijn bewegingen te meten, dan kan het fixatiepunt zonder verdere hulpmiddelen naar believen verplaatst worden. Een combinatie met stereoscopie zou dan overbodig zijn omdat, met behulp van twee aan de oogbewegingen gekoppelde spiegels twee beelden over elkaar geprojecteerd zouden kunnen worden. De achtereenvolgend gefixeerde punten zouden dan scherp verschijnen, terwijl voor- en achtergrond als dubbelbeelden op het netvlies zouden geprojecteerd worden, zoals in het dagelijkse leven.

6 De doodlopende straat waarin vele vormgevers zich vandaag bevinden, kan in een nieuwe, brede straat veranderen, als men de ideeën van Gibson zou aanvaarden. In plaats van glad te strijken, op te ruimen en naar technische funktie vorm te geven, zou men naar de gedragsfunktie van produkten moeten zoeken en de produkten zo vormgeven dat ze door hun expressieve waarden uit de anonimiteit zouden treden.

7 Als men nieuwe technologieën op het gebied van ruimtewaarneming wil evalueren, dan dient men niet een technische maar eerder een waarnemings-psychologische maatstaf te hanteren.

1 It is not important what techniques are used to achieve the coupling of observer's movements and shifts in the depicted scene. What is important is that the coupling is achieved, particularly anchored in the fixation point. this thesis

2 The visual system does not tolerate forced immobility. this thesis

-3 Stereoseopy and bir.ocular vision are two concepts that need to be distinguished strictly. this thesis

-4 The system presented here is based on observer's movement and the resulting shifts in the depicted scene. That is why it seems promising to me to combine it with the Pulfrich-effect. This effect, contrary to other stereoscopic systems, is based on movement in the scene.

5 If the convergence and accommodation of an observer could be measured together with his movements, the observer could freely choose a fixation point, not needing further devices. A combination with stereoseopy would be

superfluous because, with the help of two mirrors coupled to the eye movements, two images could be superimposed. Successive fixation points would then appear focused while the foreground and the background would be projected as double images on the retina, just as in everyday life.

6 The cul-de-sac where many designers find themselves nowadays could change to a broad avenue, if one would accept Gibson's ideas. In stead of smoothing out, tidying up and designing according to technical function, one would look for the affordance of products and design them accordingly. In this way products would step out of anonymity because of their expressive value.

7 One who wants to evaluate new technologies in space perception, should not judge according to technical but rather to perceptual standards.

P a r t 1 T h e o r y <C.J. Overbeeke)

Introduction 1 1 The classical, indirect a p p r o a c h 7

1.1 Introduction 7 1.2 The bottom up and top down approach 7

1.3 Critique of the classical approach 11

1.3.1 Introduction 11 1.3.2 Logical criticisms 12 1.3.3 Empirical criticisms 16 1.4 Definition of space perception 17

2 The direct approach 21

2.1 Introduction 21 2.2 The direct approach 21 2.3 Experiments 26 2.4 Critique of the indirect approach 30

3 Space and space perception 33

3.1 Introduction 33 3.2 Geometry 34 3.3 Experience 38 3.4 Definition of space perception 41

4 Movement parallax 43

4.1 Introduction 43 4.2 Historical overview 44 4.3 The components of movement parallax 48

4.3.1 .Movement 48 4.3.2 Shifts 51 4.3.3 Coupling between movements and shifts 52

4.4 Definition of movement parallax 53

5 Technical set-up 55 5.1 Introduction 55 5.2 Technical set-up 56

6.2 Pilot experiment 61 6.2.1 Method 62 6.2.2 Results 64 6.2.3 Conclusions 65 6.3 Main experiment 66 6.3.1 Method 66 6.3.2 Results 73 6.4 Control experiment 74 6.5 Audititory experiment 76 7 Conclusions 81 7.1 Introduction 81 7.2 Theoretical conclusions 81 7.2.1 One-to-one relationship 82 7.2.2 Relativism 82 7.2.3 Affordance 83

P a r t 2 T e c h n o l o g y (M.H. Stratmann) 85

2.3 Stereo systems 106 2.3.1 Mirror, prism and lens stereoscopy 108

2.3.2 Anaglyphic stereoscopy 113 2.3.3 Polarization stereoscopy 116 2.3.4 Pulfrich stereoscopy 119 2.4 Parallax systems 120 2.4.1 Passive parallax systems 121 2.4.2 Active parallax systems 123 2.5 Stereoparallax systems 128 2.5.1 Passive stereoparallax systems 129

3.2 3.3 4 4.1 4.2 4.3 4.4 4.4.1 4.4.2 4.5 4.6 4.6.1 4.6.2 4.7 5 5.1 5.2 5.2.1 5.2.2 5.2.3 Theory The Gap

The new system Introduction Setups

Swivelling arm setup

Camera dolly with potentiometer measurement Camera dolly

Measurement of movement Main experiment

Present setup

Camera guidance mechanism Measurement of movement DelftT-ouvain collaboration Conclusions Introduction Technological conclusions Coupling Fixation point Areas of application 147 151 153 153 154 157 162 162 166 169 180 181 186 188 191 191 191 192 195 195 References 199 Appendix 1 Appendix 2 Appendix 3 Appendix 4 Appendix 5 Summary Part 1 Samenvatting Deel l Summary Part 2 Samenvatting Deel 2

tend to be forced back on the defensive by their colleagues. The example of Galileo speaks volumes. In about 1950 the American perceptual psychologist J.J. Gibson proposed a paradigm shift that has fundamental implications for the way man sees himself. The new paradigm was concerned with the 'mind'. Gibson claimed that the 'mind' derives its rules from nature and not las Kant, for example, asserted) that the 'mind' imposes its rules on nature.

Gibson arrived at this new paradigm through a combination of fundamental and applied research. During the Second World War it was his task to select aircrew for training as pilots, and as a perceptual psychologist he was unable to do this. When put into practice, the prevailing theory turned out to be useless: it told him nothing about what information a person needs to be able, for example, to land an

aeroplane. Gibson found that this information was contained in the world, not in the 'mind'. Hence the saying: 'Ask not what's inside your head, but what's your head inside of.' (Mace, 1977, Appendix 2)

Gibson himself, however, neglected to operationalize, i.e. to lay it down in measurable terms, his new paradigm sufficiently. This was left to his followers, and several examples will be found in this book.

This double dissertation by a researcher and a designer is a contribution to the continued operationalization of Gibson's paradigm. We looked to see what are the necessary and sufficient conditions for obtaining a three-dimensional impression from a two-dimensional screen. We found that if, first, the movements of an observer are coupled to the movements of a camera, with the observer looking at a two-dimensional screen displaying the image recorded by the camera, and if, second, the observer and the camera move round coincident points, then the observer will have a three-dimensional impression of the scene. This finding is theoretically interesting because it provides confirmation of the new paradigm; technologically it is interesting because it leads to new applications.

We reached this finding through a balanced exchange between disciplines, between the researcher on the one hand and the designer on the other. The theoretical hypotheses generated by the researcher were cyclically adjusted on the basis of successive field observations and the concomitant new technological solutions provided by the designer. Thus the importance of movement was arrived at by the

s o l u t i o n s , after which the hypothesis was adjusted again, and so on. When we had a w o r k i n g system, reached through this process of interaction, we tested the

i m p l i c a t i o n s of Gibson's theory experimentally.

T h i s d i s s e r t a t i o n is the concrete result of t h a t interaction. It will be clear from the foregoing t h a t it was difficult for us to present the text in such a way t h a t both the i n t e r a c t i o n would become clear and each c a n d i d a t e could be held responsible for a s e p a r a t e and clearly delineated part of the thesis or for the main experiment, as the r e g u l a t i o n s for doctoral c a n d i d a t e s a t Delft U n i v e r s i t y of Technology require. We therefore elected to split the dissertation into two parts: Theory and Technology. T h e Theory part contains the theoretical s t a r t i n g points and experimental testing. T h e Technology part contains the description of the solutions arrived a t from, successively, field observations and new theoretical insights, and the novelty search. T h e d i s a d v a n t a g e of this a r r a n g e m e n t , of course, is t h a t we run the risk t h a t the reader will forget t h a t the Technology part actually belongs between the t h e o r e t i c a l premises on the one hand and the e x p e r i m e n t a l testing on the other, in t h e Theory part. However, we both take personal responsability for the entire t h e s i s and its coherence. C. J. Overbeeke signs specifically for the Theory part, M, H. S t r a t m a n n for the Technology part.

We are grateful to m a n y now t h a t this dissertation is finished. In the first place we t h a n k our promotor, prof. dr. G.J.F. Smets who guided and supported us with u n c e a s i n g energy. We also t h a n k all m e m b e r s of the Laboratory of Form Theory: O n n o van Nierop for the drawings, the moving cover and so much more; P.J. S t a p p e r s for his computer wizardry; N'ico Heinrichs, and later Guus Mansveld, for t h e i r electronic knowledge. We cannot, of course, forget Mauri ts de Koning, for his e n t h o u s i a s m , and Carlita Kooman. Erik van Kuijk made excellent d r a w i n g s . H.S. L a k e had a tough job t r a n s l a t i n g from the Dutch and the G e r m a n .

PART 1 THEORY

Eppur' si rauove (Galileo)

Introduction

In this thesis I frequently use the first person plural because the thinking and practical work that resulted in a patent application for the Netherlands, Europe, the USA and Japan (Smets, Stratmann, & Overbeeke, 1985,1986a, 1986b, 1986c) was done in a team context by the Laboratory of Design Theory. The most

important contributions were made by Professor dr. G.J.F. Smets, my promotor, Dipl. Des. M.H. Stratmann, who is taking his doctorate with me, Ir. M.S.A. de Koning, a graduate of the laboratory, and myself.

Fig. 1 When an observer moves from right to left with his eyes fixed on the middle of the cylinder (rightl, the objects displayed on a screen will shift relative to each other as shown in the left-hand half of the drawing

This thesis describes the theoretical starting points and the experimental testing of a principle whereby a reliable spatial impression may be obtained from an image on a two-dimensional screen. Reliable means that subjects are able to make

systematic distance e s t i m a t e s on the basis of this principle. Reliable does not m e a n t h a t subjects a r e able to m a k e veridical (i.e. in accordance with physical

m e a s u r e m e n t ) distance estimates. Systematic errors in the e s t i m a t e s can be corrected, as long as the e s t i m a t e s are systematic.The principle is based on m o v e m e n t p a r a l l a x . Movement p a r a l l a x is the i n v a r i a n t comprising the coupling, with the fixation point as the pivot, or rotation point, between the m o v e m e n t of an observer and the associated shifts in the optic a r r a y . The shifts occur, presented schematically, as shown in Fig. 1. Behind the fixation point the objects move with the observer, in front of it they move in the opposite direction to the observer. The most familiar illustration of this phenomenon is the moon riding along beside us as we travel by t r a i n or car.

So w h a t is new about the principle? We have built a set-up in which an observer sits in front of a monitor and observes a scene registered by a video camera and reproduced on the screen. T h e observer's movements are monitored, and the d a t a passed on to a m e c h a n i s m t h a t moves the camera correspondingly. It is the coupling of the c a m e r a ' s movements round a corresponding fixation point (a point in the scene t h a t he is observing) and the movements of the observer round a coincident rotation point (the same point in the scene, but as reproduced on screen) t h a t is essentially new. In this thesis I shall show t h a t this coupling is necessary and sufficient to arrive a t reliable distance estimates on a flat screen. T h i s is our hypothesis.

T h i s hypothesis is theoretically interesting because it contributes to a resolution of the debate between two rival theories of perception: on the one hand t h a t of Gibson, known as the direct theory (e.g. Gibson, 1979; Michaels, & Carello. 1981), the o p e r a t i n g principle of which is a hitherto undescribed and unapplied implication. and on the other the classical, indirect view of space perception (described in Marr, 1982; Hochberg, 1978; Gregory, 1972), in which the importance of retinal disparity (the fact t h a t we" have two eyes) is emphasized.

T h e proposed hypothesis is technologically interesting because it leads to different applications from existing three-dimensional imaging systems, which are based on binocular disparity, pictorial monocular cues (for example shaded graphics), holographic t e c h n i q u e s or varifocai mirror systems. In contrast to s y s t e m s based on binocular d i s p a r i t y t h i s new principle can also be used monocularly. A second difference from such systems is t h a t the image they display can only be observed from one point. It is impossible to look round the objects displayed as we can with t h e objects t h a t s u r r o u n d us in everyday life (Suetens, 1983, p. 202). With the

principle described here this is possible. It can also be used for moving images, in contrast to holographic processes and varifocal systems (or at least those that have been developed so far). And it offers a more powerful spatial impression than can be achieved with monocular pictorial cues.

The technical part of the principle is described in the second part of this thesis. I shall begin with an extensive exposition of the theory because the whole question of space perception falls between two stools. It is nicely summarized in Fig. 2 (after

a painting by Magritteof 1931, Calcul Menial). On the one hand we see a

landscape with houses and trees, a world full of meaning. On the other hand we see a series of geometrical figures, a mathematical-physical world.

The perception researcher is always faced with this dichotomy. We experience a rigid world full of meaning, familiar objects, animals, plants and other people. In physics this world is embraced in terms of mathematical quantities and physical laws, no account being taken of our experience.

i

Fig. 2 Calcul Menial

The classical way of bridging this dichotomy is the calcul mental, in which the brain is supposed to enrich the mathematical-physical input by processes of memory and inference as to arrive at a meaningful percept. An example of this is shown in Fig. 3.

Von H e l m h o l t z s t a r t e d from the assumption t h a t the flat projection of the world onto the r e t i n a s of the eyes contains no information about distance. But it is possible to calculate the distance between an observer a n d an observed object from the a n g l e to which the observer's eyes are turned.

h'ig 3 According to von llelmholu the distance to an object can be calculated from the distance between the eyes (retinal disparity with a mean of 6 4 cm and a range from 5 2 to 7 6 cmi.Julcsz, 1971 )l and the angles la I fa I to which the eyes arc turned when we fixate the object

In c h a p t e r 1 of this thesis I explain huw the classical or indirect view originated and how it h a s remained the principal theory in psychophysics and cognitive psychology to this day. At the same time I shall describe the difficulties t h a t are i n h e r e n t in t h i s approach.

T h e r e is also a n o t h e r possible approach. Instead of a s s u m i n g t h a t the perceptual m e a n i n g h a s a central origin, we can s t a r t from the assumption t h a t this m e a n i n g also lies in the world. Gibson (1950, 1966, 1979) was the first to m a k e this

suggestion. He asserted t h a t we see m e a n i n g directly, and t h a t t h e r e is no dichotomy b u t r a t h e r interdependence between the world and the observer. Or, in o t h e r words, t h a t m e a n i n g arises in the reciprocity of object (the world) and subject (the observer). In c h a p t e r 2 I describe the view of Gibson and his followers. At the s a m e t i m e I point o u t t h e weak points in t h i s approach, which I shall from now on refer to as the direct theory of perception.

Why do I begin this thesis with a description of the difference between these two approaches? Because the assumptions u n d e r l y i n g both approaches, as will become a p p a r e n t , are fundamentally so different t h a t they are quite irreconcilable. It is v i t a l l y i m p o r t a n t t h a t these assumptions are explicated, because t h e y push the research in a p a r t i c u l a r direction and unavoidably lead to p a r t i c u l a r hypotheses and models. T h e scientist i n v e s t i g a t i n g perception m u s t therefore be clearly a w a r e of which a s s u m p t i o n s are implicitly incorporated into his s t a r t i n g points. In

chapters 1 and 21 explicate these assumptions for the indirect and direct approaches respectively.

In chapter 3 I set out my own view: that it is possible on the basis of a collaboration between world and observer to arrive at space perception, the experience of objects and events as standing apart from ourselves. Movement will be found to be

essential: hence the motto of this thesis. Movement parallax frames the interaction between the observer and the world in terms that can be handled mathematically and scientifically. In chapter 4, accordingly, I go more deeply into the subject of movement parallax by reference to a survey of the literature.

In chapter 5 I briefly describe the various set-ups that we have built to test the principle. This chapter has been kept short because the area is covered in detail in the second part of this thesis.

In chapter 6 I describe a number of experiments set up to test our hypothesis. The results show that people are indeed capable of using movement parallax to arrive at reliable distance estimates on a two-dimensional monitor. The results are so convincing that the system we have designed has also led to practical applications. Both the theoretical and the practical objective have been achieved.

In chapter 7 I draw the conclusions to which the research has led me. Now that I have explained what the reader can expect to find in this thesis, perhaps I should finish this introduction by saying what he should not expect. This thesis is not about processing images, it is not about retinal models, nor about determining critical psychophysical units or thresholds, about lens systems, mental processing models, or the perception of values or beauty. It is a functional approach to the problem of perception. But what do I mean by a functional approach? I wish, following on from the Gibsonian tradition, to determine how the visual system functions starting from a typically psychological approach. I do not, in so doing, reduce the problem to one of physics, nor to a matter of physiology, nor to neurology, nor to any combination of these jointly or severally prefixed by 'psycho-'. This is why I do not define the term space as a mathematical-physical system of axes, and why I do not try to determine how this system of axes is projected onto the retina according to the laws of optics, or describe how neurons, according to information theory, bundle signals together and send them on to the brain for processing. The combination of all these levels of analysis has already given scientists quite enough to lead to all sorts of misunderstandings and

pseudoproblems (Michaels, & Carello, pp. 97-105). At the same time, neither do I construct yet another mental or cognitive model to explain perception. Instead, I stay on a functional level. I merely indicate the necessary and sufficient conditions for space perception. Is this too summary an approach? I believe it will prove sufficiently innovative. The sciences to which I have referred can approach things from their own particular angles to investigate the supplementary conditions and the neural and cerebral structures necessary for space perception.

1 The classical, indirect, approach

1.1 I n t r o d u c t i o n

In this chapter I clarify the definition of space perception as will be found in all the classical textbooks on perception. I shall show that this definition is mathematical-physical and that it implicitly disregards experience. To do this I shall first have to make a brief foray into history in order to obtain a good overview of the classical approach.

Psychology as a science arose as an offshoot of both physics and philosophy. These two parent sciences can be recognized in modern psychology as the bottom up and the top down approach respectively (see e.g. Lindsay, & Norman, 1977; Wessels, 1982).

I shall explain these approaches in more detail and then demonstrate that the same axioms underly both schools of thought. The validity ofthes axioms is then tested on the basis of logical and empirical data. I then give the definition of space perception within this approach.

1.2 T h e b o t t o m u p a n d t h e top down a p p r o a c h

In 1876 Wundt founded his psychological laboratory. This is the date that is generally accepted as the birthdate of scientific psychology because the scientific methods and the scientific model of physics have been used to measure

psychological phenomena ever since (Boring, 1950).

The scientific model of the day and the methods associated with it predict the behaviour of objects by generally applicable laws expressed in absolute

measurements within the time-space system and independently of the observer, time and space being defined as independent of each other. The scientific model describes the world as bodies and forces in a space with an independently ticking clock in the background. Thus the fact that we move in a space, experience time and see objects implies, according to this explanation, the postulation of (1) an absolute three-dimensional space on the Cartesian model, (2) an absolute concept of time independent ofthat space, on the Newtonian model in which time is divided

into past, present and future according to the Aristotelian model, (3) objects placed in that space and moved by hypothetical forces, and (4) an objective observer independent of all this (Michaels, & Carello, 1981).

This model and the associated methods are also, and primarily, used in one of the most important disciplines of scientific psychology: perception research. The reasons are obvious: through our senses we come into contact with the world about us. This world can be described in physical terms and, therefore, can be measured within the framework just described. Thus the central problem of perception is: how is physically measurable energy (e.g. reflected light) converted into a percept (e.g. seeing a three-dimensional object). The answer seems simple enough. The reflected light stimulates the ends of the nerves in the retina. This stimulus is converted into neural impulses. At this instant the transition from physiology to psychology takes place, for the observer 'experiences sensations'. Then the

sensations are combined by a central process into a percept. The percept is what we see: the object (Hochberg, 1979).

* If i

?.

I I -

I

r

»■ • " I

This argument can be applied in the case of all senses. In the first three

experiments described in this thesis I confine myself to visual perception. In the fourth an auditory hypothesis is tested.

Here is an example: how do I see a chair? The point-like receptors on my retina are stimulated by the light reflected by the chair: sensations. By an unconscious cerebral process a percept is built up from these sensations and I see a chair. Fig. 1.1 shows a scheme of how this might work.

What is the task of perception psychology? To lay down in laws the relationships between the physically measurable world and the psychological sensations, and to describe the process by which sensations are combined to percepts. It will be clear that two lines of investigation arise.

The first of these lines explaining the relationship between the physical and the phenomenal world is the bottom up approach. In this approach, two branches of investigation are important: psychophysics and psychophysiology.

Psychophysics is the science that describes the relationschip between physical and psychological phenomena in mathematical terms (for an overview, see Stevens,

1975). I give an example on space perception. Baird (1970. p. 211) describes an experiment in which retinal disparity is varied systematically in order to measure the resulting difference in distance impressions perceived by the observers. The physical quantity here is the amount of retinal disparity and the psychological quantity is the amount of spatial impression. The result of this study is that there is a direct correlation between the size of the disparity and the amount of spatial impression. This finding is applied in field-glasses as in Fig. 1.2.

Fig. 1.2 A pair or field-glasses. The retinal disparity is increased by setting the objectives more than 6.4 cm apart. This gives a heightened spatial impression

Psychophysiology, on the other hand, is the science of the relationship between physiological structures and psychological phenomena (for an overview see Grossman, 1967). I give an example on space perception. Ratcliff (1982) describes the effect of brain lesions on space perception. He found that stereopsis was affected by damage to the right hemisphere of the brain.

The two branches have in common that they try to explain how a percept is built up from the bottom upwards under the control of the input.

The second line explaining the relationship between the physical and the phenomenal world is the top down approach. Here we find the general

psychological theories that stress the importance of higher, cognitive processes in perception. It is these higher processes that combine the input into a meaningful percept. For an overview of earlier, more associationist, views, the reader is referred to Boring (1950), and for an overview of recent information processing models to Wessels (1982). What is meant by the terms higher processes, cognition and the like is not always clear. I shall use them just as vaguely as cognitive psychologists themselves do, and will avoid them in describing my own work. In this connection de Gelder (1981) observes that psychologists have as many different definitions of cognition as Eskimos have terms for snow. What is clear, however, is why these higher processes are necessary. It is because the stimulus. the retinal image, is ambiguous. It is therefore necessary to have a higher authority to choose an interpretation and enrich the input. A space perception example of this is given in Fig. 1.3.

Pig. 1.3 A Necker cube can be'interpreted' in iwo (or morel ways: as a flat surface or as a cube Illustrations of this kind are advanced by the classical theory to prove the validity of starting points

1.3 Critique of the classical approach

1.3.1 Introduction

Recently I saw a television programme about the 'biological clock'. Most people have had the experience at some time or another that they have had to get up early for some important engagement and that they have woken up a few moments before the alarm clock was due to go off. The explanation for this could be that we have a sort of internal clock that wakes us on time. The programme showed that much research had already been done on this phenomenon. We were also shown how someone cut open frogs' brains in the search for the exact location of this clock (Why he was not searching for the clock in the frogs' toes will always stay a mystery to me).

I relate this anecdote to illustrate the point that it is a metaphor (here, a clock), or a particular set of suppositions, that determines where we start looking. So it is in perception psychology. If we assume that the unit of analysis for studying the transition between our environment and our experience of that environment is the pointwise stimulation of sensor cells, the research will inevitably boil down to a search for processes that stick this pointwise stimulation back together to make an object. This is why it is necessary to set out the axioms that underly the classical theories of perception.

The basic premiss of every classical theory of perception is that the sensory input is in itself insufficient to arrive at an unambiguous percept. Fig. 1.4 illustrates what is meant by this: equal visual input can produce different percepts. The key question, then, is: how do we arrive at an unambiguous and detailed experience of our environment? This question contains two presuppositions. First, that the pointwise stimulation is the unit of analysis: it is from this that the percept must be constructed. Second, since our experience is richer than the input, the input must be being enriched by some higher .cognitive or computational, process. In other words, meaning is cognitive in origin, it is superimposed upon the input. This is why this approach is called indirect.

Hochberg (1979, p. 10) summarizes the cognitive view as follows: " 1 .

What we perceive is not determined by sensory processes, but by mental structures that are implied by the concept of inference.

2.

Mental structure reflects the structures of the world."

This view contains a circular argument. I shall go into this in more detail in chapter 3.

o

Pig I 4 Wo see surface a, in the lcfl of the drawing This can be a projection of either bore, in the right of the drawing, according to the classical theory

1.3.2 Logical criticisms

The first logical criticism relates to the unit of analysis, the second to the reconstruction models and, in particular, to the cognitive model.

Criticism of the long tradition of cognitive models comes mainly from the logical quarter. For an overview, see Fodor and Pylyshyn (1981) and Turvey, Shaw, Reed and Mace (1981).

The logical criticism is summarized in the following paradoxes <de Gelder, 1981): 1.

Processes of reasoning presuppose premisses. But where do the first premisses come from? Are they innate? Or how can we learn what cannot be learned? The paradox runs as follows: the language of the brain, of thought, is a series of formal rules. Formal rules are impenetrable to content. Therefore, the formal system used in learning and in all information processing cannot itself be learned. Why not?

Because if we use a system of formal rules nothing can be learned that belongs to a formal system that is more powerful than, richer in representations than, or simply different from the system with which we learn. Cognitive development becomes impossible. Turvey, Shaw, Reed and Mace (1981) andTurvey and Carello (1981) claim that the cognitivists have taken out a massive loan from the 'Bank of

Intelligence' without asking about the origin of all that reasoning. This is also why computers cannot learn, which is something that Turing wrote long ago (1959). 2.

A second paradox is comprehension. There is a difference between knowing and comprehension. Comprehension includes the use of what is known, has to do with action. This paradox runs as follows: either a formal theory implies that knowing includes the use of this knowledge (i.e. that all the possible uses are stored in the cognitive structures), or we assume that this is not the case. But the latter means that the knowledge can never be used. Therefore, if behaviour is omitted from the definition of cognition, logically speaking no behaviour is possible. If behaviour is included in the definition, this presupposes an immense storage capacity.

3.

A third problem is embodiment (Mace, 1983). Cognition can be studied as a formal structure independently of a living organism and independently of an

environment. This is what happens in many computer models. This accordingly leaves a great gap between an individual and his environment, and eventually this can lead to solipsism (Fodor, 1980). Perception psychology, then, is concerned with formal structures, and no longer primarily with people and animals in a particular environment.

4.

A fourth problem is that of interpretation. There are two aspects to this: who or what interprets and what is the criterion of the interpretation.

At the end of every internal process there must be something or someone to interpret the result of the process, since a set of formal rules will not, by itself, lead to comprehension. This is made clear by a thought experiment (Searle, 1980, cited in Katz, 1983). Imagine a situation in which someone plays the part of a computer: he sits in a room and through a letterbox receives pieces of pa per covered in Chinese characters. Similar pieces of paper, which have no meaning for him whatever, he passes out again according to an agreed set of rules. The Chinese outside the room, who pushes the notes through the letterbox and receives a

continuous supply of answers, is conducting a conversation with the computer in Chinese. But does the person in the room also speak Chinese? Of course not, all he knows is the set of formal rules. He does not comprehend.

A second aspect of this problem is: how can the result of the comparison between the object and the percept be judged? Surely this can only be done if we first know the object? Is this where the homunculus comes in? This is a problem felt acutely in the computational theories, where feature analysers and all sorts of computing processes are used to construct internally the same object as the one in front of the observer (Marr, 1982). Although Marr avoids the term cognition, his theory still suffers from the drawbacks of the classical approach. His objective, after all, is to draft an information processing model that will provide a workable description of reality based on an 'image'. Since Marr elected for a description of images, be it a symbolic description, he landed up in the cognitive camp (Gardner, 1985).

According to Marr (1982) the information processing model goes like this. First an analysis is made of the local geometric structures of the retinal image. This leads to an initial representation: the primal sketch. Then comes a second analysis which leads to &2.5-D sketch. This sketch still depends on the point of view adopted by the observer, which is why it is called 2.5-D. According to Marr, it is necessary,

however, to arrive at a representation that is completely independent of the point of view adopted by the observer. After a third analysis this brings us to a 3-D representation, the percept. The mathematical algorithms that describe this transformation from image to 3-D representation differ from processes of reasoning only in that they are expressed mathematically, are more formalized. Logically, however, they have the same status as the processes of reasoning and are thus open to the criticisms formulated above.

A supplementary problem is perception in animals. Are we to assume that

cognition and reasoning play a part here too? There is no evidence for the existence of cognitive processes other than in primates (see Premack, 1983, for an overview). Studies with young animals and babies show that even very young organisms can experience space from the time they can move (Hochberg, 1978). Walk and Gibson (1961) have shown that babies and young animals will not crawl over a visual cliff. These paradoxes lie enclosed in the scientific model and the associated methods. Newtonian physics describes the world in terms of bodies and forces acting on those bodies. All these forces and bodies, together with the absolute time-space

also implies a high degree of realism: what we observe objectively is a true reflection of reality. Implicitly, this also implies that we measure reality: an idea which still permeates the world view of most people. Water boils at 100 degrees Celsius, and this measurement, 100 degrees, is the 'real' reality. Hence, too, the great importance that classical perception psychology attaches to illusions: the difference between what we see and what we physically measure (veridicality). Evidently the match between the percept and what we measure is not perfect: sometimes we perceive 'wrongly'. The scientific and the perceived world view no longer coincide. Of course, we can only arrive at this conclusion if we assume that physics describes the reality. The great disadvantage of this approach is that action, our behaviour, is totally decoupled from perception.

In an illuminating book Zukav (1979) describes the history of physics in this century. Since Einstein formulated his theory of relativity at the beginning of the century physicists have no longer regarded the concepts of space and time as independentof each other. But in addition to this some physicists, sometimes much against their own will, have discovered that the observer evidently does not, after all, stand outside the observed phenomenon. Indeed, physicists such as Bohm even wonder if some phenomena exist at all outside the process of perception: some experiments produce different results depending on whether or not an observer is present. So why should psychology still use the ideas of classical physics?

The conclusion from all this can only be that the social sciences in general, and psychology in particular, need to develop their own research model and their own measuring methods- their own optics, for example -just as they once had to invent their own methods of experimentation, different from those of physics, to cater for the reflexivity of its measurements (Richelle, 1968). A first, and still incomplete, attempt at elaborating a separate model and separate measuring methods for the social sciences was undertaken by J.J. Gibson (1950,1966,1979 etc.). This is the subject of the next chapter of this thesis.

A second point of criticism relates to the unit of analysis. In a work of genius, Stevens (1975) demonstrated that the idea of measuring does not necessarily imply the counting of steps or units. The early phsychophysicists sought a unit of

sensation because they wanted to count. In 1887 von Helmholtz wrote: "... unless the concrete operations could be mirrored by the mathematical laws of additivity, the operations do not qualify as measurement" (quoted in Stevens, 1975, pp. 44-45). But Stevens asserts with powerful logical and empirical arguments that this is erroneous: "Measurement occurs whenever an element is matched, equated, or

conjoined to a n e l e m e n t of a n o t h e r d o m a i n " (ibid. p . 46). So m e a s u r e m e n t does not necessarily imply chopping percepts into little pieces to produce J N D ' s (just noticeable differences), b u t merely the m a t c h i n g of two q u a n t i t i e s .

1.3.3 E m p i r i c a l c r i t i c i s m

T h e idea t h a t relational p a t t e r n s between elements are more i m p o r t a n t t h a n e l e m e n t s themselves arose more or less simultaneously in various places in the late forties a n d e a r l y fifties. In Louvain, Belgium, Michotte (1946) d e m o n s t r a t e d t h a t c a u s a l i t y is perceived directly. In Uppsala. Sweden, J o h a n s s o n (1950) discovered t h a t a few points moving in a specified p a t t e r n are sufficient for the perception of, say, a person w a l k i n g (Fig. 2.1). In the USA Land, the inventor of the Polaroid c a m e r a , discovered in a series of ingenious experiments (Land, 1965, 1967,1977) t h a t it is not the composition of the energy reflected by a p a r t i c u l a r area of colour t h a t d e t e r m i n e s the perception of its colour, b u t t h e ratios between reflected e n e r g i e s of the v a r i o u s colour a r e a s independent of the composition of the

i l l u m i n a t i o n (Fig. 2.2). I shall r e t u r n to this in chapter 2. Also in the USA, Gibson (1950) observed w h e n he h a d to select pilots d u r i n g the w a r t h a t t h e classical theory of perception was totally i n a d e q u a t e to account for a m a n ' s ability to land an aircraft.

W h a t is common to all these insights? Movement cannot be divorced from perception: for Michotte, because otherwise no causality w a s perceived, for J o h a n s s o n , because w i t h o u t m o v e m e n t t h e points were completely meaningless, for Land because he explains colour constancy u n d e r different conditions of i l l u m i n a t i o n and hence d u r i n g movement, and for Gibson because, according to him, i t would be a complete waste of time to study perception p h e n o m e n a w i t h o u t t a k i n g account of m o v e m e n t . Until t h a t time the psychologists of perception had concerned t h e m s e l v e s exclusively with motionless two-dimensional pictures, i.e. with s t a t i c perception. They a s s u m e d t h a t the simpler problems of s t a t i o n a r y i m a g e s would have to be solved first, before more complicated m a t t e r s could be tackled: those of t h r e e - d i m e n s i o n a l perception and motion perception. W h a t they failed to see was t h a t t h e last is easier t h a n the first.

B u t I a n t i c i p a t e my n e x t chapter. Let us first look a t the definition of perception in t h e classical approach.

1.4 D e f i n i t i o n of space p e r c e p t i o n

According to the classical theory, space perception is arrived at through inference (Gregory, 1972).

On the one hand we have a three-dimensional world that can be described in physical and mathematical terms according to the Cartesian systems of axes. This world is projected onto the two two-dimensional retinas. Where does the third dimension come from? It is inferred (the term was first used by von Helmholtz (Gigerenzer, & Murray, 1987)), and is thus cognitive in origin. This inference takes place on the basis of cues in the retinal image: pictorial cues. These are techniques that have been used since the Renaissance to suggest space in a painting. The resultofthis process of inference is the percept: we experience space.

How was this view arrived at? In 1625 Scheiner (Gregory, 1972) discovered that the world was projected onto the retina as an inverted image. And the retina is two-dimensional. The classical approach starts from these two-dimensional images on the two retinas. The third dimension has been lost and must therefore be

reconstructed. The eye, then, is like a camera (obscura). Haber (1978) writes: "One of the most potent though misguiding metaphors in psychology (...) is that the eye is like a camera; particularly, that the retinal image is like a snapshot." A metaphor can be a powerful aid but also a terrible trap.

The computer metaphor is well on the way to causing even greater damage within psychology. It is not because a computer can be given a particular task to solve in a particular way that man also solves that task in that way. To use the working of the computer as a model for the working of, say, the brain, or, even worse, of the human 'mind', carries with it the risk that after a while we shall no longer be able to see the difference between the working of the computer and the working of the brain. A computer represents all information, whether it is in the form of images or words or mathematical formulae, in the same way: that is, in bits. But this does not mean that the brain also represents images and words in the same way, or, indeed, that anything is represented at all. This is why Gibson is so set against this metaphor (Johansson, von Hofsten, & Jansson, 1980b).

I return to my subject: space perception. The definition of space perception in the classical theory is: the reconstruction of the third dimension starting from the retinal projections, using memory and reasoning processes on the basis of pictorial cues or physiological properties of the visual system.

Let us now look a t a number of examples of pictorial cues. Haber (1978) writes: "See any introductory psychology textbook from the past hundred years for an up-to-date list (of pictorial cues)."

- shadows: the volume and distance of objects is clarified by the shadow that they cast and that is cast on them.

- interposition: objects that are closer to the observer can partly or wholly obscure his view of objects that are further away.

perspective, viz.:

- linear perspective: all the lines in a scene lead to a particular perspective point.

Fig. 1.5 This drawing contains several pictorial cues gradients, height in the visual field. known size, interposition, shadow effects, linear perspective etc

- texture gradients: the texture of a surface becomes more dense the further it is from the observer.

- detail perspective: loss of detail at increasing distance.

- heightof position in the visual field: the higher an object appears in the visual field, the further away it appears to be.

- reflection: the way light sources are reflected from the object.

known size: an observer can estimate the distance to an object if he knows its size.

Fig. 1.6 Retinal disparity The projection of the cube on the left relina dilTers slightly from the projection on the right retina

Alongside these pictorial cues the physiological propertiesof the visual system itself can provide information on distance. The most thoroughly studied

physiological property is retinal disparity. Our eyes are about 6.4 cm apart, so that the images projected on the two retinas are slightly different (Fig. 1.6; and the figure caption of Fig. 3). One of the possible explanations is that distances can be 'computed' from these differences.

la)

O

o

o

o

o

o

o

Fig. 1 7 Accommodation: the lens is more convex when a nearer object is focused (a). Convergence: the eyes are more turned when a nearer object is fixated (b)

This property of the visual system, also termed stereopsis, is still regarded by many scientists as the primary mechanism of space perception. To experience space then, it would be necessary to have two eyes. This view is refuted in chapter 3. It should be noted that stereoscopy works only horizontally: vertically objects project identical images onto the retinas.

Spatial information is also conveyed by the convergence and accommodation of the eyes (Fig. 1.7). Though one might be justified in doubting whether convergence does actually convey distance information: after all, in order to converge the eyes have to 'know' the distance of the object to be fixated (Woodworth, & Schlosberg, 1962). Anyway, convergence and accommodation only offer information on distance on objects that are not too far away (Sedgwick, 1986).

Besides all these cues (on the one hand, properties of what is being looked at, and on the other, properties of the visual system) spatial information is also provided by the fact that an organism moves: the interaction between the world and the observer. This view is part of the theory of direct perception, which is discussed in the next chapter. In chapters 3 and 4 the importance of the interaction between observer and environment will be discussed. We shall see that movement is a

2 The direct approach

2.1 I n t r o d u c t i o n

In this chapter I examine the theory of direct perception. I shall first set out the theory, after which I shall discuss a number of experiments in support of it and then look at its drawbacks.

2.2 T h e d i r e c t a p p r o a c h

In discussing the direct theory I draw on the work of Gibson (1979), Michaels and Carello (1981), Bruce and Green (1985) and Mace (1986).

Mace (1986, p. 138), writes: 'The major claims of Gibson's ecological approach to perception are now well known (...):

that the perception of the environment is direct and unmediated by images or representations;

that no form of memory, schemata, or other cognitive structure contributes to perception; that information is 'in the world';

that perception is a matter of extracting invariants of the optic array; that perceiving is more like resonance than it is like 'processing';

that the properties of the environment directly perceived include meaningful properties reflecting the animal's interests and utilities;

that computation is not involved in perceiving." (my subdivision)

This is quite a lot to swallow in one go. What it means will be explained in this section.

When Gibson formulated his new theory of perception (Gibson, 1950, 1966, 1979) he had to avoid the traps of the old theory. His great merit is that he has given the term information a completely new meaning. Information is thougth of as

invariants in the optic array. Let me explain.

The information used by the visual system must be present in the ambient light, the light that surrounds us. Everyone is agreed on that. Gibson asserts that the

ambient light is unambiguously structured. In ordinary language: imagine a room with a light source and a table. The table reflects light in all directions, and the light is again reflected by the walls of the room, and so on; in short, it is structured. The light in the room now contains information about the presence of the table in the room and is thus unambiguously structured. By unambiguously, Gibson means that the light would be structured differently if there were some other object in the room. Since an organism moves in the ambient light, it takes continuously samples of the structured light. These samples inform it directly about the environment, without the need for a reconstruction process, since the information in the ambient light is unambiguous. We saw in the last chapter that the unit of analysis, the point-like stimulation of the retina, unavoidably leads to reconstruction processes. Gibson wonders why something should have to be reconstructed, and, having been reconstructed, why it should need to be interpreted, if the information is already present in the environment. The structured light that reaches our eyes Gibson calls the optic array.

The great difference between this approach and the classical one is that in the classical approach it is assumed that the light that is projected onto the retina is

not unambiguous, and hence that the retinal projections can be interpreted in more

than one way.

The idea of structured light can be understood by looking at a hologram. When a hologram is taken the light that is structured by an object is recorded. If we look at the hologram under a suitable light source, we see the object as if it were actually present. This shows that an object can be fully specified by a record of structured light, without there being any need for an 'image'of that object.

When we move, certain parts of the structured light will change and others will not. The changing parts, the variants, inform us about our own movement relative to a stable environment. The unchanging parts Gibson calls invariants: they inform us about stable properties of the environment. He distinguishes two sorts of invariant: structural and transformational (Michaels, & Carello, 1981). Structural invariants are the relational patterns that remain the same while we move. Transformational invariants describe constant modes of change in time. Invariants are the basis of our perception.

Let me give an example of a structural invariant: when we move in a room the textural relational pattern between the objects in the room will remain the same, regardless of where we stand. These relational patterns are structurally invariant.

On the basis of this structural invariant we perceive the objects as rigid and at a constant distance from each other.

Let me give an example of a transformational invariant: Johansson (1950) found the transformation that enables us to see a moving person. He showed subjects points on a screen: when these points are transformed in a particular relational pattern to each other in space and time, we see a walking or dancing person (Fig. 2.1).

•

Kig 2.1 Johansson's point figures (1950) When the figures are projected in sequence onlo a screen, we see a man walking

According to Cutting (1983,1986), distinguishing between structural and transformational invariants is pointless, since invariants indicate the extent to which a structure does not change under transformation. Thus, according to him, the term transformational invariant is meaningless. I endorse his view and will make no futher distiction between structural and transformational invariants. For a discussion of the mathematical and psychological meaning of the term invariant I refer the interested reader to Cutting (1983).

Observe that, since invariants are defined solely in relation to variants, change (or, in other words, movement) is necessary for the detection of non-change. This has been shown empirically many times, as in the Ganzfeld study (Metzger, 1930; Gibson, 1979; I shall return to this in 2.3).

So Gibson asserts that, when the environment reflects light, this light is structured in an unambiguous way. This structure is described as invariants, and these invariants are picked up by the perceptual system. By perceptual system Gibson

means means the whole continuously and actively exploring organism, not merely the eyes. Gibson also supposes that an organism is capable of picking up this information, this structured light, without interpretation and without cognitive intervention. Organism and environment are attuned to one another, form a unity. Active exploring, the attuning to the environment that we do mainly with our eyes is something that we are hardly ever aware of. When it is done with another sense, on the other hand, then we do notice it. Dogs, and to an even greater extent bats, which are less sensitive to light, are more sensitive to sound: they point their ears constantly to feel the acoustic array. I use the word 'feel' here deliberately, since in common parlance something is only 'real' if it can be touched and felt. Even the classical theory has no difficulty in calling perception by touch 'direct': since there is no medium between object and tactile nerves (Katz, 1925). Just as an object is touched, so, according to the direct theory, the optic or acoustic array can be felt by the visual or acoustic system.

The way in which the information is picked up, according to the direct theory, from the optic or acoustic array, is illustrated by two metaphors: the radio metaphor (Michaels, & Carelio, 1981) and the planimeter metaphor (Runeson, 1977). A radio station broadcasts information by structuring electromagnetic waves and

projecting them into the ether. A radio resonates to the structured waves to which it is tuned. But how does a radio pick up higher structures, i.e. the structured waves, without first isolating the component parts of those waves: that is, without computation? This is clarified by a second metaphor: the polar planimeter. With a polar planimeter it is possible to obtain the surface area of any plane immediately, without any intervening 'lower order' calculations. Likewise there must be some detection mechanism that can immediately, without 'lower order' calculations, detect higher order structures, invariants.

What does an organism see? Since Gibson supposes that an organism and its environment (or niche) form a unity, he must also suppose that this organism is tuned to features of the niche that are useful to it. An organism does not see invariants, it sees affordances (a neologism meaning 'to afford action'). Actions of an organism in a niche presuppose that this organism also perceives the

possibilities available to it for monitoring and controlling these actions. Walking implies that we can differentiate between walkable and non-walkable surfaces, and that we are able to see possible pitfalls. People, for example, are well able to see their ideal step height for a flight of stairs. It turns out to be a constant fraction of the length of their tibia (Warren, & Shaw, 1985). We do not see objects in a room:

we see the affordance of objects. Meaning is no longer something that is pasted over a 'reconstructed object': it is something that is seen directly.

The concept of affordance is not entirely new (Reed, & Jones, 1982, p. 409). The German Gestaltists already referred to Aufforderungscharakter (Lewin), translated into English as valence, or demand character (Koffka). The concepts of'grasp-ableness', 'pick-up-ableness' and 'throw-ableness' were already being used by Tolman (1933, cited in Cutting, 1986). According to Gibson it is this meaning that

is perceived. The form of the object is generally not perceived. It is this meaning, then, that must be described in physical terms, namely as invariants, as a pattern that remains constant through change. It will be clear, I hope, that I refer here not to the semantic but to the perceptual meaning (Shaw, Turvey, & Mace, 1982). By semantic meaning I indicate meaning of a linguistic, not a behavioural, character. Meanings of this kind are usually learned, and thus have to do with cognitive processes (Glass, & Holyoak, 1986). By perceptual meaning or behavioural meaning Gibson means that aspect of perception that makes action possible. In Shaw, Turvey, and Mace (1982) Gibson says: 'The whole idea of affordances is to get rid of the ancient assumption that meanings are attached to sensations by association. We've got to get rid of that ancientmodel that values and meanings are subjective in the sense of being learned by association or reinforcement as attachments to inputs."

This, then, is broadly speaking Gibson's theory, the theory of direct perception. It is direct because the essential information is picked up from the ambient light by the visual system (Michaels, & Carello, 1981). Gibson himself calls his theory

ecological (1979). But this term ecology has meanwhile acquired another meaning besides its original one (the theory of the relationships between organisms and the environment in which they live), namely the protection of the environment (Mace, 1983). I shall avoid the term for this reason. Another commonly used term for the theory is event perception (Johansson, von Hofsten, & Jansson, 1980b). By using this term Johansson stresses the importance of his unit of measurement i .e. events instead of points. However, I prefer the term 'direct' to 'event perception' because 'direct' better expresses the contrast with the classical, indirect theory.

Gibson also calls his approach realistic. Realism is the philosophical school of thought which supposes that there exists a reality independent of perception, and that (to some extent at least) we can perceive that reality. In a remarkable article Katz asserts (1983) that, since Gibson supposes there to be a logical dependence between an organism and its environment, he and his followers (see e.g. Shaw,

Turvey, & Mace, 1982) are not true realists. Realism implies dualism, i.e. logical independence. Logical dependence between organism and environment, then, according to Katz, would be better termed duality. This latter term expresses the importance of the reciprocity between organism and environment and the fact that without that relationship there is nothing at all. Accordingly Katz proposes the term 'direct relativism'. This ties in with the new insights in physics, where researchers are beginning to ask to what extent reality exists independently of the observer (Zukav, 1979).

2.3 E x p e r i m e n t s

The purpose of this section is to show by means of experiments that the direct theory works, even though there are a great many methodological problems still to be overcome (see 2.4). I shall begin by describing a reinterpretation of the Ganzfeld experiments of Metzger( 1930) by Gibson. This reinterpretation explains how Gibson arrived at his concept of structured light. I shall then goon to discuss experiments that were conducted within the classical framework, but which nevertheless produce results that contradict the classical theory and are in agreement with the direct theory (Land, 1965,1967,1977). These experiments make it clear that the relational patterns between the elements are more important than the elements themselves. I then discuss a typical example of an experiment carried out in the framework of the direct theory (Pittenger, & Shaw, 1975; Pittenger, Shaw, & Mark, 1979). Following on from that I shall refer to one of my own experiments.(Overbeeke, 1986). I then discuss an experiment in which the classical and the direct theory are played out against each other (Smets, &

Overbeeke, 1987). Most of the experiments conducted within the framework of the direct theory are concerned with the structural aspects of perception. In this experiment we show that affective aspects of perception can also be perceived directly.

I have chosen these four experiments to illustrate the various new concepts that are introduced by the direct theory.

Gibson supposes that light is structured, and hence that it is intrinsically informative. Where does this notion of structured light come from? Gibson (1979) reinterpreted the Ganzfeld experiments of Metzger (1930). A Ganzfeld is a totally homogeneously painted unstructured surface that completely fills the field of vision of an subject. If the subject is then asked what he can see, the answer is 'a thick fog'. It is easy to experience this phenomenon for oneself by taking an orange

ping-pong ball, cutting it in half along the seam, and placing the two halves on one's eyes. One then sees an orange fog. But how is this possible? Surely the cells in the retina are being stimulated? Certainly: but by unstructured light. When one moves, everything is variant. Therefore, Gibson concludes, it is not the stimulation of the retinal cells that is essential for perception, but the factofwhetherornotthe stimulating light is structured. A distinction must be drawn between energy and information: a distinction that is not drawn in the classical approach at the level of the receptor. With this reinterpretation Gibson provided an elegant demonstration of the importance of structured light.

c

=4"

T

R

k

r

D _

_l]L_

_liLT

r

.

rr^—

Pig. 2.2 An experiment by Land. Two identical 'Mondrians' are being illuminated by a scperate set of red, green and blue projectors with independent briglhness controls. A photometer (left in the drawing) can be pointed at any single area to measure the flux, one wave band at the time, coming to the eye from that area In a typical experiment, the projectors can be adjusted so that a white area in the Mondrian at the left and e.g. a green area in the right Mondrian are both sending the same triplet of radiant energies to the eye Under actual viewing conditions the white area continues to look white and the green continues to look green, because the pattern of reflected light from the white areas and its surrounding areas and the green area and its surrounding areas stay unchanged. If then these surrounding areas are covered, the subject sees the white and the green color as having the same color. Land also developped a mouse that, when moved over a Mondrian, calculates the color of a given area from the pattern of reflected light of the surrounding areas and that area

Land (1965,1967,1977) showed that the ratios of the light reflected by areas of different colour are more important for the perception of colour than the reflected light of individual areas. This is a second important implication of the direct theory: relational patterns are more important than seperate elements. Land demonstrated that the same wavelength, surrounded by other wavelengths, can result in completely different perceptions of colour. This is not explained by the classical trichromatic theory (Michaels, & Carello, 1981), which explains our perception of colour in terms of individual wavelengths. However, according to Land there is one property that IS invariant under transformations obtained through a change in lighting: relational patterns between the wavelength and the surrounding wavelengths. These relational patterns will remain the same in different lighting conditions (Fig. 2.2). These stable relational patterns are the basis of colour perception.

Fig. 2.3 The geometrical transformation pattern of the human skull (upper part of the figure: the circles represent the skull of an adult and the dots the rescaled skull of a baby) as a function of age affords detecting the age This 'growing older' invariant can be applied both to organisms that grow (chick, monkey, dog) and to objects (Volkswagen). Subjects describe the Volkswagens as young or old (Pittenger, Shaw, & Mark , 1979)

W i t h i n t h e f r a m e w o r k of t h e d i r e c t t h e o r y a s e a r c h w a s t h e n e m b a r k e d u p o n to find