'Stereopsis' (originally) means solid sight and refers to the multitude of sources of information that may be used to determine the structure and layout of the three-dimensional world. Binocular stereopsis refers specifically to the information available from having two eyes (Howard and Rogers 1995). Traditionally, three-dimensional vision has been regarded as a difficult problem for any biological or machine vision system because it is claimed that three-dimensional information is 'lost' in a two-dimensional retinal or camera image (Rock 1984). This view is misleading. A consideration of the geometry shows that there is no information about the depth or distance of an array of infinitely small points in space in the pattern of light reaching a single viewing position (or vantage point). In 1709, the philosopher
Berkeley wrote: 'I think it is agreed by all that distance of itself, and immediately, cannot be seen.' However, the world we live in consists of extended surfaces and it can be shown that these provide three-dimensional information even when there is only a single vantage point. Traditionally, these sources of information have been referred to as painters' 'cues' (such as perspective, interposition, or shading), because they correspond to the techniques that artists have used to represent depth in a two-dimensional painting (Helmholtz 1909). It is important to note that the word 'cue' (a hint or prompt) has the connotation of uncertainty or ambiguity which is consistent with the idea of three-dimensional vision being problematic. Whilst it is true that there are many demonstrations such as the Ames room and Ittelson and Kilpatrick's playing cards (Gregory 1997) which purport to show the unreliability of the pictorial cues, it is important to distinguish between (
a) the nature and availability of the three-dimensional information (the computational theory of three-dimensional vision) and (
b) the characteristics of the mechanisms used to extract that information. The unreliability or ambiguity of any particular 'cue' may be due to the intrinsic unreliability of the information or the characteristics of the particular visual system.
The computational theory of perspective, for example, refers to the geometric fact that the angular size of an object or feature varies inversely with the viewing distance — doubling the viewing distance halves the angular size. It follows that any surface composed of similar-sized texture elements will create a gradient of angular (or image) size — a texture gradient — that provides information about the surface's orientation to the line of sight. Similarly, the amount of light reflected off any matt surface depends on its orientation with respect to the light source and so the spatial changes in light reflected off any surface will depend on the three-dimensional shape of the surface. In both these examples, the information is based on the physics of the situation, just as the particular spectral wavelengths reflected off a surface depend on the physical reflectance characteristics (colour) of the surface. In order to use texture gradient or shading information we have to make certain
assumptions about the nature of the world, such as the approximate homogeneity of size of the texture elements or the reflectance uniformity of the surface, but as long as these assumptions are overwhelmingly true for the particular world we live in, texture gradients and shading should be regarded as sources of information rather than mere 'cues'.
Accordingly, the Ames room and Ittelson and Kilpatrick's playing card demonstrations can be interpreted as showing that what we perceive is entirely
consistent with the perspective or interposition information provided, rather than illustrating the poverty of the 'cues'. The fact that the Ames room is actually trapezoidal or Ittelson and Kilpatrick's playing cards are actually arranged in reversed depth order is quite irrelevant (though they show the importance of object assumptions, which here are incorrect, for these strange objects). If the pattern of light — the optic array — reaching the eye from the Ames room peephole is
identical to that which would be created by a normal rectangular room, then no visual system could ever distinguish between the two and hence this tell us nothing about the characteristics of our perceptual systems.
So far, we have considered just two of the many sources of three-dimensional information available at a single vantage point. If the visual world can be sampled from two or more vantage points, new sources of information become available. First, simple geometry shows that the
differences between the optic arrays created at two spatially separated vantage points provide complete information about the structure of the three-dimensional world (Koenderink and van Doorn 1976). In other words, there is three-dimensional information in the small differences or disparities between the two retinal images that we refer to as binocular stereopsis or binocular parallax. In theory, binocular disparities could provide us with complete information about both the local three-dimensional structure (depth) and the absolute distance to objects in the scene. However, because we are able to converge our eyes onto a particular object (which may be at any distance away from us) so that its image falls on corresponding points in the two retinas, binocular disparities only provide information about the local depth structure, in the absence of information about the convergence distance.
Second, geometry also shows that there is three-dimensional information in the
changing optic array when the vantage point is moved, which we refer to as motion parallax (Koenderink and van Doorn 1975). Rogers and Graham (1979) provided a convincing demonstration that the human visual system is able to use motion parallax information. The invention of the stereoscope (Fig. 1) by Charles Wheatstone in 1838 provided the first evidence that the human visual system is capable of using the disparities between the two retinal images to judge the three-dimensional structure of objects and scenes. The formal similarity between the information available from two
simultaneous retinal images (binocular stereopsis), on the one hand, and the
succession of retinal images over time (motion parallax), on the other, is reflected in the similarity of our perceptions in the two cases (Rogers and Graham 1979: 82). It could be that the more recently evolved mechanisms of binocular parallax (that depend on the development of forward-facing eyes) utilize similar tricks and strategies to the more ancient mechanisms of motion parallax.
In order to investigate the characteristics and limitations of human binocular stereopsis, it is useful to be able to isolate and manipulate binocular disparities independently of other sources of three-dimensional information. Bela Julesz's invention of random dot stereograms (rds's) in 1959 provided this tool. By using two arrays of random black-and-white dots presented separately to the two eyes, the shape or form of any depicted three-dimensional surface is effectively camouflaged and not visible in either eye's view alone (Fig. 2). Using this technique, Julesz showed conclusively that binocular disparities alone are sufficient to produce a vivid and unambiguous impression of three-dimensional structure and layout. Random dot stereograms were used by Tyler (1974) and Rogers and Graham (1982) to characterize the sensitivity of the human visual system to spatial changes of disparity.
In 1971, Julesz pointed out that the visual system needs to 'know' which features in one eye match with the corresponding features in the other eye's view in order to extract binocular disparities. For a random dot stereogram composed of thousands of identical black-and-white dots, this is not a trivial task and Julesz referred to this as the 'correspondence problem'. To solve the problem,
Marr and Poggio (1976) developed the first of a series of computational algorithms that successfully solved the correspondence problem for rds's. Their solution exploited certain computational
constraints — of similarity, uniqueness, and smoothness — that were derived from the structure of the particular world we live in. Ullman (1979) suggested that structure-from-motion information might be extracted in a similar way from the sequence of images reaching the single eye. The neural basis of binocular stereopsis was first studied by Barlow, Blakemore, and Pettigrew (1967), who identified binocular cells in the visual cortex of the cat which had receptive fields in either the same or disparate regions of the two retinas. stereoscopic vision
Fig. 1. Diagram of Wheatstone's stereoscopic apparatus. Two mirrors at A′,A reflect the drawings at E′,E and produce a 'solid' image when viewed simultaneously from very close range. (From The Stereoscope, by Sir David Brewster (1856).)
Fig. 2.
(Published 2004)— Brian Rogers
Bibliography- Barlow, H. B., Blakemore, C., and Pettigrew, J. D. (1967). 'The neural mechanism of binocular depth discrimination'. Journal of Physiology, 193.
- Berkeley, G. (1709). An Essay Towards a New Theory of Vision. (repr. 1922).
- Gregory, R. L. (1997) Eye and Brain (5th edn.).
- Helmholtz, H. von. (1909). Physiological Optics. (Eng. trans. J. P. C. Southall, 1962).
- Howard, I. P., and Rogers B. J. (1995). Binocular Vision and Stereopsis.
- Julesz, B. (1971). Foundations of Cyclopean Perception.
- Koenderink, J. J., and van Doorn, A. J. (1975). 'Invariant properties of the motion parallax field due to movement of rigid bodies relative to an observer'. Optica Acta, 22.
- — — — — (1976). 'Geometry of binocular vision and a model for stereopsis'. Biological Cybernetics, 21.
- Marr, D., and Poggio, T, (1976). 'Cooperative computation of stereo disparity'. Science, 194.
- Rock, I. (1984). Perception.
- Rogers, B. J., and Graham, M. E. (1979). 'Motion parallax as an independent cue for depth perception'. Perception, 8.
- — — — — (1982). 'Similarities between motion parallax and stereopsis in human depth perception'. Vision Research, 22.
- Tyler, C. W. (1974). 'Depth perception in disparity gratings'. Nature, 251.
- Ullman, S. (1979). The Interpretation of Visual Motion.
