Gestalt Isomorphism and the Quantification of Spatial Perception Steven Lehar

Abstract

Scientific theory is necessarily based on certain philosophical assumptions that define the foundations on which that science is built. The philosophical underpinnings are not always apparent in mature sciences, where the correct philosophical groundwork has been established for so long that alternative philosophies appear too absurd for serious consideration. However in the case of sciences in an embryonic state of development, errors in the philosophical foundations can lead to grave errors in the science built on them. Nowhere is this more true today than in the science of mind and brain.

Theories of visual perception can be separated into two classes, depending on their relation to a most significant philosophical distinction, i. e. the distinction between epistemological monism, or naive realism, versus epistemological dualism, or the two- worlds hypothesis. Therefore debates over the relative merits of opposing theories of vision are often at cross-purposes whenever the competing theories are founded on different philosophical assumptions.

Such theories cannot be meaningfully compared without discussion of the differences in the underlying philosophy. According to the naive realist view, the world we see around us is identfied as the objective external world, even though the limitations of our senses and the properties of light allow us to experience only a small subset of the properties of that world. In other words, the naive realist view holds that the world we see is the world itself. This is the natural intuitive understanding of vision that we accept from the earliest days of childhood. The problem with this view however becomes clear on consideration of the role of the eye as the sense organ of vision. For the flow of visual information occurs exclusively in one direction, from the world through the eye to the brain. If the brain is the organ of consciousness, then it cannot in principle experience the world directly, but only indirectly, in response to the two-dimensional images sent to it from the eyes.

This fact is in conflict with our subjective experience of objects and surfaces outside of ourselves, because our conscious experience appears to escape the confines of our physical being, to extend into the external world beyond our sensory receptors. The causal chain of vision therefore refutes the naive realist view of vision, as explained by KÖHLER (1929). It is due to this naive realist view therefore that consciousness is often considered to be somehow mysterious, forever beyond our capacity to comprehend, for there is no known physical mechanism that can possibly account for the external nature of visual experience.

The solution to this paradox was discovered centuries ago by Immanuel KANT (1781) by the principle of epistemological dualism. KANT reasoned that we cannot actually experience the world itself as it is, but only an internal perceptual replica of the world. There are, in other words, two worlds of reality, the nouminal and the phenomenal world. The nouminal world is the objective external world, which is the source of the light that stimulates the retina. This is the world studied by science, and is populated by invisible entities such as atoms, electrons, and invisible forms of radiation. The phenomenal world is the internal perceptual world of conscious experience, which is a copy of the external world of objective reality constructed in our brain on the basis of the image received from the retina. The only way we can perceive the nouminal world is by its effects on the phenomenal world. Therefore the world we experience as external to our bodies is not actually the world itself, but only an internal virtual reality replica of that world generated by perceptual processes within our head.

Curiously this most central issue of vision has not received much attention in recent decades, and failure to understand this most significant issue has led to endless confusion in theories of visual representation. For the naive realist view suggests a very much simplified concept of the nature of the internal representation in vision. In the context of naive realism, introspective examination of the internal representation of vision, i. e. examination of the sensation within one's apparent head while viewing the world, reveals an abstract non-spatial entity as the internal code for external objects. This naive realist perspective therefore makes plausible many of the simplistic models of vision proposed over the centuries, and continues to cause confusion in modern neural network models of visual representation.

One reason for the persistent confusion on this issue is due to the fact that even the description of the causal chain of vision is somewhat ambiguous, since it can be interpreted in two alternative ways. Consider the statement that light from this page stimulates an image in your eye which in turn promotes the formation of a percept of the page. The ambiguity inherent in this statement can be revealed by the question "where is the percept?". There are two alternative correct answers to this question, although each is correct in a different spatial context. One answer is that the percept is up in your head, which is correct in the external or naive realist context of your perceived head being identified with your objective physical head, and since your visual cortex is contained within your head, that must also be the location of the patterns of energy corresponding to your percept of the page.

The problem with this answer however is that no percept is experienced within your head where you imagine your visual cortex to be located. The other correct answer is that the percept of the page is right here in front of you where you experience the image of a page. This answer is correct in the internal spatial context of the entire perceived world around you being within your head. However the problem with this answer is that there is now no evidence of the objective external page that serves as the source of the light. The problem is that the vivid spatial structure you see before you is serving two mutually inconsistent roles, both as a mental icon representing the objective external page which is the original source of the light, and as an icon of the final percept of the page; i. e. the page you see before you represents both ends of the causal chain. And our mental image of the problem switches effortlessly between the internal and external contexts to focus on each end of the causal chain in turn. It is this automatic switching of mental context that makes this issue so elusive, because it hinders a consideration of the problem as a whole.

I propose an alternative mental image to disambiguate the two spatial contexts. I propose that out beyond the farthest things you can perceive in all directions, i. e. above the dome of the sky, and below the solid earth under your feet, or beyond the walls and ceiling of the room you see around you, is located the inner surface of your true physical skull, beyond which is an unimaginably immense external world of which the world you see around you is merely a miniature internal replica. In other words, the head you have come to know as your own is not your true physical head, but only a miniature perceptual copy of your head in a perceptual copy of the world, all of which is contained within your real head in the external objective world.

This mental image is more than just a metaphorical device, for the perceived and objective worlds are not spatially superimposed, as is often assumed, but the perceived world is completely contained within your head in the objective world (KOFFKA 1935, p. 27-36). The advantage of this mental image is that it provides two separate and distinct icons for the separate and distinct internal and external worlds, that can now coexist within the same mental image. This no longer allows the automatic switching between spatial contexts that tends to confuse the issue. Furthermore, this insight emphasizes the indisputable fact that every aspect of the solid spatial world that we perceive to surround us is in fact primarily a manifestation of activity within an internal representation, and only in secondary fashion is it also representative of more distant objects and events in the external world.

The Gestalt Principle of Isomorphism

Gestalt theory is founded on the philosophy of epistomological dualism (KÖHLER 1938, pp 102-141,) For the illusory percepts studied by Gestalt theory, such as the moving light of the apparent motion effect, or the illusory surfaces of the Kanizsa and the Ehrenstein figures, are virtually indistinguishable from actual objects and surfaces in the visual world. These illusions therefore demonstrate that the brain is capable of constructing vivid spatial experiences that appear to consciousness as if they were raw sensations of real objects in the world. This in turn casts doubt on the objective reality of the non-illusory objects and surfaces in the visual world within which the illusory objects appear embedded, indicating that they too are internal copies of external objects and surfaces, rather than being those objects and surfaces themselves. It is this insight into the internal nature of the world we see around us that motivates the Gestalt principle of isomorphism.

The theory of isomorphism was an outgrowth (KÖHLER 1947, p. 57-60) of MÜLLER's psychophysical axiom (MÜLLER 1896) which states that the subjective experience of perception cannot be of higher dimensionality than the neurophysiological state by which that experience is encoded. More generally this concept is simply an expression of the materialist view that the properties of mind and consciousness are a direct consequence of electrochemical interactions within the physical brain. Isomorphism differs subtly from MÜLLER's axiom in that it states explicitly what is only implied by MÜLLER, that in the case of structured experience, equal dimensionality between percept and representation implies similarity of structure or form (KÖHLER 1947, p. 60-63). In the domain of color perception isomorphism is not controversial. Before the advent of neurophysiological confirmation, psychophysical experiments established the fact that the subjective experience of color can be reduced to the three dimensions of hue, intensity, and saturation.

Perceived color therefore is of much lower dimensionality than the corresponding properties of physical light. It would be clearly absurd for example to propose that the neurophysiological mechanism underlying the experience of color should encode any less than three dimensions of information while producing three dimensions of color experience. Curiously, in the realm of spatial perception this very obvious principle has not been accepted in contemporary psychology. Phenomenological examination of spatial perception reveals a world composed of solid volumes bounded by colored surfaces embedded in a spatial void.

Every point on every visible surface is perceived at an explicit spatial location in three- dimensions, and all of the visible points on a perceived object like a cube or a sphere are perceived simultaneously in the form of continuous surfaces in depth. Furthermore, the perception of multiple transparent surfaces reveals that multiple depth values can be perceived at any spatial location. However proposed models of spatial perception very rarely allow for such an explicit representation of depth. MARR's 21/2-D sketch (MARR 1982) for example encodes the spatial percept as a two-dimensional map of surface orientations, like a two- dimensional array of needles pointing normal to the perceived surface. KOENDERINK & VAN DOORN (1976, 1980, 1982) propose a representation where each point in the two-dimensional map is labeled as either elliptic, hyperbolic, or parabolic, together with a number expressing the Gaussian curvature of the perceived surface at that point.

TODD & REICHEL (1989) propose an ordinal map where each point in a two-dimensional map records the order relations of depth and/or orientation among neighboring surface regions. GROSSBERG (1987a,

1987b, McLOUGHLIN & GROSSBERG 1998) proposes a depth mapping based on disparity between two-dimensional left and right eye maps. None of these compressed representations are isomorphic with our subjective perception of a full volumetric depth world. In particular, all of these representations have a problem with encoding multiple surfaces at different depths, as in the perception of transparency, or encoding the volume of empty space that is perceived between the observer and a visible surface.

Naive Realism in Neural Network Theory

There are two possible approaches to the investigation of visual processing, a bottom-up approach by studying the elements of neurocomputation, and a top-down approach by studying the nature of the subjective experience of vision. Eventually these two approaches must meet somewhere in the middle, although to date, the gap between them remains as wide as ever. Neurophysiological studies of the visual cortex in experimental animals suggest a hierarchical visual representation composed of different levels of "feature detectors', i. e. cells that respond to the presence of particular features in the visual field. This concept of visual representation has served as a primary motivation behind many neural network models of vision (MARR 1982, BIEDERMAN 1987, HUBEL 1988).

Neural network theory suggests therefore that the internal visual representation is an abstraction or reduced dimensionality encoding of the objects and surfaces in the phenomenal world. The notion of perception by abstraction is supported by the practice of information compression, for example as used in digital image processing. The principle behind this kind of compression is the elimination of redundancy, either in the form of repeated values, or repeated sequences or patterns. For example images containing large regions of uniform brightness can be encoded in terms of the contrast along the edges bounding those regions, from which the brightness of the region can be reconstructed when necessary. In fact the representation of retinal ganglion cells appears to express exactly this kind of compressed image, since ganglion cells respond only along image edges, or spatial transitions of brightness in the visual field, and produce no response within regions of uniform brightness. ATTNEAVE (1954) suggests that the Gestalt principles of similarity, proximity, good continuation, symmetry etc. represent regularities in the visual world that offer an opportunity for information compression, to reduce to manageable proportions the overwhelming complexity of the visual world.

For example a regular geometrical form can be encoded by its vertices only, which define the limits of the straight portions between them by the property of good continuation, just as the edges define the limits of the two-dimensional regions of uniform brightness that they separate. In some sense therefore the compressed representation encodes the same information as the full brightness image in which that information is expressed in redundant form, i. e. with complete boundaries separating regions explicitly painted in with repeated brightness values. However the abstracted or reduced representation, while undoubtedly an essential component of perception, is not sufficient by itself to account for the nature of visual experience. For the subjective experience of perception is not of an edge image, but of a filled-in surface brightness image. If the retinal ganglion cells do in fact encode only transitions of brightness across image edges, then some process downstream of the retinal image must reverse the process and fill in the surface brightness values to account for the subjective experience of visual perception. In fact the identification of this constructive or generative aspect of perception represents one of the most significant contributions of Gestalt theory.

Perceptual Modeling v. s. Neural Modeling

One reason for the reluctance to accept a volumetric model of spatial perception is the apparent lack of neurophysiological evidence, given the two-dimensional structure of the visual cortex. KÖHLER himself felt it necessary to propose a radical model of neural representation in the form of an electric field theory (KÖHLER & HELD 1949) to account for the spatial nature of perception. According to field theory, the subjective percept of spatial structure is correlated with electric fields in the brain whose spatial pattern mirrors the spatial structure of the perceived world. KÖHLER's field theory was eventually disproven, at least in the specific formulation he proposed. Unfortunately the refutation of KÖHLER's field theory has been generally perceived as an indictment of the principle of isomorphism itself. However the validity of isomorphism stands independent of any specific neural hypothesis. If KÖHLER's field theory cannot be verified neurophysiologically, then some other mechanism of spatial representation must be sought that is isomorphic with the experience of spatial perception.

If the neural network paradigm of visual representation in terms of spiking neurons and spatial receptive fields cannot be resolved with the principle of isomorphism, then it is our notions of neural representation that are in need of revision, not the principle of isomorphism. The question remains therefore how are we to model perception in the absence of a viable neurophysiological theory to supply the basic elements or building blocks for a model of perception? I propose a perceptual modeling approach, i. e. to model the percept as observed subjectively rather than the neurophysiological mechanism by which it is supposedly subserved. In other words the perceptual model should be expressed in terms of solid volumes bounded by colored surfaces embedded in a spatial void, as observed in visual experience. This perceptual modeling approach must eventually converge with theories of neural representation, at which point it will be possible to relate the perceptual variables of color and shape to neurophysiological variables such as voltages or spiking frequencies as required. In fact, until a mapping is established between subjective experience and the neurophysiological state, a perceptual model is the only valid model to match to psychophysical data, which explicitly measures the subjective experience of perception rather than the corresponding neurophysiological state.

A Quantitative Phenomenology

Given the insights developed above, the dimensions of conscious experience can be established by direct phenomenological observation, just as were the dimensions of color perception. Since colored surfaces can be perceived at any location through a range of depths, and since transparent surfaces can be perceived simultaneously at multiple depths, the data structure required to encode the information of spatial perception must involve a volumetric manifold representing external space. Every point or region in that manifold can be in one of two states, transparent or opaque, and regions that are in the opaque state also take on a three-dimensional color value expressed in terms of hue, intensity, and saturation. The presence in this manifold of an opaque region encoding a particular color value is therefore by definition equivalent to a subjective experience of a colored surface at the corresponding location in phenomenal space, whether that experience is perceptual, i. e. a veridical effigy of an external surface, or illusory as in the case of dreams or hallucinations. This is exactly the model of spatial perception suggested by KANT when he says "On the occurence of a color-sensation [one's mind] reacts by producing a perceptual experience in which one is immediately presented with a color as pervading a certain region at a certain external position. All the regions which a color can ever be presented to one as occupying ... constitute a single three-dimensional spatial system."(BROAD 1978 p. 29).

Given this kind of explicit spatial representation of subjective experience, the function of visual perception can now be expressed as a transformation from the two-dimensional visual input (or pair of two- dimensional images in the binocular case) to a solid three-dimensional volumetric representation of the spatial percept generated by that input. Whatever the neurophysiological reality of the perceptual mechanism, at least this information must be encoded neurophysiologically to account for the subjective experience of spatial perception. Merely expressing the problem in these terms eliminates a number of commonly accepted models of spatial representation.

Boundedness

This kind of phenomenological analysis of spatial perception immediately raises several fundamental issues about the required representation. One issue is the question of boundedness, i. e. how an explicit spatial representation can encode the infinity of external space in a finite volumetric system. The solution to this problem can be found by inspection. For phenomenological observation reveals that perceived space is not infinite, but is bounded. This can be seen most clearly in the night sky, where the distant stars produce a dome-like percept that presents the stars at equal distance from the observer, and that distance is perceived to be less than infinite. The lower half of perceptual space is usually filled with a percept of the ground underfoot, but it too becomes hemispherical when viewed from far enough above the surface, for example from an airplane or a hot air balloon. The dome of the sky above, and the bowl of the earth below therefore define a finite approximately spherical space (HEELAN 1983) that encodes distances out to infinity within a representational structure that is both finite and bounded. While the properties of perceived space are approximately Euclidean near the body, there are peculiar global distortions evident in perceived space that provide clear evidence of the phenomenal world being an internal rather than external entity.

Consider the phenomenon of perspective, for example how railroad tracks viewed in perspective appear to converge to a point in the distance. The reason why they converge has nothing to do with their objective geometrical arrangement, for parallel lines neither converge, nor do they meet at a point. However in perceived space the tracks are observed both to converge and to meet at a point, and that point is perceived at a finite distance beyond which the tracks are no longer represented. This property of perceived space is so familiar in everyday experience as to seem totally unremarkable. And yet this most prominent violation of Euclidean geometry offers clear evidence for the non-Euclidean nature of perceived space. For the two rails are perceived to be straight and parallel throughout their length, even though they are also perceived to meet at a point up ahead and behind, while at the same time passing to either side of a percipient standing between them. The tracks must therefore in some sense be perceived as being bowed, and yet while bowed, they are also perceived as being straight. This can only mean that the space itself must be curved.

The curved properties of perceived space have been quantified in psychophysical experiments dating to observations by HELMHOLTZ (1925). Subjects in a dark room were presented with a horizontal line of point lights at eye level in the frontoparallel plane, and instructed to adjust their displacement in depth until they were perceived to lie in a straight line in depth. The resultant line of lights curves inwards towards the observer, the amount of curvature being a function of the distance of the line of lights from the observer. The HILLEBRAND- BLUMENFELD alley experiments (HILLEBRAND 1902, BLUMENFELD 1913) extended this work with different configurations of lights, and mathematical analysis of the results (LUNEBURG 1950, BLANK 1958) characterized the nature of perceived space as Riemannian with constant Gaussian curvature  (see GRAHAM 1965 and FOLEY 1978 for a review). In other words, perceived space bows outward around the observer, as seen in the bowed railway tracks.

The observed warping of perceived space is exactly the property that allows the finite representational space to encode an infinite external space. This property is achieved by using a variable representational scale, i. e. the ratio of the physical distance in the manifold relative to the distance in external space that it represents. This scale is observed to vary as a function of distance from the center of the manifold, such that objects close to the body are encoded at a larger representational scale than objects in the distance, and beyond a certain limiting distance the representational scale, at least in the depth dimension, falls to zero, i. e. objects beyond a certain distance lose all perceptual depth. This is seen for example where the sun and moon and distant mountains appear as if cut out of paper and pasted against the dome of the sky.

LEHAR & McLOUGHLIN (1998) propose a transformation to perceptual space using a polar coordinate system centered on the percipient, in which azimuth and elevation angles are preserved, but the radial distance is encoded in terms of vergence, or angle of convergence between eyes in a binocular system. In other words, point P(a, b, r) in Euclidean space is transformed to point Q(a, b,(pi-v)) in perceptual space, where a and b represent azimuth and elevation angles, while the radial distance r is compressed to the vergence representation v by the equation

V = 2 atan(1/2r)

The vergence measure maps the infinity of Euclidean distance to a finite bounded range, as suggested in Figure 1a.

A vergence representation maps Euclidean distince into a finite bounded range. Since azimuth and elevation angles are also closed dimensions, this transformation maps the infinity of Euclidean space into a finite spherical space as suggested in Figure 1b.

In a polar coordinate system the vergence measure of radial distance maps the infinity of Euclidean space into a bounded spherical representation. The outer surface of the sphere represents perceptual infinity. Figure 1c shows how such a compression of the depth dimension would encode the visual space around a man walking down a road.