Look carefully at the text in front of you. Observe the sharp, clean contrast between the black letters and the white background of the paper. Why is it that you do not notice the shapes of the white spaces between the letters? The letters stand out to you in crisp contrast, and there is no possibility of confusing the letters from the spacing between.
The reason for this often-ignored fact is the way the human brain processes visual information. Visual information is processed in layers, each abstracting more complex and subtle information. Why do we notice the outlines of objects so clearly? How can we tell depth even with one eye? The answers for many of these questions lie in the connectivity of the visual system of the human brain. These connections create an emergent system that can represent abstract mathematical models of geometric representation.
Forming an Original Model of Human Vision
In the 1960s, two scientists, David Hubel and Torsten Wiesel of Harvard Medical School, observed a distinct pattern by which mammals observed edges. Using microelectrodes inserted in the primary visual cortex of a cat, they saw that some neurons fired rapidly when the cat was shown a line of a specific angle and not at all when shown lines at other angles. They were able to show that specific neurons, called “simple cells,” detected specific angles in a specific location of the image seen by the eye.
After carefully mapping the location of photoreceptor cells corresponding to various orientations, Hubel and Wiesel found that these cells were organized in a columnar fashion. Different cells tuned to different orientations are arranged in stacks, forming a column. Then all the cells in a specific column correspond to a specific small area of the entire image seen by the eye. The result is that for any small segment of the entire visual image, a single cell in each column fires rapidly and thus represents any edges that might be found in that area. In effect, one of the functions of the primary visual cortex is to recognize lines tangent to straight and curved edges in the images seen by the eye. This finding formed the corpus of their research, which eventually garnered Hubel and Wiesel the 1981 Nobel Prize in Physiology or Medicine.
Light Receptors and Vision
The way that the visual cortex recognizes edges is grounded in the way different neurons are connected. As soon as light hits the rods and cones of a mammalian retina, different light receptors are triggered based on the intensity of the light bombarding that receptor. Connected to the receptors are two types of cells known as on-center and off-center retinal ganglion cells. These names are derived from the fact that they respectively recognize the presence of light or dark spots in their visual fields.
The receptors are connected such that they are active if one cone has light shining on it and the circle of adjacent cones do not have light shining on it. Light on the center cone activates or “polarizes” the on-center cell while light on the surrounding cones deactivate or “depolarizes” the cell. Only when both conditions are met will the on-center cell emit the most activity. Conversely, the off-center cell is active only when the center cone does not have light shining on it and the surrounding cones have light shining on them. Due to the way different cones are interconnected, more abstract features such as light or dark spots can be observed.
Similarly, edge detection based on orientation columns operate by well-designed connections amongst the different on-center and off-center neurons. Each orientation column connects to all of the retinal ganglion cells within its respective visual field, and each neuron connects to different sets of ganglion cells based on the orientation that it is designed to recognize. If a series of ganglion cells corresponding to a line, represented by a linear sequence of dots, fires simultaneously, then their simultaneous activation leads to heightened activity of one specific orientation neuron. What is happening in the brain when one orientation neuron fires within each column can be represented mathematically as calculating lines tangent to all the edges within the visual field. This was the framework described by Hubel and Wiesel, and it is the currently accepted model for how the brain recognizes edges.
Expanding the Model
Though this model elegantly describes the way edges are recognized, it fails to explain how the brain recognizes curves. The problem of curve recognition is how the brain recognizes whether a piece of one curve is a continuation of another. Consider, for example, the rippling curves of hair. Within that segment of the visual field, there will be many parallel edge detectors recognizing the angle of the hair at different positions as described by Hubel and Wiesel’s model. However, the model does not answer how one knows that these two adjacent slopes are in fact part of the same curve. Why do we see rippling hair as flowing rather than granular?
Dr. Steven Zucker, computational neuroscientist at Yale University’s Department of Computer Science, has spent the past twenty years expanding the Hubel and Wiesel model to understand how the brain assimilates more complex visual cues, such as curves, textures, depth, and color. His theory is founded on understanding how connections between different orientations of different columns affect the way curves are perceived. An engineer by training and a computer scientist by trade, he offers a unique perspective to neuroscience. “Many people work down below, in the biology, but I work up here developing mechanisms and theories about how the biology down there comes together,” Zucker said.
Zucker believes the answer first lies in the way curves can be represented. An intuitive definition of a curve might be a collection of points that lie together in a pre-defined two-dimensional space. This is the way Hubel and Wiesel’s model describes curves and is also the traditional Euclidean definition of a curve. However, consider another definition of a curve. At any point on the curve, there exists a tangent, a line that touches the curve at one point. One can just as easily define the curve by describing how this tangent
changes as its position along the curve. This interpretation of curves allows the possibility of constructing a theory that can associate the orientation in one segment of the visual field to another orientation in another field. In fact, this theory predicts that one way to recognize continuous curves is by the existence of horizontal connections between neurons of adjacent orientation columns that recognize similar but not identical angles.
In such a case, the presence of a curve would cause simultaneous firing of these neurons, whose synchronized activity would tune into the presence of a continuous curve rather than separate points. Anatomically, this suggests that in a map of neural connections within a block of several orientation columns, a specific edge detection neuron would have well-defined connections to nearby edge detection neurons. Indeed, independent research found a strong conservation in the connections between neurons of adjacent orientation columns. The standard deviation of the number of connections between neurons targeting different angles was much smaller for those that connected approximately 30° from each other, which was predicted by Zucker’s model. This independent experimental evidence confirms Zucker’s theory and lends credibility to this model for accurately describing the actual algorithm coded in the mammalian visual system.
This theory also has many implications not addressed by Hubel and Wiesel. One facet that Zucker’s theory can explain is monocular depth cues. Depth can be described using the same tangential angle system used for curves, and the rate at which the angles change allows for depth perception in the absence of binocular vision. Even though the full extent of Zucker’s theories have yet to be fleshed out, there have been many applications that are based on his model. Several edge-detection software algorithms, for example, use his model to robustly describe edges and curves. Zucker himself continues to explore the possibilities that the model offers. “Not only can the stuff we do go back and explain the findings of biology, but it can also be used elsewhere, such as for robotic vision.”
About the Author
HENRY ZHENG is a sophomore Molecular Biophysics and Biochemistry major in Pierson College. He currently works in a computational neuroscience lab on fluorescent modeling of calcium ion flux in the mammalian visual system.
Acknowledgements
The author would like to thank Professor Steven Zucker for his work in computational vision and for taking time to explain his research.
Further Reading
“Brain Facts: A Primer on the Brain and Nervous System.” (2008). Soc. for Neuroscience. <https://www.sfn.org/skins/main/pdf/brainfacts/2008/brain_facts.pdf>
Hubel, D. H., Wiesel, T. N. (2005). “Brain and visual perception: The Story of a 25-Year Collaboration.” Oxford University Press, 106.
Zucker, S. W. “Differential Geometry from Frenet Point of View: Boundary Detection, Stereo, Texture and Color.” <www.cs.yale.edu/homes/vision/zucker/Zucker05_DiffGeometry.pdf>