Before 1830 (short of making
an actual working model, or toy theater), there was no way of making representations
or images that had movement. Nor would it have occurred to anyone that
a sensation or illusion of movement
could be conveyed by still pictures.
How could pictures convey movement if they had none themselves? The very
idea was paradoxical, a contradiction. But the zoetrope proved that individual
images could be fused in the brain to give an illusion of continuous motion,
an idea that was soon to give rise to the motion picture.
Zoetropes (and many other similar devices, with a variety of names) were extremely popular in James's time, and few middle-class Victorian households were without one. All of these instruments contained a drum or disc on which a series of drawings—of animals moving, ball games, acrobats in motion, plants growing—was painted or pasted. The drawings could be viewed one at a time through radial slits in the drum, but when the drum was set into motion, the separate drawings flicked by in rapid succession, and at a critical speed, this suddenly gave way to the perception of a single, steady moving picture. When one slowed the drum again, the illusion vanished. Though zoetropes were usually seen as toys, providing a magical illusion of motion, they were originally designed (often by scientists or philosophers) with a sense that they could serve a very serious purpose: to illuminate the mechanisms both of vision and of animal motion.
Had James been writing a few years later, he might indeed have used the analogy of a motion picture. A movie, with its taut stream of thematically connected images, its visual narrative integrated by the viewpoint and values of its director, is not at all a bad metaphor for the stream of consciousness itself. And the technical and conceptual devices of cinema—zooming, fading, dissolving, omission, allusion, association and juxtaposition of all sorts—rather closely mimic (and perhaps are designed to mimic) the streamings and veerings of consciousness.
It is an analogy that Henri Bergson used twenty years later, in his 1908 book Creative Evolution, where he devoted an entire section to "The Cinematographic Mechanism of Thought, and the Mechanistic Illusion":
We take snapshots, as it were, of the passing reality, and...we have only to string these on a becoming, ...situated at the back of the apparatus of knowledge, in order to imitate what there is that is characteristic in this becoming itself.... We hardly do anything else than set going a kind of cinematograph inside us.... The mechanism of our ordinary knowledge is of a cinematographical kind.Were James and Bergson intuiting a truth in comparing visual perception —and indeed, the flow of consciousness itself—to such a mechanism? Are the brain mechanisms that give coherence to perception and consciousness somehow analogous to motion picture cameras and projectors? Does the eye/brain actually "take" perceptual stills and somehow fuse them to give a sense of continuity and motion? No clear answer was forthcoming during their lifetimes.
The stills may be clear-cut and sharp, and succeed one another without superimposition or overlap, but more commonly they are somewhat blurred, as with a too-long photographic exposure, and they persist for so long that each is still visible when the next "frame" is seen, and three or four frames, the earlier ones progressively fainter, are apt to be superimposed on each other. While the effect is somewhat like that of a film (albeit an improperly shot and presented one, in which each exposure has been too long to freeze motion completely and the rate of presentation too slow to achieve fusion), it also resembles some of E.J. Marey's "chronophotographs" of the 1880s, in which one sees a whole array of photographic moments or time frames superimposed on a single plate. [1]
I heard several accounts of such visual effects while working in the late 1960s with a large number of migraine patients, and when I wrote about this in my 1970 book Migraine, I noted that the rate of flickering in these episodes seemed to be between six and twelve per second. There might also be, in cases of migraine delirium, a flickering of kaleidoscopic patterns or hallucinations. (The flickering might then accelerate to restore the appearance of normal motion or of a continuously mod- ulated hallucination.) Finding no good accounts of the phenomenon in the medical literature—perhaps not entirely surprising, for such attacks are brief, rare, and not readily predicted or provoked—I used the term "cinematographic" vision for them; for patients always compared them to films run too slow.
This was a startling visual phenomenon, for which, in the 1960s, there was no good physiological explanation. But I could not help wondering then whether visual perception might in a very real way be analogous to cinematography, taking in the visual environment in brief, instantaneous, static frames, or "stills," and then, under normal conditions, fusing these to give visual awareness its usual movement and continuity—a "fusion" which, seemingly, was failing to occur in the very abnormal conditions of these migraine attacks.
Such visual effects may also occur in certain seizures, as well as in intoxications (especially with hallucinogens such as LSD). And there are other visual effects that may occur. Moving objects may leave a smear or wake in the direction they move; images may repeat themselves; and afterimages may be greatly prolonged. I have experienced this myself, following the drinking of sakau, a hallucinogen and intoxicant popular in Micronesia. I described some of these effects in a journal, and later in my book The Island of the Colorblind:
Ghost petals ray out from a flower on our table, like a halo around it; when it is moved...it leaves a slight train, a visual smear...in its wake. Watching a palm waving, I see a succession of stills, like a film run too slow, its continuity no longer maintained.
These standstills were especially severe with one patient, Hester Y.
Once I was called to the ward because Mrs. Y. had started a bath, and there was now a flood in the bathroom. I found her standing completely motionless in the middle of the flood.
She jumped when I touched her, and said, "What happened?"
"You tell me," I answered.
She said that she had started to run a bath for herself, and there was an inch of water in the tub...and then I touched her, and she suddenly realized that the tub must have run over and caused a flood. But she had been stuck, transfixed, at that perceptual moment when there was just an inch of water in the bath.Such standstills showed that consciousness could be brought to a halt, stopped dead, for substantial periods, while automatic, nonconscious function -maintenance of posture or breathing, for example- continued as before.
Another striking example of perceptual standstill could be demonstrated with a common visual illusion, that of the Necker cube. Normally, when we look at this ambiguous perspective drawing of a cube, it switches perspective every few seconds, first seeming to project, then to recede, and no effort of will suffices to prevent this switching back and forth. The drawing itself does not change, nor does the retinal image. The switching is a cortical process, a conflict in consciousness itself, as it vacillates between alternative perceptual interpretations. This switching is seen in all normal subjects, and can be observed with functional brain imaging. But a post-encephalitic patient, during a standstill state, may see the same unchanging perspective for minutes or hours at a time.
The normal flow of consciousness,
it seemed,
Thus I was primed to be further fascinated when, in 1983, Josef Zihl and his colleagues in Munich published a single, very fully described case of motion blindness: a woman who became permanently unable to perceive motion following a stroke.
(The stroke had damaged the highly specific areas of the visual cortex which physiologists have shown in experimental animals to be crucial for motion perception.) In this patient, whom they call L.M., there were "freeze frames" lasting several seconds, during which Mrs. M. would see a prolonged, motionless image and be visually unaware of any movement around her, though her flow of thought and perception was otherwise normal.There are clear differences between cinematic vision and the sort of motion blindness described by Zihl; and perhaps between these and the very long visual and sometimes global freezes experienced by some post-encephalitic patients.
For example,Such a condition was utterly bewildering, and sometimes quite dangerous.
- Mrs. M. might begin a conversation with a friend standing in front of her, but not be able to see her friend's lips moving or facial expressions changing. And if the friend moved around behind her, Mrs. M. might continue to "see" him in front of her, even though his voice now came from behind.
- She might see a car "frozen" a considerable distance from her, but find, when she tried to cross the road, that it was now almost upon her;
- she would see a "glacier," a frozen arc of tea coming from the spout of the teapot, but then realize that she had overfilled the cup, and that there was now a puddle of tea on the table.
These differences imply that there must be a number of different mechanisms or systems for the perception of visual motion and the continuity of visual consciousness—and this accords with evidence obtained from perceptual and psychological experiments. Some or all of these mechanisms may fail to work as they should in certain intoxications, some attacks of migraine, and some forms of brain damage—but can they also reveal themselves under normal conditions?In Purves's view, it is precisely this decomposition of what we see into a succession of moments that enables the brain to detect and compute motion; for all it has to do is to note the differing positions of objects between successive "frames," and from these calculate the direction and speed of motion.[3]Dale Purves and his colleagues at Duke University have explored wagon- wheel illusions in great detail, and they have confirmed that this type of illusion or misperception is universal among their subjects. Having excluded any other cause of discontinuity (intermittent lighting, eye movements, etc.), they conclude that the visual system processes information "in sequential episodes," at the rate of three to twenty such episodes per second. Normally, these sequential images are experienced as an unbroken perceptual flow. Indeed, Purves et al. suggest, we may find movies convincing precisely because we ourselves break up time and reality much as a movie camera does, into discrete frames, which we then reassemble into an apparently continuous flow.
- An obvious example springs to mind, which many of us have seen and perhaps puzzled over when watching evenly rotating objects—fans, wheels, propeller blades—or when walking past fences or palings, when the normal continuity of motion seems to be interrupted. Thus, occasionally, as I lie in bed looking up at my ceiling fan, the blades seem suddenly to reverse direction for a few seconds, and then to return equally suddenly to their original forward motion. Sometimes the fan seems to hover or stall, and sometimes to develop additional blades or dark bands broader than the blades.
- It is similar to what happens when, in a film, the wheels of stagecoaches sometimes appear to be going slowly backward or scarcely moving. This wagon-wheel illusion, as it is called, reflects a lack of synchronization between the rate of filming and that of the rotating wheels. But I can have a real-life wagon-wheel illusion even when I look at my fan with the morning sun flooding into my room, bathing everything in a continuous, even light. Is there, then, some flickering or lack of synchronization in my own perceptual mechanisms—analogous, again, to the action of a movie camera?
But this is not enough. We do not merely calculate movement as a robot might—we perceive it. We perceive motion, just as we perceive color or depth, as a unique qualitative experience that is vital to our visual awareness and consciousness.
Something beyond our understanding occurs in the genesis of qualia, the transformation of an objective cerebral computation to a subjective experience.James dreamed of zoetropes as a metaphor for the conscious brain, Bergson of cinematography —but these were, of necessity, no more than tantalizing analogies and images. It has only been in the last twenty or thirty years that neuroscience could even start to address such issues as the neural basis of consciousness.
- Philosophers argue endlessly over how these transformations occur, and whether we will ever be capable of understanding them.
- Neuroscientists, by and large, are content for the moment to accept that they do occur, and to devote themselves to finding the underlying basis or "neural correlates" of consciousness, starting from such elemental forms of consciousness as the perception of motion.
Indeed, from having been an almost untouchable subject before the 1970s, the neuroscientific study of consciousness has now become a central concern, one that engages scientists all over the world. Every level of consciousness is now being explored, from the most elemental perceptual mechanisms (mechanisms common to many animals besides ourselves) to the higher reaches of memory, imagery, and self-reflective consciousness.
It is now possible to monitor simultaneously the activities of a hundred or more individual neurons in the brain, and to do this in unanesthetized animals given simple perceptual and mental tasks. We can examine the activity and interactions of large areas of the brain by means of imaging techniques like functional MRIs and PET scans, and such non-invasive techniques can be used with human subjects, to see which areas of the brain are activated in complex mental activities.
In addition to physiological studies, there is the relatively new realm of computerized neural modeling ["brain-based devices" (in cache), see also Brain-Based Devices], using populations or networks of virtual neurons, and seeing how these organize themselves in response to various stimuli and constraints.
From Gerald Edelman: "From Brain Dynamics to Consciousness: A Prelude to the Future of Brain-Based Devices", Lecture 1 of 12 of IBM Research's Almaden Institute Conference on Cognitive Computing, June 2006 (in acamedia cache: Flash Video (167 MB), QuickTime Movie (610 MB), 16. August 2009). |
Movie: A sphere of 100 000 spiking neurons: effect of re-entrant signaling on dynamic clustering of neurons. Black dots are 100 000 spiking neurons connected by re-entrant signaling. When the neurons are exposed to some input (here white noise) re-entrant signaling generates patterns among the neurons, and these patterns have spacial structure and half-live. In selectional systems like this one physical space and time are non-existent (as demonstrated above) and are replaced by such spacial and temporal structures (G. Edelman, The theory of neural Darwinism, Part 3, Web Of Stories, July 2005).
|
All of these approaches, along with
concepts not available to earlier generations, now combine to make the
quest for the neural correlates of consciousness the most fundamental and
exciting adventure in neuroscience today.
Instead of seeing the brain as rigid, fixed in mode, programmed like a computer, there is now a much more biological and powerful notion of "experiential selection," of experience literally shaping the connectivity and function of the brain (within genetic, anatomical, and physiological limits, of course).
Such a selection of neuronal groups (groups consisting of perhaps a thousand or so individual neurons), and its effect on shaping the brain over the lifetime of an individual, is seen as analogous to the role of natural selection in the evolution of species; hence Gerald M. Edelman, who was a pioneer in such thinking in the 1970s, speaks of "neural Darwinism". J.P. Changeux, the French neuroscientist, is more concerned with the connections of individual neurons, and speaks of "the Darwinism of synapses." Both Changeux and Edelman will soon publish highly readable, general accounts of their work.
William James himself always insisted that consciousness was not a "thing" but a "process." The neural basis of these processes, for Edelman, is one of dynamic interaction between neuronal groups in different areas of the cortex (and between the cortex and the thalamus, and other parts of the brain). He speaks here of "re-entrant" (i.e., reciprocal) interactions, and sees consciousness as arising from the enormous number of such interactions between memory systems in the anterior parts of the brain and systems concerned with perceptual categorization in the posterior parts of the brain.
I was startled when I first came across this passage a few months ago, because their formulation seemed to rest upon the same notion of consciousness that James and Bergson had intimated a century ago, and that had been in my mind since I first heard accounts of cinematic vision from my migraine patients in the 1960s. Here, however, was something more, a possible substrate for consciousness based in neuronal activity.
But the "snapshots" that Crick and Koch postulate are not uniform, like cinematic ones. The duration of successive snapshots, they feel, is not likely to be constant; moreover, the time of a snapshot for shape, say, may not coincide with one for color. While this "snapshotting" mechanism for visual sensory inputs is probably a fairly simple and automatic one, a relatively low-order neural mechanism, each visual percept must include a great number of visual attributes, all of which are bound together on some pre-conscious level.[5] How, then, are the various snapshots "assembled" to achieve apparent continuity, and how do they reach the level of consciousness?
While a particular motion, for example, may be represented by neurons firing at a particular rate in the motion centers of the visual cortex, this is only the beginning of an elaborate process. To reach consciousness, this neuronal firing, or some higher representation of it, must cross a certain threshold of intensity and be maintained above it—consciousness, for Crick and Koch, is a threshold phenomenon. To do that, this group of neurons must engage other parts of the brain (usually in the frontal lobes) and ally itself with millions of other neurons to form a "coalition." Such coalitions, they conceive, can form and dissolve in a fraction of a second, and involve reciprocal connections between the visual cortex and many other areas of the brain. These neural coalitions in different parts of the brain "talk" to one another in a continuous back-and-forth interaction. A single conscious visual percept may thus entail the parallel and mutually influencing activities of billions of nerve cells.
Finally, the activity of a coalition, or coalition of coalitions, if it is to reach consciousness, must not only cross a threshold of intensity, but must be held there for a certain time—roughly a hundred milliseconds. This is the duration of a "perceptual moment."[6]
To explain the apparent continuity of visual consciousness, Crick and Koch suggest that the activity of the coalition shows "hysteresis," that is, a persistence outlasting the stimulus. This notion is very similar, in a way, to the "persistence of vision" theories advanced in the nineteenth century.[7] In his Physiological Optics of 1860, Hermann Helmholtz wrote, "All that is necessary is that the repetition of the impression shall be fast enough for the after-effect of one impression not to have died down perceptibly before the next one comes." Helmholtz and his contemporaries supposed that this aftereffect occurred in the retina, but for Crick and Koch it occurs in the coalitions of neurons in the cortex. The sense of continuity, in other words, results from the continuous overlapping of successive perceptual moments. It may be that the forms of cinematographic vision I have described —with either sharply separated stills or blurred and overlapping ones— represent abnormalities of excitability in the coalitions, with either too much, or too little, hysteresis.[8]
Vision, in ordinary circumstances, is seamless and gives no indication of the underlying processes on which it depends. It has to be decomposed, experimentally or in neurological disorders, to show the elements that com- pose it. Thus it is decomposed vision —the flickering, perseverative, time-blurred images experienced in certain intoxications or severe migraines— which above all lends credence to the notion that consciousness is composed of discrete moments.
(Christopher Isherwood starts his Berlin Diary with an extended photographic simile: "I am a camera with its shutter open, quite passive, recording, not thinking. Recording the man shaving at the window opposite and the woman in the kimono washing her hair. Some day, all this will have to be developed, carefully printed, fixed." But we deceive ourselves if we imagine that we can ever be passive, impartial observers. Every perception, every scene, is shaped by us, whether we intend it, know it, or not. We are the directors of the film we are making—but we are, equally, its subjects too: every frame, every moment, is us, is ours— our forms (as Proust says) are outlined in each one, even if we have no existence, no reality, other than this.)But how then do our frames, our momentary moments, hold together? How, if there is only transience, do we achieve continuity?
Finally, then, we come around to Proust's image, itself slightly reminiscent of photography (and even of Hume), that we consist entirely of "a collection of moments," even though these flow into one another like Borges's river.[11]
[2] Music, with its rhythm and flow, can be of crucial importance in such freezings, allowing patients to resume their suddenly arrested flow of movement, perception, and thought. Music sometimes seems able to act as a sort of model or template for the sense of time and movement such patients have temporarily lost, and which they need to regain. Thus a parkinsonian patient in the midst of a standstill may be able to move when music is played. Indeed, they may be completely unable to walk, but able to dance to music. Neurologists intuitively use musical terms here, and speak of parkinsonism as a "kinetic stutter" and normal movement as "kinetic melody." William Harvey, writing in 1627, referred to animal motion as "the silent music of the body."
[3] Whether or not this is so, the brain can also create motion on its own: one can "see" motion when, objectively, there is none, as in the well-known waterfall illusion.
[4] No paradigms or concepts, however original, ever come totally out of the blue. While population-thinking in relation to the brain only emerged in the 1970s, there was an important antecedent twenty-five years earlier, Donald Hebb's famous 1949 book The Organization of Behavior. Hebb sought to bridge the great gap between neurophysiology and psychology with a general theory which could relate neural processes to mental ones, and, in particular, show how experience could modify, in effect shape, the brain. The potential for modification, Hebb felt, was vested in the synapses which connect brain cells to each other—a single cerebral neuron, we now know, can have up to ten thousand synapses, and the brain as a whole has upward of a hundred trillion, so the capacities for modification are practically infinite. Hebb's original concept was soon to be confirmed, and set the stage for new ways of thinking. Every neuroscientist who now thinks about consciousness is thus indebted to Hebb.
[5] The mechanisms of binding seem to entail the synchronization of neuronal firing in a range of sensory areas. Sometimes it may fail to occur, and Crick cites a comic instance of this in his remarkable 1994 book The Astonishing Hypothesis: "A friend walking in a busy street 'saw' a colleague and was about to address him when he realized that the black beard belonged to another passerby and the bald head and spectacles to another."
[6] The term "perceptual moment" was first used by the psychologist J.M. Stroud in the 1950s, in his paper on "The Fine Structure of Psychological Time." The perceptual moment represented for him the "grain" of psychological time, that duration (about a tenth of a second, he estimated from his experiments) which it took to integrate sensory information as a unit. There was some thought at this time that the alpha rhythms of the brain might be connected with the underlying neurological mechanism for such perceptual moments, since its "ticks" also followed one another at intervals of roughly a tenth of a second. But, as Crick and Koch remark, Stroud's "perceptual moment" hypothesis was virtually ignored for the next half-century.
[7] In his delightful book A Natural History of Vision, Nicholas Wade quotes Seneca, Ptolemy, and other classical authors, who, observing that a flaming torch swung rapidly in a circle appeared to form a continuous ring of fire, realized that there must be a considerable duration or persistence of visual images (or, in Seneca's term, a "slowness" of vision). An impressively accurate measurement of this duration —as 8/60 of a second—was made in 1765, but it was only in the nineteenth century that the persistence of vision was systematically exploited in such instruments as the zoetrope. It seems too that motion illusions akin to the wagon-wheel effect were well known as much as two thousand years ago.
[8] An alternative explanation, Crick and Koch suggest (personal communication), is that the blurring and persistence of snapshots is due to their reaching short-term memory (or a short-term visual memory buffer) and slowly decaying there.
[9] J.Y. Lettvin and his colleagues at MIT described the experiments demonstrating this in a famous paper called "What the Frog's Eye Tells the Frog's Brain."
[10] Edelman provides the following description in his latest book, Wider Than the Sky: The Phenomenal Gift of Consciousness: "Imagine an animal with primary consciousness in the jungle. It hears a low growling noise, and at the same time the wind shifts and the light begins to wane. It quickly runs away, to a safer location. A physicist might not be able to detect any necessary causal relation among these events. But to an animal with primary consciousness, just such a set of simultaneous events might have accompanied a previous experience, which included the appearance of a tiger. Consciousness allowed integration of the present scene with the animal's past history of conscious experience, and that integration has survival value whether a tiger is present or not. An animal without primary consciousness might have many of the individual responses that the conscious animal has and might even survive. But, on average, it is more likely to have lower chances of survival—in the same environment it is less able than the conscious animal to discriminate and plan in light of previous and present events."
[11]
I would like to acknowledge the great help of Francis Crick, Christof Koch, and Ralph M. Siegel, who have reviewed this article and made many valuable comments.