On Intelligence, Part One; A Book Review

The Ongoing Search for the Operating Algorhythm of the Human Neocortex.

            I just finished reading a fascinating little book entitled, On Intelligence, by Jeff Hawkins. Jeff Hawkins is a computer expert who made a great deal of money developing the Palm Pilot and other mobile devices.  And with some of that money, he founded and endowed a neuroscience institute.   He did this because for the past thirty years, neuroscience has been his main passion.  In fact, if he had been offered the right neuroscience fellowship, he would not have spent his career designing computers.

            The mission of his institute, and his own life-long mission, has been to discover the operating algorhythm for the human neocortex.   At one point in his career, he sent a letter to the CEO of Intel and suggested that discovering this information would be one of the greatest scientific discoveries of all time, because once we understood how the brain really thinks, we could duplicate this setup in silicon and have a brain that works like a human only a million times faster.

            The reason there would be an advantage to have a machine think like a human is that we humans process information much more efficiently than any computers we have ever built.  We can process almost any problem in less than one hundred separate steps.  To prove this, Hawkins gives the following example:  Give any human a simple sorting task;  say, looking at photographs and deciding which photos contain a picture of a cat.  If a cat is discovered, the subject presses a button.  The time it takes any normal human to do this is less than a third of a second per photo.  But we know that it takes 3 milliseconds for an individual neuron to fire.  So if the problem can be solved in 300 milliseconds, then there cannot be more than a hundred steps.  Hawkins says we now use thousands of lines of programming to solve even very trivial problems, and the cat picture exercise is not really all that trivial.   We think it's trivial because any four-year-old can do it---but we have yet to build a computer that can do it.

            The Intel CEO replied that he could see the advantage in having such knowledge,  but did not think it wise to put money into it at that time, because the state of the art was such that it would be 25 or 30 years before the technology would exist to do such research effectively.    But that was 30 years ago.   Hawkins says that though we have not yet discovered the algorhythm of the neocortex, we are closing in on it, and he believes that in the next five or ten years, we will probably make this discovery.

            According to Hawkins, there are four ways in which neocortical memory differs from computer memories:  the neocortex stores sequences of patterns, it recalls them auto-associatively, it stores them in a hierarchy, and it stores them in an "invariant" form.   By invariant form, he means a generalized form, so abstract that it captures the essence of all things which belong to the pattern without listing the details.   When you see a dog, you conclude that it is a dog because it matches some general internal image of dog which you carry around.  This is true even though the specific dog you are seeing today is not exactly the same as any dog you have ever seen before. Even if you were at a St Patrick's Day parade and the dog you saw was dyed green,  you would still be sure it was a dog.  Your internal "invariant" representation of dog is so general that it works for any color of dog---even green.   But all of our invariant representations are formed from our experience. These generalized representations are, in fact, stereotypes---Hawkins even used the word stereotypes.   (When people tell us that we have to change our ways of thinking--to get beyond using stereotypes--  we should remember that as long as we are mammals, we can't really do this.  Sometimes we can consciously re-examine our models to see if they are rational, but we cannot remove them from the process.  Our neocortex has no method of processing any information except by comparing each new input to some internal model which we have already formed.  That's how it works, even at a cellular level. )   So how does our cortex take a multitude of very specific images and form this very general, abstract image?   No one knows.  That is still one of the unsolved mysteries of how the neocortex works. 

            When we recall our memories, we do it by auto-association.  All parts of any memory are linked together with other parts of the same memory, and with parts of other memories,  either in the time sequence of occurrence, or by place, or some other linkage.  When we try to repeat a story about something that happened, sometimes the only way we can remember the whole story is to take it from the beginning and recall each part as it happened--because that is how it was stored---one piece at a time.  The great thing about auto-associative memory is that when your current sensory input can only supply part of a pattern, your memory can usually retrieve the rest of it. This is particularly useful in understanding speech.  When you are trying to have a conversation in an area where there is any background noise at all, then your ears don't really capture every part of every word.  But you hear these words anyway because your brain automatically fills in the missing parts.  It uses your vast internal library of invariant representations of  phonemes, words, and whole phrases to do this. Without this ability, human speech might never have evolved.  It works like Autocorrect.  But like Autocorrect, it  sometimes it makes the wrong guess about what is being said.   Remember, just because you clearly remember hearing something does not mean anybody actually said it.

            The main point of reading this book is a chapter entitled, "How the Cortex Works."  In this 70 pages of fairly dense reading, he explains the nuts and bolts of how the neocortex is structured.  He explains that the mammalian brain is arranged in hierarchies stacked upon hierarchies.   From the input of a single nerve fiber connected to a single neuron, to complex thought patterns about life itself,  all information flows through hierarchies, and the hierarchies identify patterns.  At every level, patterns are identified and  stored, sometimes for a few milliseconds, sometimes for a lifetime.  And as information flows up through the hierarchies,  simple patterns are assembled into larger, more complex patterns.  Only at the very highest level are these patterns anything we see or hear or feel consciously.  Most are just millions of bits of light or sound or feeling that make up the raw input necessary to formulate our conscious internal model of the world.   Some patterns are spatial and some are about time-sequence, and some are both.  The input to any single neuron in the cortex is compared to inputs to adjacent neurons, so as  to construct spatial patterns.   And the input at any instant is compared to a series of recent previous inputs, so as to construct a time-sequence pattern.  And whole patterns are compared to previous patterns.   The brain at every level constructs pattern of events---both events in space and events in time--and forwards this information to higher levels of a complex hierarchy.   

            But information does not just flow up to the top of the hierarchy---it also flows back down.    At any level of processing, after a pattern has been identified, information from that pattern first flows up to the next higher level, but then flows back down to the next lower level, to provide the cells at that level a prediction as to what kind of an input is most likely to occur  next. More than anything else, this predictive ability is  what defines human thought.  Higher levels analyze patterns and inform lower levels what to expect next. Hawkins calls this model the "memory/prediction" model.   And if the next input is exactly as expected, then the cell does nothing.  But if the input is different than expected, then it sends an output to the next higher level of the hierarchy.  And therein lies the secret for the fabulous efficiency of the mammalian brain. It doesn't waste resources processing useless information.  It's  like a chain of command in an air defense network, where higher headquarters sends a message to some lonely radar outpost which says, "This is what you should expect to see on your radar screen in a few minutes.  If this is what you see--then take no action.  But if you see anything else, call us!"  Most of the brain is fairly quiet most of the time, because most of our inputs are within parameters that have already been predicted. At any given time, the main fire house in a large city is interested in knowing about the few buildings that are on fire---not about the half million buildings that aren't.  

            Physically,  the human neocortex is just the thin outer covering of the brain.  It has to be wrinkled and convoluted to follow the contours of the brain, but if it were folded out flat, it would be the size of a large dinner napkin, about 20 by 20 inches, and about 2 millimeters thick, about as thick as a stack of six playing cards.  It has six separate layers, each about as thick as one playing card.  Scientists label these layers from 1 to 6, with 6 being the innermost layer and 1 being the outermost layer.    Any one cell feeds data to the cells directly above or below it,  so we may visualize the processing units as "columns" of cells.    Sensory input from below enters at layer 4,  and the impulse travels up its column to layers 2 and 3.   An output from layer 2 or 3  is sent as an input to layer 4 of the next higher level of the hierarchy.   Level 1 has very few cells, but mostly a mass of horizontal fibers passing information laterally.

            But if a column receives a signal input from a higher level of the hierarchy or from adjacent columns, that signal arrives via layer 1, where  it is conveyed horizontally to all appropriate places.  Eventually, it activates synapses in layer 1 of dendrites connected to cells from layers 2, 3 and 5, causing those cells to fire.   Layer 5 acts as an output buffer for sending information to adjacent columns.  Some of the cells in layers 2 and 3 have axons connecting to  synapses of cells in layer 6,  and which can cause them to fire.   Layer 6  acts as an output buffer to send information to lower levels of the hierarchy.   But while the cells in any given column are part of their own local  hierarchy,  most outputs are fed as inputs either to adjacent columns or to some other patch of cortex, all of which are parts of the larger hierarchy.   In processing vision, there are four areas of cortex involved in processing visual input,  labeled V1, V2, V3, and IT.  The raw input is handled by (V1), whereas (IT) produces the complete images which we consciously see and remember.  So if each patch of cortex has six layers,   and  if our vision has a hierarchy that  requires four separate patches of cortex, then there must be many levels of processing in the overall hierarchy of vision.  To visualize this set up,  it may be helpful to imagine the four visual cortex regions, V1 to IT, as if they were cut out and stacked up, one on top of another like pancakes, even though this is not what happens physically.  And while the four areas of the visual cortex make up a hierarchy, each area still has six layers and its own internal hierarchy. 

            Hawkins also shows, briefly, how a column of cells in the neocortex  can form memory.    Some of the cells in layer 2 have thousands of synapses in layer 1.   When one of these cells receives the right combination of inputs from below, it will fire.  If some of its synapses in layer 1  are active when that cell fires, then those synapse connections will be strengthened.  Repeated firing with those same synapses being active will eventually strengthen them to the point that the cell will begin to fire whenever that same combination of active synapses appears,  even if there is no input from below.    When this happens, the cell has "learned" and will   "remember" that it usually fires whenever this combination of synapses is active.  So the pattern that is formed,  with the cell firing and these particular synapses being active,  is now remembered---and the cell can complete this pattern when only part of it is present.

            Hawkins goes on to explain how the same principles operate in the motor cortex.  In the sensory cortex we have sensory information flowing up the hierarchy,  forming patterns that are larger in scale and more general as we near the top of the hierarchy.  And the predictions flow in the opposite direction, becoming smaller in scale and increasingly specific as we descend to the level of individual nerve inputs.   It is the same structure in the motor cortex, except that instead of predictions flowing down the hierarchy, it is muscle commands that become increasingly detailed and specific as they reach individual muscle fibers, while sensory information from those same muscles flows the opposite direction.   

            Hawkins wrote this book in conjunction with veteran science writer Sandra Blakeslee, so it's concisely written in fairly simple prose, with a minimum of jargon.    Most parts of it are fairly easy to follow.  He explains what the object of his quest is, and why he wants to find it.  He discusses why he believes we may  soon be able to build a silicon version of the human neocortex.  Of course, whether we should actually want to do this is another matter entirely----and the subject of my next post.