Tuesday, January 09, 2007

On Intelligence II: Does Hierarchy Shape Matter?

In my first entry on Jeff Hawkins' On Intelligence, I said it became a filter for an unbelievable number of the things I notice and wonder about...and yet I haven’t added new entries about it. Truth is, it's mostly because I fear it may seem so bizarre to use a single set of ideas as a filter for so many things, that you'll find my selection of blog topics even more oddly random than you already do. But it's time. To kick it off, I'll briefly mention step 1: The Hawkins Memory-Prediction model of the brain focuses on hierarchy. Ignoring the details, parts of your brain get information (typically from the senses) and if it matches what they expect, they do nothing. If it doesn't, they pass the information up the hierarchy. It's sort of like an entry-level worker escalating a new problem to his or her boss. And like the CEO (which, in a sense, you are), you don't have time or mental cycles to stay on top of what's happening at the low levels in the hierarchy, so you generally keep track of what's being passed up the chain. You could say, "Your brain--and therefore, you--only notice something if it differs from expectation." Simple, elegant, and able to explain many of our perceptions.

OK, step 2. The more accurate the predictions are at the lower levels of the hierarchy, the less they have to pass upward. How do the predictions get more accurate? By experiencing information more often, predicting something, and checking results. Practice makes perfect. They get "smarter." Suffice to say this works identically for observing something--as in knowing immediately whether a dirty, brown rock is a clump of dirt, a piece of quartz, or a diamond OR for doing something--like playing a sonata on a violin or designing a brilliant ad for a new soft drink (is there such a thing?).

Step 3. If a talented person spends 18 hours a day playing the violin, the violin-connected parts of their brain will be very smart. The lower levels of the hierarchy won't need to escalate messages often. When this happens, the theory claims, the upper levels of the hierarchy don't just take a vacation. Instead, they think higher thoughts. They look for connections between the things the lower levels are doing. It's like a lucky manager leading such a great team that she gets to spend time focusing on long-term strategy, integration between functions, or new ways to think about everyday tasks. The hierarchy of the dedicated violinist becomes very "deep." It has many levels because what is complicated and "escalated" one day becomes rote and simple the next. Of course, our master violinist will probably be a lousy diamond finder.

Now things get interesting (at least for me ;-). Imagine another person who is a dilettante. He dabbles in a thousand things, paying attention in the moment, but gaining no expertise. The theory would say his brain is constantly passing messages upward. Little is rote. The lower levels have mastered little, so escalation is the norm. You could say this person's brain hierarchy is extremely flat. Poor guy, right? But along the way, he is certainly creating connections and, if the theory is right, his dabbling brain is still making constant predictions. And he's really fun at parties. And since the brain doesn't "know" when it's playing the violin and when it's looking at dirty rocks, it seems clear that experiences in one field will start to inform predictions in others. I'm not speculating that if you look at enough rocks you'll be able to play the violin. But if your brain only has random data, it's going to use it the best way it can.

So here's my quandary: Of these two people, who would you trust to build your kid a treehouse? Or set up your Tivo? Or make you dinner? Or join your bowling team?

If you had them both on your team doing something neither had done before, would you assign them to different types of tasks?

Too ridiculous a question? Ok, then how about just this: What would you encourage for your child? What do you wish was encouraged for you when you were young? (Assuming this sort of thing can be encouraged at all.)

Will a flat hierarchy drive more diverse connections and relationships? Will a deep hierarchy--more practiced in "thinking about thinking"--provide more abstract, in-depth considerations in other fields?

3 comments:

Boris Kazachenko said...

Regarding "On intelligence", I loved the book more than any other, but disagree about bunch of things with Hawkins (I think he is pretty sloppy).

He doesn't seem to see a contradiction in alternatively characterizing a neocortex as a hierarchy of generality & that of novelty, even though it's mutually exclusive: novelty is a lack of known generality.

If the latter, it would be instantly flooded with random noise, as there's plenty of that around & randomness is always novel by definition.

If the former, as I think is the case, & as he implies by stressing the "invariance", then it is the recurrent, not novel, patterns that are sent upwards.
So, we notice the unexpected only if it becomes sufficiently "expected" on lower levels, & we notice the already expected on higher levels if the expectation is made stronger|weaker by match|miss accumulated on lower levels. That's a standart Bayesian approach.

Furthermore, the idea of actually forwarding patterns doesn't seem to be biologically plausible. For example, things like color, contrast, or line orientation are only experienced in corresponding areas of visual cortex, not in it's higher levels or in prefrontal cortex. Rather, the biological higher levels represent recurrence & differences among lower level patterns. - what I call higher orders of their syntax.

Higher-level patterns have greater spatio-temporal scope, which can only be achieved by comparing them accross that scope. That could (& biologically probably is) done by increasing the span of interconnection on lower levels. But conceptually it would be more efficient to have a parallel search on a higher level by the copies of sufficiently recurrent lower-level patterns. This would integrate their search span & provide analytical depth without increasing a reaction time.

Which leads me to your question of the "shape". I was thinking about it myself, perhaps because the shape
of my hierarchy must be seriously abnormal.

Assuming that a neocortex has a fixed volume & resources, there must be a trade-off between the number of pattern-representing units (columns?) & the range of connections among them. Crudely (& perhaps inaccurately), it's a trade-off between grey & white matter. This would correspond to your dichotomy of "deep" vs "flat" hierarchy.

However, in terms of performance this would not be a difference between an expert & a dilettante, as the required trade-off depends on the nature of expertise.
Rather, the trade-off is between speed & precision favored by wider parallel search of a flat hierarchy, & higher generality & analytical depth achieved by a deep hierarchy.
Analytical depth would slow down the reaction time with the number of comparison-projection levels the signal has to go through. But it would also increase the range of consideration that goes into the reaction because each level selects for greater generality.
Notice that I use comparison-projection, which I think is is far more meaningful than Hawkins' memory-prediction :).

So, to get to your next question, what determines the trade-off?
It's probably both genes & upbringing, acting through the level of psychological stress. The stress favors speed vs depth, & instant gratification vs discipline.

My theory of intelligence is introduced on the blog:
http://intelligence-is-it.blogspot.com/

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

Your blog keeps getting better and better! Your older articles are not as good as newer ones you have a lot more creativity and originality now keep it up!

 
Add to Technorati Favorites