发信人: cone (大象无形), 板面: AI
标 题: [转载] On Intelligence [zz]
发信站: 飘渺水云间 (Wed Oct 29 18:51:34 2008), 转信
【 原文由 cone 发表于 Simulate 讨论区 】
Jeff Hawkins with Sandra Blakeslee
Contents
Prologue
1. Artificial Intelligence
2. Neural Networks
3. The Human Brain
4. Memory
5. A New Framework of Intelligence
6. How the Cortex Works
7. Consciousness and Creativity
8. The Future of Intelligence
Epilogue
Appendix: Testable Predictions
Bibliography
Acknowledgments
On
Intelligence
Prologue
This book and my life are animated by two passions.
For twenty-five years I have been passionate about mobile computing. In the
high-tech world of Silicon Valley, I am known for starting two companies
, Palm Computing and Handspring, and as the architect of many handheld computers
and cell phones such as the PalmPilot and the Treo.
But I have a second passion that predates my interest in computers— one
I view as more important. I am crazy about brains. I want to understand how
the brain works, not just from a philosophical perspective, not just in
a general way, but in a detailed nuts and bolts engineering way. My desire
is not only to understand what intelligence is and how the brain works,
but how to build machines that work the same way. I want to build truly
intelligent
machines.
The question of intelligence is the last great terrestrial frontier of science
. Most big scientific questions involve the very small, the very large, or
events that occurred billions of years ago. But everyone has a brain. You
are your brain. If you want to understand why you feel the way you do, how
you perceive the world, why you make mistakes, how you are able to be creative
, why music and art are inspiring, indeed what it is to be human, then you
need to understand the brain. In addition, a successful theory of intelligence
and brain function will have large societal benefits, and not just in helping
us cure brain-related diseases. We will be able to build genuinely intelligent
machines, although they won't be anything like the robots of popular fiction
and computer science fantasy. Rather, intelligent machines will arise from
a new set of principles about the nature of intelligence. As such, they
will help us accelerate our knowledge of the world, help us explore the universe
, and make the world safer. And along the way, a large industry will be created
.
Fortunately, we live at a time when the problem of understanding intelligence
can be solved. Our generation has access to a mountain of data about the
brain, collected over hundreds of years, and the rate at which we are gathering
more data is accelerating. The United States alone has thousands of
neuroscientists
. Yet we have no productive theories about what intelligence is or how the
brain works as a whole. Most neurobiologists don't think much about overall
theories of the brain because they're engrossed in doing experiments to
collect more data about the brain's many subsystems. And although legions
of computer programmers have tried to make computers intelligent, they have
failed. I believe they will continue to fail as long as they keep ignoring
the differences between computers and brains.
What then is intelligence such that brains have it but computers don't? Why
can a six-year-old hop gracefully from rock to rock in a streambed while
the most advanced robots of our time are lumbering zombies? Why are three
-year-olds already well on their way to mastering language while computers
can't, despite half a century of programmers' best efforts? Why can you
tell a cat from a dog in a fraction of a second while a supercomputer cannot
make the distinction at all? These are great mysteries waiting for an answer
. We have plenty of clues; what we need now are a few critical insights.
You may be wondering why a computer designer is writing a book about brains
. Or put another way, if I love brains why didn't I make a career in brain
science or in artificial intelligence? The answer is I tried to, several
times, but I refused to study the problem of intelligence as others have
before me. I believe the best way to solve this problem is to use the
detailed biology of the brain as a constraint and as a guide, yet think about
intelligence
as a computational problem— a position somewhere between biology and computer
science. Many biologists tend to reject or ignore the idea of thinking of
the brain in computational terms, and computer scientists often don't believe
they have anything to learn from biology.
Also, the world of science is less accepting of risk than the world of business
. In technology businesses, a person who pursues a new idea with a reasoned
approach can enhance his or her career regardless of whether the particular
idea turns out to be successful. Many successful entrepreneurs achieved
success only after earlier failures. But in academia, a couple of years spent
pursuing a new idea that does not work out can permanently ruin a young
career. So I pursued the two passions in my life simultaneously, believing
that success in industry would help me achieve success in understanding
the brain. I needed the financial resources to pursue the science I wanted
, and I needed to learn how to
affect change in the world, how to sell new ideas, all of which I hoped to
get from working in Silicon Valley.
In August 2002 I started a research center, the Redwood Neuroscience Institute
(RNI), dedicated to brain theory. There are many neuroscience centers in
the world, but no others are dedicated to finding an overall theoretical
understanding of the neocortex— the part of the human brain responsible
for intelligence. That is all we study at RNI. In many ways, RNI is like
a start-up company. We are pursuing a dream that some people think is
unattainable
, but we are lucky to have a great group of people, and our efforts are starting
to bear fruit.
* * *
The agenda for this book is ambitious. It describes a comprehensive theory
of how the brain works. It describes what intelligence is and how your brain
creates it. The theory I present is not a completely new one. Many of the
individual ideas you are about to read have existed in some form or another
before, but not together in a coherent fashion. This should be expected.
It is said that "new ideas" are often old ideas repackaged and reinterpreted
. That certainly applies to the theory proposed here, but packaging and
interpretation
can make a world of difference, the difference between a mass of details
and a satisfying theory. I hope it strikes you the way it does many people
. A typical reaction I hear is, "It makes sense. I wouldn't have thought
of intelligence this way, but now that you describe it to me I can see how
it all fits together." With this knowledge most people start to see themselves
a little differently. You start to observe your own behavior saying, "I
understand what just happened in my head." Hopefully when you have finished
this book, you will have new insight into why you think what you think and
why you behave the way you behave. I also hope that some readers will be
inspired to focus their careers on building intelligent machines based on
the principles outlined in these pages.
I often refer to this theory and my approach to studying intelligence as
"real intelligence" to distinguish it from "artificial intelligence." AI
scientists tried to program computers to act like humans without first answering
what intelligence is and what it means to understand. They left out the
most important part of building intelligent machines, the intelligence!
"Real intelligence" makes the point that before we attempt to build intelligent
machines, we have to first understand how the brain thinks, and there is
nothing artificial about that. Only then can we ask how we can build
intelligent
machines. The book starts with some background on why previous attempts
at understanding intelligence and building intelligent machines have failed
. I then introduce and develop the core idea of the theory, what I call the
memory-prediction framework. In chapter 6 I detail how the physical brain
implements the memory-prediction model— in other words, how the brain actually
works. I then discuss social and other implications of the theory, which
for many readers might be the
most thought-provoking section. The book ends with a discussion of intelligent
machines— how we can build them and what the future will be like. I hope
you find it fascinating. Here are some of the questions we will cover along
the way: Can computers be intelligent?
For decades, scientists in the field of artificial intelligence have claimed
that computers will be intelligent when they are powerful enough. I don'
t think so, and I will explain why. Brains and computers do fundamentally
different things.
Weren't neural networks supposed to lead to intelligent machines?
Of course the brain is made from a network of neurons, but without first
understanding what the brain does, simple neural networks will be no more
successful at creating intelligent machines than computer programs have
been.
Why has it been so hard to figure out how the brain works?
Most scientists say that because the brain is so complicated, it will take
a very long time for us to understand it. I disagree. Complexity is a symptom
of confusion, not a cause. Instead, I argue we have a few intuitive but
incorrect assumptions that mislead us. The biggest mistake is the belief
that intelligence is defined by intelligent behavior.
What is intelligence if it isn't defined by behavior?
The brain uses vast amounts of memory to create a model of the world. Everything
you know and have learned is stored in this model. The brain uses this memory
-based model to make continuous predictions of future events. It is the ability
to make predictions about the future that is the crux of intelligence. I
will describe the brain's predictive ability in depth; it is the core idea
in the book.
How does the brain work?
The seat of intelligence is the neocortex. Even though it has a great number
of abilities and powerful flexibility, the neocortex is surprisingly regular
in its structural details. The different parts of the neocortex, whether
they are responsible for vision, hearing, touch, or language, all work on
the same principles. The key to understanding the neocortex is understanding
these common principles and, in particular, its hierarchical structure.
We will examine the neocortex in sufficient detail to show how its structure
captures the structure of the world. This discussion will be the most technical
part of the book, but interested nonscientist readers should be able to
understand it.
What are the implications of this theory?
This theory of the brain can help explain many things, such as how we are
creative, why we feel conscious, why we exhibit prejudice, how we learn,
and why "old dogs" have trouble learning "new tricks." I will discuss a
number of these topics. Overall, this theory gives us insight into who we
are and why we do what we do.
Can we build intelligent machines and what will they do?
Yes. We can and we will. Over the next few decades, I see the capabilities
of such machines evolving rapidly and in interesting directions. Some people
fear that intelligent machines could be dangerous to humanity, but I argue
strongly against this idea. We are not going to be overrun by robots. It
will be far easier to build machines that outstrip our abilities in high
-level thought such as physics and mathematics than to build anything like
the walking, talking robots we see in popular fiction. I will explore the
incredible directions in which this technology is likely to go.
My goal is to explain this new theory of intelligence and how the brain works
in a way that anybody will be able to understand. A good theory should be
easy to comprehend, not obscured in jargon or convoluted argument. I'll
start with a basic framework and then add details as we go. Some will be
reasoning just on logical grounds; some will involve particular aspects of
brain circuitry. Some of the details of what I propose are certain to be
wrong, which is always the case in any area of science. A fully mature theory
will take years to develop, but that doesn't diminish the power of the core
idea.
* * *
When I first became interested in brains many years ago, I went to my local
library to look for a good book that would explain how brains worked. As
a teenager I had become accustomed to being able to find well-written books
that explained almost any topic of interest. There were books on relativity
theory, black holes, magic, and mathematics—whatever I was fascinated with
at the moment. Yet my search for a satisfying brain book turned up empty
. I came to realize that no one had any idea how the brain actually worked
. There weren't even any bad or unproven theories; there simply were none
. This was unusual. For example, at that time no one knew how the dinosaurs
had died, but there were plenty of theories, all of which you could read
about. There was nothing like this for brains. At first I had trouble
believing
it. It bothered me that we didn't know how this critical organ worked. While
studying what we did know about brains, I came to believe that there must
be a straightforward explanation. The brain wasn't magic, and it didn't
seem to me that the answers would even be that complex. The mathematician
Paul Erdos believed that the simplest mathematical proofs already exist
in some ethereal book and a mathematician's job was to find them, to "read
the book." In the same way, I felt that the explanation of intelligence
was "out there." I could taste it. I wanted to read the book.
For the past twenty-five years, I have had a vision of that small,
straightforward
book on the brain. It was like a carrot keeping me motivated during those
years. This vision has shaped the book you are holding in your hands right
now. I have never liked complexity, in either science or technology. You
can see that reflected in the products I have designed, which are often
noted for their ease of use. The most powerful things are simple. Thus this
book proposes a simple and straightforward theory of intelligence. I hope
you enjoy it.
1
Artificial Intelligence
When I graduated from Cornell in June 1979 with a degree in electrical
engineering
, I didn't have any major plans for my life. I started work as an engineer
at the new Intel campus in Portland, Oregon. The microcomputer industry
was just starting, and Intel was at the heart of it. My job was to analyze
and fix problems found by other engineers working in the field with our
main product, single board computers. (Putting an entire computer on a single
circuit board had only recently been made possible by Intel's invention
of the microprocessor.) I published a newsletter, got to do some traveling
, and had a chance to meet customers. I was young and having a good time,
although I missed my college
sweetheart who had taken a job in Cincinnati.
A few months later, I encountered something that was to change my life's
direction. That something was the newly published September issue of Scientific
American, which was dedicated entirely to the brain. It rekindled my childhood
interest in brains. It was fascinating. From it I learned about the
organization
, development, and chemistry of the brain, neural mechanisms of vision, movement
, and other specializations, and the biological basis for disorders of the
mind. It was one of the best Scientific American issues of all time. Several
neuroscientists I've spoken to have told me it played a significant role
in their career choice, just as it did for me.
The final article, "Thinking About the Brain," was written by Francis Crick
, the codiscoverer of the structure of DNA who had by then turned his talents
to studying the brain. Crick argued that in spite of a steady accumulation
of detailed knowledge about the brain, how the brain worked was still a
profound mystery. Scientists usually don't write about what they don't know
, but Crick didn't care. He was like the boy pointing to the emperor with
no clothes. According to Crick, neuroscience was a lot of data without a
theory. His exact words were, "what is conspicuously lacking is a broad
framework of ideas." To me this was the British gentleman's way of saying
, "We don't have a clue how this thing works."
It was true then, and it's still true today.
Crick's words were to me a rallying call. My lifelong desire to understand
brains and build intelligent machines was brought to life. Although I was
barely out of college, I decided to change careers. I was going to study
brains, not only to understand how they worked, but to use that knowledge
as a foundation for new technologies, to build intelligent machines. It
would take some time to put this plan into action.
In the spring of 1980 I transferred to Intel's Boston office to be reunited
with my future wife, who was starting graduate school. I took a position
teaching customers and employees how to design microprocessor-based systems
. But I had my sights on a different goal: I was trying to figure out how
to work on brain theory. The engineer in me realized that once we understood
how brains worked, we could build them, and the natural way to build artificial
brains was in silicon. I worked for the company that invented the silicon
memory chip and the microprocessor; therefore, perhaps I could interest
Intel in letting me spend part of my time thinking about intelligence and
how we could design brainlike memory chips. I wrote a letter to Intel's
chairman, Gordon Moore. The letter can be distilled to the following:
Dear Dr. Moore,
I propose that we start a research group devoted to understanding how the
brain works. It can start with one person— me— and go from there. I am
confident we can figure this out. It will be a big business one day.
— Jeff Hawkins
Moore put me in touch with Intel's chief scientist, Ted Hoff. I flew to
California
to meet him and lay out my proposal for studying the brain. Hoff was famous
for two things. The first, which I was aware of, was for his work in designing
the first microprocessor. The second, which I was not aware of at the time
, was for his work in early neural network theory.
Hoff had experience with artificial neurons and some of the things you could
do with them. I wasn't prepared for this.
After listening to my proposal, he said he didn't believe it would be possible
to figure out how the brain works in the foreseeable future, and so it didn
't make sense for Intel to support me. Hoff was correct, because it is now
twenty-five years later and we are just starting to make significant progress
in understanding brains. Timing is everything in business.
Still, at the time I was pretty disappointed.
I tend to seek the path of least friction to achieve my goals. Working on
brains at Intel would have been the simplest transition. With that option
eliminated I looked for the next best thing. I decided to apply to graduate
school at the Massachusetts Institute of Technology, which was famous for
its research on artificial intelligence and was conveniently located down
the road. It seemed a great match. I had extensive training in computer
science— "check." I had a desire to build intelligent machines, "check."
I wanted to first study brains to see how they worked…"uh, that's a problem
." This last goal, wanting to understand how brains worked, was a nonstarter
in the eyes of the scientists at the MIT artificial intelligence lab.
It was like running into a brick wall. MIT was the mother-ship of artificial
intelligence. At the time I applied to MIT, it was home to dozens of bright
people who were enthralled with the idea of programming computers to produce
intelligent behavior. To these scientists, vision, language, robotics, and
mathematics were just programming problems. Computers could do anything
a brain could do, and more, so why constrain your thinking by the biological
messiness of nature's computer? Studying brains would limit your thinking
. They believed it was better to study the ultimate limits of computation
as best expressed in digital computers. Their holy grail was to write computer
programs that would first match and then surpass human abilities. They took
an ends-justify-the-means approach; they were not interested in how real
brains worked. Some took pride in ignoring neurobiology.
This struck me as precisely the wrong way to tackle the problem. Intuitively
I felt that the artificial intelligence approach would not only fail to
create programs that do what humans can do, it would not teach us what
intelligence
is. Computers and brains are built on completely different principles. One
is programmed, one is self-learning. One has to be perfect to work at all
, one is naturally flexible and tolerant of failures. One has a central
processor
, one has no centralized control.
The list of differences goes on and on. The biggest reason I thought computers
would not be intelligent is that I understood how computers worked, down
to the level of the transistor physics, and this knowledge gave me a strong
intuitive sense that brains and computers were fundamentally different.
I couldn't prove it, but I knew it as much as one can intuitively know anything
. Ultimately, I reasoned, AI might lead to useful products, but it wasn't
going to build truly intelligent machines.
In contrast, I wanted to understand real intelligence and perception, to
study brain physiology and anatomy, to meet Francis Crick's challenge and
come up with a broad framework for how the brain worked. I set my sights
in particular on the neocortex— the most recently developed part of the
mammalian brain and the seat of intelligence. After understanding how the
neocortex worked, then we could go about building intelligent machines,
but not before.
Unfortunately, the professors and students I met at MIT did not share my
interests. They didn't believe that you needed to study real brains to
understand
intelligence and build intelligent machines. They told me so. In 1981 the
university rejected my application.
* * *
Many people today believe that AI is alive and well and just waiting for
enough computing power to deliver on its many promises. When computers have
sufficient memory and processing power, the thinking goes, AI programmers
will be able to make intelligent machines. I disagree. AI suffers from a
fundamental flaw in that it fails to adequately address what intelligence
is or what it means to understand something. A brief look at the history
of AI and the tenets on which it was built will explain how the field has
gone off course.
The AI approach was born with the digital computer. A key figure in the early
AI movement was the English mathematician Alan Turing, who was one of the
inventors of the idea of the general-purpose computer. His masterstroke
was to formally demonstrate the concept of universal computation: that is
, all computers are fundamentally equivalent regardless of the details of
how they are built. As part of his proof, he conceived an imaginary machine
with three essential parts: a processing box, a paper tape, and a device
that reads and writes marks on the tape as it moves back and forth. The
tape was for storing information— like the famous 1's and 0's of computer
code (this was before the invention
of memory chips or the disk drive, so Turing imagined paper tape for storage
). The box, which today we call a central processing unit (CPU), follows
a set of fixed rules for reading and editing the information on the tape.
Turing proved, mathematically, that if you choose the right set of rules
for the CPU and give it an indefinitely long tape to work with, it can perform
any definable set of operations in the universe. It would be one of many
equivalent machines now called Universal Turing Machines. Whether the problem
is to compute square roots, calculate ballistic trajectories, play games
, edit pictures, or reconcile bank transactions, it is all 1's and 0's
underneath
, and any Turing Machine can be programmed to handle it. Information processing
is information processing is information processing. All digital computers
are logically equivalent.
Turing's conclusion was indisputably true and phenomenally fruitful. The
computer revolution and all its products are built on it. Then Turing turned
to the question of how to build an intelligent machine. He felt computers
could be intelligent, but he didn't want to get into arguments about whether
this was possible or not. Nor did he think he could define intelligence
formally, so he didn't even try. Instead, he proposed an existence proof
for intelligence, the famous Turing Test: if a computer can fool a human
interrogator into thinking that it too is a person, then by definition the
computer must be intelligent. And so, with the Turing Test as his measuring
stick and the Turing Machine as his medium, Turing helped launch the field
of AI. Its central dogma: the brain is just another kind of computer. It
doesn't matter how you design an artificially intelligent system, it just
has to produce humanlike behavior.
The AI proponents saw parallels between computation and thinking. They said
, "Look, the most impressive feats of human intelligence clearly involve
the manipulation of abstract symbols— and that's what computers do too.
What do we do when we speak or listen? We manipulate mental symbols called
words, using well-defined rules of grammar. What do we do when we play chess
? We use mental symbols that represent the properties and locations of the
various pieces.
What do we do when we see? We use mental symbols to represent objects, their
positions, their names, and other properties. Sure, people do all this with
brains and not with the kinds of computers we build, but Turing has shown
that it doesn't matter how you implement or manipulate the symbols. You
can do it with an assembly of cogs and gears, with a system of electronic
switches, or with the brain's network of neurons— whatever, as long as
your medium can realize the functional equivalent of a Universal Turing Machine
."
This assumption was bolstered by an influential scientific paper published
in 1943 by the neurophysiologist Warren McCulloch and the mathematician
Walter Pitts. They described how neurons could perform digital functions—
that is, how nerve cells could conceivably replicate the formal logic at
the heart of computers. The idea was that neurons could act as what engineers
call logic gates. Logic gates implement simple logical operations such as
AND, NOT, and OR.
Computer chips are composed of millions of logic gates all wired together
into precise, complicated circuits. A CPU is just a collection of logic
gates.
McCulloch and Pitts pointed out that neurons could also be connected together
in precise ways to perform logic functions. Since neurons gather input from
each other and process those inputs to decide whether to fire off an output
, it was conceivable that neurons might be living logic gates. Thus, they
inferred, the brain could conceivably be built out of AND-gates, OR-gates
, and other logic elements all built with neurons, in direct analogy with
the wiring of digital electronic circuits. It isn't clear whether McCulloch
and Pitts actually believed the brain worked this way; they only said it
was possible. And, logically speaking, this view of neurons is possible.
Neurons can, in theory, implement digital
functions. However, no one bothered to ask if that was how neurons actually
were wired in the brain. They took it as proof, irrespective of the lack
of biological evidence, that brains were just another kind of computer.
It's also worth noting that AI philosophy was buttressed by the dominant
trend in psychology during the first half of the twentieth century, called
behaviorism. The behaviorists believed that it was not possible to know
what goes on inside the brain, which they called an impenetrable black box
. But one could observe and measure an animal's environment and its behaviors
— what it senses and what it does, its inputs and its outputs. They conceded
that the brain contained reflex mechanisms that could be used to condition
an animal into adopting new behaviors through reward and punishments. But
other than this, one did not need to study the brain, especially messy
subjective
feelings such as hunger, fear, or what it means to understand something.
Needless to say, this research philosophy eventually withered away throughout
the second half of the twentieth century, but AI would stick around a lot
longer.
As World War II ended and electronic digital computers became available for
broader applications, the pioneers of AI rolled up their sleeves and began
programming. Language translation? Easy! It's a kind of code breaking. We
just need to map each symbol in System A onto its counterpart in System
B. Vision? That looks easy too. We already know geometric theorems that deal
with rotation, scale, and displacement, and we can easily encode them as
computer algorithms— so we're halfway there. AI pundits made grand claims
about how quickly computer intelligence would first match and then surpass
human intelligence.
Ironically, the computer program that came closest to passing the Turing
Test, a program called Eliza, mimicked a psychoanalyst, rephrasing your
questions
back at you. For example, if a person typed in, "My boyfriend and I don'
t talk anymore," Eliza might say, "Tell me more about your boyfriend" or
"Why do you think your boyfriend and you don't talk anymore?" Designed as
a joke, the program actually fooled some people, even though it was dumb
and trivial. More serious efforts included programs such as Blocks World
, a simulated room containing blocks of different colors and shapes. You
could pose questions to Blocks World such as "Is there a green pyramid on
top of the big red cube?" or "Move the blue cube on top of the little red
cube." The program would answer your question or try to do what you asked
.
It was all simulated— and it worked. But it was limited to its own highly
artificial world of blocks. Programmers couldn't generalize it to do anything
useful.
The public, meanwhile, was impressed by a continuous stream of seeming successes
and news stories about AI technology. One program that generated initial
excitement was able to solve mathematical theorems. Ever since Plato, multistep
deductive inference has been seen as the pinnacle of human intelligence,
so at first it seemed that AI had hit the jackpot. But, like Blocks World
, it turned out the program was limited. It could only find very simple theorems
, which were already known. Then there was a large stir about "expert systems
," databases of facts that could answer questions posed by human users. For
example, a medical expert system might be able to diagnose a patient's disease
if given a list of symptoms. But again, they turned out to be of limited
use and didn't exhibit anything close to generalized intelligence.
Computers could play checkers at expert skill levels and eventually IBM's
Deep Blue famously beat Gary Kasparov, the world chess champion, at his
own game. But these successes were hollow. Deep Blue didn't win by being
smarter than a human; it won by being millions of times faster than a human
. Deep Blue had no intuition. An expert human player looks at a board position
and immediately sees what areas of play are most likely to be fruitful or
dangerous, whereas a computer has no innate sense of what is important and
must explore many more options. Deep Blue also had no sense of the history
of the game, and didn't know anything about its opponent. It played chess
yet didn't understand chess, in the
same way that a calculator performs arithmetic but doesn't understand
mathematics
.
2008-05-08 15:33:43 ChaosConst (为往圣继绝学)
有txt格式的也行啊，谁贴一个？2009-03-03 11:48:07 俯首拜阳明
发信人: cone (大象无形), 板面: AI标 题: [转载] On Intelligence [zz]
发信站: 飘渺水云间 (Wed Oct 29 18:51:34 2008), 转信
【 原文由 cone 发表于 Simulate 讨论区 】
Jeff Hawkins with Sandra Blakeslee
Contents
Prologue
1. Artificial Intelligence
2. Neural Networks
3. The Human Brain
4. Memory
5. A New Framework of Intelligence
6. How the Cortex Works
7. Consciousness and Creativity
8. The Future of Intelligence
Epilogue
Appendix: Testable Predictions
Bibliography
Acknowledgments
On
Intelligence
Prologue
This book and my life are animated by two passions.
For twenty-five years I have been passionate about mobile computing. In the
high-tech world of Silicon Valley, I am known for starting two companies
, Palm Computing and Handspring, and as the architect of many handheld computers
and cell phones such as the PalmPilot and the Treo.
But I have a second passion that predates my interest in computers— one
I view as more important. I am crazy about brains. I want to understand how
the brain works, not just from a philosophical perspective, not just in
a general way, but in a detailed nuts and bolts engineering way. My desire
is not only to understand what intelligence is and how the brain works,
but how to build machines that work the same way. I want to build truly
intelligent
machines.
The question of intelligence is the last great terrestrial frontier of science
. Most big scientific questions involve the very small, the very large, or
events that occurred billions of years ago. But everyone has a brain. You
are your brain. If you want to understand why you feel the way you do, how
you perceive the world, why you make mistakes, how you are able to be creative
, why music and art are inspiring, indeed what it is to be human, then you
need to understand the brain. In addition, a successful theory of intelligence
and brain function will have large societal benefits, and not just in helping
us cure brain-related diseases. We will be able to build genuinely intelligent
machines, although they won't be anything like the robots of popular fiction
and computer science fantasy. Rather, intelligent machines will arise from
a new set of principles about the nature of intelligence. As such, they
will help us accelerate our knowledge of the world, help us explore the universe
, and make the world safer. And along the way, a large industry will be created
.
Fortunately, we live at a time when the problem of understanding intelligence
can be solved. Our generation has access to a mountain of data about the
brain, collected over hundreds of years, and the rate at which we are gathering
more data is accelerating. The United States alone has thousands of
neuroscientists
. Yet we have no productive theories about what intelligence is or how the
brain works as a whole. Most neurobiologists don't think much about overall
theories of the brain because they're engrossed in doing experiments to
collect more data about the brain's many subsystems. And although legions
of computer programmers have tried to make computers intelligent, they have
failed. I believe they will continue to fail as long as they keep ignoring
the differences between computers and brains.
What then is intelligence such that brains have it but computers don't? Why
can a six-year-old hop gracefully from rock to rock in a streambed while
the most advanced robots of our time are lumbering zombies? Why are three
-year-olds already well on their way to mastering language while computers
can't, despite half a century of programmers' best efforts? Why can you
tell a cat from a dog in a fraction of a second while a supercomputer cannot
make the distinction at all? These are great mysteries waiting for an answer
. We have plenty of clues; what we need now are a few critical insights.
You may be wondering why a computer designer is writing a book about brains
. Or put another way, if I love brains why didn't I make a career in brain
science or in artificial intelligence? The answer is I tried to, several
times, but I refused to study the problem of intelligence as others have
before me. I believe the best way to solve this problem is to use the
detailed biology of the brain as a constraint and as a guide, yet think about
intelligence
as a computational problem— a position somewhere between biology and computer
science. Many biologists tend to reject or ignore the idea of thinking of
the brain in computational terms, and computer scientists often don't believe
they have anything to learn from biology.
Also, the world of science is less accepting of risk than the world of business
. In technology businesses, a person who pursues a new idea with a reasoned
approach can enhance his or her career regardless of whether the particular
idea turns out to be successful. Many successful entrepreneurs achieved
success only after earlier failures. But in academia, a couple of years spent
pursuing a new idea that does not work out can permanently ruin a young
career. So I pursued the two passions in my life simultaneously, believing
that success in industry would help me achieve success in understanding
the brain. I needed the financial resources to pursue the science I wanted
, and I needed to learn how to
affect change in the world, how to sell new ideas, all of which I hoped to
get from working in Silicon Valley.
In August 2002 I started a research center, the Redwood Neuroscience Institute
(RNI), dedicated to brain theory. There are many neuroscience centers in
the world, but no others are dedicated to finding an overall theoretical
understanding of the neocortex— the part of the human brain responsible
for intelligence. That is all we study at RNI. In many ways, RNI is like
a start-up company. We are pursuing a dream that some people think is
unattainable
, but we are lucky to have a great group of people, and our efforts are starting
to bear fruit.
* * *
The agenda for this book is ambitious. It describes a comprehensive theory
of how the brain works. It describes what intelligence is and how your brain
creates it. The theory I present is not a completely new one. Many of the
individual ideas you are about to read have existed in some form or another
before, but not together in a coherent fashion. This should be expected.
It is said that "new ideas" are often old ideas repackaged and reinterpreted
. That certainly applies to the theory proposed here, but packaging and
interpretation
can make a world of difference, the difference between a mass of details
and a satisfying theory. I hope it strikes you the way it does many people
. A typical reaction I hear is, "It makes sense. I wouldn't have thought
of intelligence this way, but now that you describe it to me I can see how
it all fits together." With this knowledge most people start to see themselves
a little differently. You start to observe your own behavior saying, "I
understand what just happened in my head." Hopefully when you have finished
this book, you will have new insight into why you think what you think and
why you behave the way you behave. I also hope that some readers will be
inspired to focus their careers on building intelligent machines based on
the principles outlined in these pages.
I often refer to this theory and my approach to studying intelligence as
"real intelligence" to distinguish it from "artificial intelligence." AI
scientists tried to program computers to act like humans without first answering
what intelligence is and what it means to understand. They left out the
most important part of building intelligent machines, the intelligence!
"Real intelligence" makes the point that before we attempt to build intelligent
machines, we have to first understand how the brain thinks, and there is
nothing artificial about that. Only then can we ask how we can build
intelligent
machines. The book starts with some background on why previous attempts
at understanding intelligence and building intelligent machines have failed
. I then introduce and develop the core idea of the theory, what I call the
memory-prediction framework. In chapter 6 I detail how the physical brain
implements the memory-prediction model— in other words, how the brain actually
works. I then discuss social and other implications of the theory, which
for many readers might be the
most thought-provoking section. The book ends with a discussion of intelligent
machines— how we can build them and what the future will be like. I hope
you find it fascinating. Here are some of the questions we will cover along
the way: Can computers be intelligent?
For decades, scientists in the field of artificial intelligence have claimed
that computers will be intelligent when they are powerful enough. I don'
t think so, and I will explain why. Brains and computers do fundamentally
different things.
Weren't neural networks supposed to lead to intelligent machines?
Of course the brain is made from a network of neurons, but without first
understanding what the brain does, simple neural networks will be no more
successful at creating intelligent machines than computer programs have
been.
Why has it been so hard to figure out how the brain works?
Most scientists say that because the brain is so complicated, it will take
a very long time for us to understand it. I disagree. Complexity is a symptom
of confusion, not a cause. Instead, I argue we have a few intuitive but
incorrect assumptions that mislead us. The biggest mistake is the belief
that intelligence is defined by intelligent behavior.
What is intelligence if it isn't defined by behavior?
The brain uses vast amounts of memory to create a model of the world. Everything
you know and have learned is stored in this model. The brain uses this memory
-based model to make continuous predictions of future events. It is the ability
to make predictions about the future that is the crux of intelligence. I
will describe the brain's predictive ability in depth; it is the core idea
in the book.
How does the brain work?
The seat of intelligence is the neocortex. Even though it has a great number
of abilities and powerful flexibility, the neocortex is surprisingly regular
in its structural details. The different parts of the neocortex, whether
they are responsible for vision, hearing, touch, or language, all work on
the same principles. The key to understanding the neocortex is understanding
these common principles and, in particular, its hierarchical structure.
We will examine the neocortex in sufficient detail to show how its structure
captures the structure of the world. This discussion will be the most technical
part of the book, but interested nonscientist readers should be able to
understand it.
What are the implications of this theory?
This theory of the brain can help explain many things, such as how we are
creative, why we feel conscious, why we exhibit prejudice, how we learn,
and why "old dogs" have trouble learning "new tricks." I will discuss a
number of these topics. Overall, this theory gives us insight into who we
are and why we do what we do.
Can we build intelligent machines and what will they do?
Yes. We can and we will. Over the next few decades, I see the capabilities
of such machines evolving rapidly and in interesting directions. Some people
fear that intelligent machines could be dangerous to humanity, but I argue
strongly against this idea. We are not going to be overrun by robots. It
will be far easier to build machines that outstrip our abilities in high
-level thought such as physics and mathematics than to build anything like
the walking, talking robots we see in popular fiction. I will explore the
incredible directions in which this technology is likely to go.
My goal is to explain this new theory of intelligence and how the brain works
in a way that anybody will be able to understand. A good theory should be
easy to comprehend, not obscured in jargon or convoluted argument. I'll
start with a basic framework and then add details as we go. Some will be
reasoning just on logical grounds; some will involve particular aspects of
brain circuitry. Some of the details of what I propose are certain to be
wrong, which is always the case in any area of science. A fully mature theory
will take years to develop, but that doesn't diminish the power of the core
idea.
* * *
When I first became interested in brains many years ago, I went to my local
library to look for a good book that would explain how brains worked. As
a teenager I had become accustomed to being able to find well-written books
that explained almost any topic of interest. There were books on relativity
theory, black holes, magic, and mathematics—whatever I was fascinated with
at the moment. Yet my search for a satisfying brain book turned up empty
. I came to realize that no one had any idea how the brain actually worked
. There weren't even any bad or unproven theories; there simply were none
. This was unusual. For example, at that time no one knew how the dinosaurs
had died, but there were plenty of theories, all of which you could read
about. There was nothing like this for brains. At first I had trouble
believing
it. It bothered me that we didn't know how this critical organ worked. While
studying what we did know about brains, I came to believe that there must
be a straightforward explanation. The brain wasn't magic, and it didn't
seem to me that the answers would even be that complex. The mathematician
Paul Erdos believed that the simplest mathematical proofs already exist
in some ethereal book and a mathematician's job was to find them, to "read
the book." In the same way, I felt that the explanation of intelligence
was "out there." I could taste it. I wanted to read the book.
For the past twenty-five years, I have had a vision of that small,
straightforward
book on the brain. It was like a carrot keeping me motivated during those
years. This vision has shaped the book you are holding in your hands right
now. I have never liked complexity, in either science or technology. You
can see that reflected in the products I have designed, which are often
noted for their ease of use. The most powerful things are simple. Thus this
book proposes a simple and straightforward theory of intelligence. I hope
you enjoy it.
1
Artificial Intelligence
When I graduated from Cornell in June 1979 with a degree in electrical
engineering
, I didn't have any major plans for my life. I started work as an engineer
at the new Intel campus in Portland, Oregon. The microcomputer industry
was just starting, and Intel was at the heart of it. My job was to analyze
and fix problems found by other engineers working in the field with our
main product, single board computers. (Putting an entire computer on a single
circuit board had only recently been made possible by Intel's invention
of the microprocessor.) I published a newsletter, got to do some traveling
, and had a chance to meet customers. I was young and having a good time,
although I missed my college
sweetheart who had taken a job in Cincinnati.
A few months later, I encountered something that was to change my life's
direction. That something was the newly published September issue of Scientific
American, which was dedicated entirely to the brain. It rekindled my childhood
interest in brains. It was fascinating. From it I learned about the
organization
, development, and chemistry of the brain, neural mechanisms of vision, movement
, and other specializations, and the biological basis for disorders of the
mind. It was one of the best Scientific American issues of all time. Several
neuroscientists I've spoken to have told me it played a significant role
in their career choice, just as it did for me.
The final article, "Thinking About the Brain," was written by Francis Crick
, the codiscoverer of the structure of DNA who had by then turned his talents
to studying the brain. Crick argued that in spite of a steady accumulation
of detailed knowledge about the brain, how the brain worked was still a
profound mystery. Scientists usually don't write about what they don't know
, but Crick didn't care. He was like the boy pointing to the emperor with
no clothes. According to Crick, neuroscience was a lot of data without a
theory. His exact words were, "what is conspicuously lacking is a broad
framework of ideas." To me this was the British gentleman's way of saying
, "We don't have a clue how this thing works."
It was true then, and it's still true today.
Crick's words were to me a rallying call. My lifelong desire to understand
brains and build intelligent machines was brought to life. Although I was
barely out of college, I decided to change careers. I was going to study
brains, not only to understand how they worked, but to use that knowledge
as a foundation for new technologies, to build intelligent machines. It
would take some time to put this plan into action.
In the spring of 1980 I transferred to Intel's Boston office to be reunited
with my future wife, who was starting graduate school. I took a position
teaching customers and employees how to design microprocessor-based systems
. But I had my sights on a different goal: I was trying to figure out how
to work on brain theory. The engineer in me realized that once we understood
how brains worked, we could build them, and the natural way to build artificial
brains was in silicon. I worked for the company that invented the silicon
memory chip and the microprocessor; therefore, perhaps I could interest
Intel in letting me spend part of my time thinking about intelligence and
how we could design brainlike memory chips. I wrote a letter to Intel's
chairman, Gordon Moore. The letter can be distilled to the following:
Dear Dr. Moore,
I propose that we start a research group devoted to understanding how the
brain works. It can start with one person— me— and go from there. I am
confident we can figure this out. It will be a big business one day.
— Jeff Hawkins
Moore put me in touch with Intel's chief scientist, Ted Hoff. I flew to
California
to meet him and lay out my proposal for studying the brain. Hoff was famous
for two things. The first, which I was aware of, was for his work in designing
the first microprocessor. The second, which I was not aware of at the time
, was for his work in early neural network theory.
Hoff had experience with artificial neurons and some of the things you could
do with them. I wasn't prepared for this.
After listening to my proposal, he said he didn't believe it would be possible
to figure out how the brain works in the foreseeable future, and so it didn
't make sense for Intel to support me. Hoff was correct, because it is now
twenty-five years later and we are just starting to make significant progress
in understanding brains. Timing is everything in business.
Still, at the time I was pretty disappointed.
I tend to seek the path of least friction to achieve my goals. Working on
brains at Intel would have been the simplest transition. With that option
eliminated I looked for the next best thing. I decided to apply to graduate
school at the Massachusetts Institute of Technology, which was famous for
its research on artificial intelligence and was conveniently located down
the road. It seemed a great match. I had extensive training in computer
science— "check." I had a desire to build intelligent machines, "check."
I wanted to first study brains to see how they worked…"uh, that's a problem
." This last goal, wanting to understand how brains worked, was a nonstarter
in the eyes of the scientists at the MIT artificial intelligence lab.
It was like running into a brick wall. MIT was the mother-ship of artificial
intelligence. At the time I applied to MIT, it was home to dozens of bright
people who were enthralled with the idea of programming computers to produce
intelligent behavior. To these scientists, vision, language, robotics, and
mathematics were just programming problems. Computers could do anything
a brain could do, and more, so why constrain your thinking by the biological
messiness of nature's computer? Studying brains would limit your thinking
. They believed it was better to study the ultimate limits of computation
as best expressed in digital computers. Their holy grail was to write computer
programs that would first match and then surpass human abilities. They took
an ends-justify-the-means approach; they were not interested in how real
brains worked. Some took pride in ignoring neurobiology.
This struck me as precisely the wrong way to tackle the problem. Intuitively
I felt that the artificial intelligence approach would not only fail to
create programs that do what humans can do, it would not teach us what
intelligence
is. Computers and brains are built on completely different principles. One
is programmed, one is self-learning. One has to be perfect to work at all
, one is naturally flexible and tolerant of failures. One has a central
processor
, one has no centralized control.
The list of differences goes on and on. The biggest reason I thought computers
would not be intelligent is that I understood how computers worked, down
to the level of the transistor physics, and this knowledge gave me a strong
intuitive sense that brains and computers were fundamentally different.
I couldn't prove it, but I knew it as much as one can intuitively know anything
. Ultimately, I reasoned, AI might lead to useful products, but it wasn't
going to build truly intelligent machines.
In contrast, I wanted to understand real intelligence and perception, to
study brain physiology and anatomy, to meet Francis Crick's challenge and
come up with a broad framework for how the brain worked. I set my sights
in particular on the neocortex— the most recently developed part of the
mammalian brain and the seat of intelligence. After understanding how the
neocortex worked, then we could go about building intelligent machines,
but not before.
Unfortunately, the professors and students I met at MIT did not share my
interests. They didn't believe that you needed to study real brains to
understand
intelligence and build intelligent machines. They told me so. In 1981 the
university rejected my application.
* * *
Many people today believe that AI is alive and well and just waiting for
enough computing power to deliver on its many promises. When computers have
sufficient memory and processing power, the thinking goes, AI programmers
will be able to make intelligent machines. I disagree. AI suffers from a
fundamental flaw in that it fails to adequately address what intelligence
is or what it means to understand something. A brief look at the history
of AI and the tenets on which it was built will explain how the field has
gone off course.
The AI approach was born with the digital computer. A key figure in the early
AI movement was the English mathematician Alan Turing, who was one of the
inventors of the idea of the general-purpose computer. His masterstroke
was to formally demonstrate the concept of universal computation: that is
, all computers are fundamentally equivalent regardless of the details of
how they are built. As part of his proof, he conceived an imaginary machine
with three essential parts: a processing box, a paper tape, and a device
that reads and writes marks on the tape as it moves back and forth. The
tape was for storing information— like the famous 1's and 0's of computer
code (this was before the invention
of memory chips or the disk drive, so Turing imagined paper tape for storage
). The box, which today we call a central processing unit (CPU), follows
a set of fixed rules for reading and editing the information on the tape.
Turing proved, mathematically, that if you choose the right set of rules
for the CPU and give it an indefinitely long tape to work with, it can perform
any definable set of operations in the universe. It would be one of many
equivalent machines now called Universal Turing Machines. Whether the problem
is to compute square roots, calculate ballistic trajectories, play games
, edit pictures, or reconcile bank transactions, it is all 1's and 0's
underneath
, and any Turing Machine can be programmed to handle it. Information processing
is information processing is information processing. All digital computers
are logically equivalent.
Turing's conclusion was indisputably true and phenomenally fruitful. The
computer revolution and all its products are built on it. Then Turing turned
to the question of how to build an intelligent machine. He felt computers
could be intelligent, but he didn't want to get into arguments about whether
this was possible or not. Nor did he think he could define intelligence
formally, so he didn't even try. Instead, he proposed an existence proof
for intelligence, the famous Turing Test: if a computer can fool a human
interrogator into thinking that it too is a person, then by definition the
computer must be intelligent. And so, with the Turing Test as his measuring
stick and the Turing Machine as his medium, Turing helped launch the field
of AI. Its central dogma: the brain is just another kind of computer. It
doesn't matter how you design an artificially intelligent system, it just
has to produce humanlike behavior.
The AI proponents saw parallels between computation and thinking. They said
, "Look, the most impressive feats of human intelligence clearly involve
the manipulation of abstract symbols— and that's what computers do too.
What do we do when we speak or listen? We manipulate mental symbols called
words, using well-defined rules of grammar. What do we do when we play chess
? We use mental symbols that represent the properties and locations of the
various pieces.
What do we do when we see? We use mental symbols to represent objects, their
positions, their names, and other properties. Sure, people do all this with
brains and not with the kinds of computers we build, but Turing has shown
that it doesn't matter how you implement or manipulate the symbols. You
can do it with an assembly of cogs and gears, with a system of electronic
switches, or with the brain's network of neurons— whatever, as long as
your medium can realize the functional equivalent of a Universal Turing Machine
."
This assumption was bolstered by an influential scientific paper published
in 1943 by the neurophysiologist Warren McCulloch and the mathematician
Walter Pitts. They described how neurons could perform digital functions—
that is, how nerve cells could conceivably replicate the formal logic at
the heart of computers. The idea was that neurons could act as what engineers
call logic gates. Logic gates implement simple logical operations such as
AND, NOT, and OR.
Computer chips are composed of millions of logic gates all wired together
into precise, complicated circuits. A CPU is just a collection of logic
gates.
McCulloch and Pitts pointed out that neurons could also be connected together
in precise ways to perform logic functions. Since neurons gather input from
each other and process those inputs to decide whether to fire off an output
, it was conceivable that neurons might be living logic gates. Thus, they
inferred, the brain could conceivably be built out of AND-gates, OR-gates
, and other logic elements all built with neurons, in direct analogy with
the wiring of digital electronic circuits. It isn't clear whether McCulloch
and Pitts actually believed the brain worked this way; they only said it
was possible. And, logically speaking, this view of neurons is possible.
Neurons can, in theory, implement digital
functions. However, no one bothered to ask if that was how neurons actually
were wired in the brain. They took it as proof, irrespective of the lack
of biological evidence, that brains were just another kind of computer.
It's also worth noting that AI philosophy was buttressed by the dominant
trend in psychology during the first half of the twentieth century, called
behaviorism. The behaviorists believed that it was not possible to know
what goes on inside the brain, which they called an impenetrable black box
. But one could observe and measure an animal's environment and its behaviors
— what it senses and what it does, its inputs and its outputs. They conceded
that the brain contained reflex mechanisms that could be used to condition
an animal into adopting new behaviors through reward and punishments. But
other than this, one did not need to study the brain, especially messy
subjective
feelings such as hunger, fear, or what it means to understand something.
Needless to say, this research philosophy eventually withered away throughout
the second half of the twentieth century, but AI would stick around a lot
longer.
As World War II ended and electronic digital computers became available for
broader applications, the pioneers of AI rolled up their sleeves and began
programming. Language translation? Easy! It's a kind of code breaking. We
just need to map each symbol in System A onto its counterpart in System
B. Vision? That looks easy too. We already know geometric theorems that deal
with rotation, scale, and displacement, and we can easily encode them as
computer algorithms— so we're halfway there. AI pundits made grand claims
about how quickly computer intelligence would first match and then surpass
human intelligence.
Ironically, the computer program that came closest to passing the Turing
Test, a program called Eliza, mimicked a psychoanalyst, rephrasing your
questions
back at you. For example, if a person typed in, "My boyfriend and I don'
t talk anymore," Eliza might say, "Tell me more about your boyfriend" or
"Why do you think your boyfriend and you don't talk anymore?" Designed as
a joke, the program actually fooled some people, even though it was dumb
and trivial. More serious efforts included programs such as Blocks World
, a simulated room containing blocks of different colors and shapes. You
could pose questions to Blocks World such as "Is there a green pyramid on
top of the big red cube?" or "Move the blue cube on top of the little red
cube." The program would answer your question or try to do what you asked
.
It was all simulated— and it worked. But it was limited to its own highly
artificial world of blocks. Programmers couldn't generalize it to do anything
useful.
The public, meanwhile, was impressed by a continuous stream of seeming successes
and news stories about AI technology. One program that generated initial
excitement was able to solve mathematical theorems. Ever since Plato, multistep
deductive inference has been seen as the pinnacle of human intelligence,
so at first it seemed that AI had hit the jackpot. But, like Blocks World
, it turned out the program was limited. It could only find very simple theorems
, which were already known. Then there was a large stir about "expert systems
," databases of facts that could answer questions posed by human users. For
example, a medical expert system might be able to diagnose a patient's disease
if given a list of symptoms. But again, they turned out to be of limited
use and didn't exhibit anything close to generalized intelligence.
Computers could play checkers at expert skill levels and eventually IBM's
Deep Blue famously beat Gary Kasparov, the world chess champion, at his
own game. But these successes were hollow. Deep Blue didn't win by being
smarter than a human; it won by being millions of times faster than a human
. Deep Blue had no intuition. An expert human player looks at a board position
and immediately sees what areas of play are most likely to be fruitful or
dangerous, whereas a computer has no innate sense of what is important and
must explore many more options. Deep Blue also had no sense of the history
of the game, and didn't know anything about its opponent. It played chess
yet didn't understand chess, in the
same way that a calculator performs arithmetic but doesn't understand
mathematics
.
In all c
2009-03-03 11:48:51 俯首拜阳明
要完整版的联系: herberthuige@gmail.com2009-06-13 16:30:24 学峰刘 (明天还有明天的饭)
可以下到pdf和有声版. 我就是混着读完/听完的.2009-10-15 00:32:40 Sugarcube (Actually, I'm in Eindhoven)
http://dl.torrentrea这有
2010-02-04 11:31:51 piao
新浪共享可以下载http://ishare.iask.s
> 我来回应