Intelligence as an Emergent Behavior or, The Songs of Eden
Could we build a thinking machine by simply hooking together a large network of artificial neurons and waiting for intelligence to spontaneously emerge? Not likely, but by studying the properties of biological and emergent systems, a carefully constructed network of artificial neurons could be inoculated with thought, similar to yeast's role in making beer. The clue may be in the "songs" of apes.
Originally published
Winter 1988 in Daedalus,
Journal of the American Academy of Arts and Sciences. Published
on KurzweilAI.net on May 2, 2002.
Sometimes a system with many simple components will exhibit a behavior
of the whole that seems more organized than the behavior of the
individual parts. Consider the intricate structure of a snowflake.
Symmetric shapes within the crystals of ice repeat in threes and
sixes, with patterns recurring from place to place and within themselves
at different scales. The shapes formed by the ice are consequences
of the local rules of interaction that govern the molecules of water,
although the connection between the shapes and the rules is far
from obvious. After all, these are the same rules of interaction
that cause water to suddenly turn to steam at its boiling point
and cause whirlpools to form in a stream. The rules that govern
the forces between water molecules seem much simpler than crystals
or whirlpools or boiling points, yet all of these complex phenomena
are called emergent behaviors of the system.
It would be very convenient if intelligence were an emergent behavior
of randomly connected neurons in the same sense that snowflakes
and whirlpools are the emergent behaviors of water molecules. It
might then be possible to build a thinking machine by simply hooking
together a sufficiently large network of artificial neurons. The
notion of emergence would suggest that such a network, once it reached
some critical mass, would spontaneously begin to think.
This is a seductive idea, since it allows for the possibility of
constructing intelligence without first understanding it. Understanding
intelligence is difficult and probably a long way off. The possibility
that it might spontaneously emerge from the interactions of a large
collection of simple parts has considerable appeal to a would-be
builder of thinking machines. Unfortunately, as a practical approach
to construction, the idea tends to be unproductive. The concept
of emergence, in itself, offers neither guidance on how to construct
such a system nor insight into why it would work.
Ironically, this apparent inscrutability accounts for much of the
idea's continuing popularity, since it offers a way to believe in
physical causality while simultaneously maintaining the impossibility
of a reductionist explanation of thought. For some, our ignorance
of how local interactions produce emergent behavior offers a reassuring
fog in which to hide free will.
There has been a renewal of interest in emergent behavior in the
form of neural networks and connectionist models, spin glasses and
cellular automata, and evolutionary models. The reasons for this
interest have little to do with philosophy in one way or the other,
but rather are a combination of new insights and new tools. The
insights come primarily from a branch of physics called "dynamical
systems theory." The tools come from the development of new
types of computing devices. Just as in the 1950's we thought of
intelligence in terms of servomechanism, and in the 60's and 70's
in terms of sequential computers, we are now beginning to think
in terms of parallel machines. This is not a deep philosophical
shift, but it is of great practical importance, since it is now
possible to study large emergent systems experimentally.
Inevitably, anti-reductionists interpret such progress as a schism
within the field between symbolic rationalists who oppose them and
gestaltists who support them. I have often been asked which "side"
I am on. Not being a philosopher, my inclination is to focus on
the practical aspects of this question: How would we go about constructing
an emergent intelligence? What information would we need to know
in order to succeed? How can this information be determined by experiment?
The emergent system that I can most easily imagine would be an
implementation of symbolic thought, rather than a refutation of
it. Symbolic thought would be an emergent property of the system.
The point of view is best explained by the following parable about
the origin of human intelligence. As far as I know, this parable
of human evolution is consistent with the available evidence (as
are many others), but since it is chosen to illustrate a point it
should be read as a story rather than as a theory. It is reversed
from most accepted theories of human development in that it presents
features that are measurable in the archeological records such as
increased brain size, food sharing, and neoteny, as consequences
rather than as causes of intelligence.
Once upon a time, about two and a half million years ago, there
lived a race of apes that walked upright. In terms of intellect
and habit they were similar to modern chimpanzees. The young apes,
like many young apes today, had a tendency to mimic the actions
of others. In particular, they had a tendency to imitate sounds.
If one ape went "ooh, eeh, eeh," it would be likely that
the other one would repeat, "ooh, eeh, eeh." (I do not
know why apes do this, but they do. As do many species of birds.)
Some sequences of sounds were more likely to be repeated than others.
I will call these "songs."
For the moment let us ignore the evolution of the apes and consider
the evolution of the songs. Since the songs were replicated by the
apes, and since they sometimes died away and were occasionally combined
with others, we may consider them, very loosely, a form of life.
They survived, bred, competed with one another, and evolved according
to their own criterion of fitness. If a song contained a particularly
catchy phrase that caused it to be repeated often, then that phrase
was likely to be repeated and incorporated into other songs. Only
songs that had a strong tendency to be repeated survived.
The survival of the song was only indirectly related to the survival
of the apes. It was more directly affected by the survival of other
songs. Since the apes were a limited resource, the songs had to
compete with one another for a chance to be sung. One successful
strategy for competition was for a song to specialize; that is,
for it to find a particular niche where it would be likely to be
repeated. Songs that fit particularly well with a specific mood
or activity of an ape had a special survival value for this reason.
(I do not know why some songs fit well with particular moods, but
since it is true for me I do not find it hard to believe for my
ancestors.)
Up to this point the songs were not of any particular value to
the apes. In a biological sense they were parasites, taking advantage
of the apes' tendency to imitate. Once the songs began to specialize,
however, it became advantageous for an ape to pay attention to the
songs of others and to differentiate between them. By listening
to songs, a clever ape could gain useful information. For example,
an ape could infer that another ape had found food, or that it was
likely to attack. Once the apes began to take advantage of the songs,
a mutually beneficial symbiosis developed. Songs enhanced their
survival by conveying useful information. Apes enhanced their survival
by improving their capacity to remember, replicate, and understand
songs. The blind forces of evolution created a partnership between
the songs and the apes that thrived on the basis of mutual self-interest.
Eventually this partnership evolved into one of the world's most
successful symbionts: us.
Unfortunately, songs do not leave fossils, so unless some natural
process has left a phonographic trace, we may never know if this
is what really happened. But if the story is true, the apes and
the songs became the two components of human intelligence. The songs
evolved into the knowledge, mores, and mechanism of thought that
together are the symbolic portion of human intelligence. The apes
became apes with bigger brains, perhaps optimized for late maturity
so that they could learn more songs. "Homo Sapiens" is
a cooperative combination of the two.
It is not unusual in nature for two species to live together so
interdependently that they appear to be a single organism. Lichens
are symbionts of a fungus and an alga living so closely intertwined
that they can only be separated under a microscope. Bean plants
need living bacteria in their roots to fix the nitrogen from the
soil, and in return the bacteria need nutrients from the bean plants.
Even the single celled "Paramecium Bursarra" uses green
algae living inside itself to synthesize food.
There may be an example even closer to the songs and the apes,
where two entirely different forms of "life" form a symbiosis.
In "The Origins of Life," Freeman Dyson suggests that
biological life is a symbiotic combination of two different self-reproducing
entities with very different forms of replication. Dyson suggests
that life originated in two stages. While most theories of the origin
of life start with nucleotides replicating in some "primeval
soup," Dyson's theory starts with metabolizing drops of oil.
In the beginning, these hypothetical replicating oil drops had
no genetic material, but were self-perpetuating chemical systems
that absorbed raw materials from their surroundings. When a drop
reached a certain size it would split, with about half of the constituents
going to each part. Such drops evolved efficient metabolic systems
even though their rules of replication were very different from
the Mendelian rules of modern life. Once the oil drops became good
at metabolizing, they were infected by another form of replicators,
which, like the songs, have no metabolism of their own. These were
parasitic molecules of DNA which, like modern viruses, took advantage
of the existing machinery of the cells to reproduce. The metabolizers
and the DNA eventually co-evolved into a mutually beneficial symbiosis
that we know today as life.
This two-part theory of life is not conceptually far from the two-part
story of intelligence. Both suggest that a pre-existing homeostatic
mechanism was infected by an opportunistic parasite. The two parts
reproduced according to different set of rules, but were able to
co-evolve so successfully that the resulting symbiont appears to
be a single entity.
Viewed in this light, choosing between emergence and symbolic computation
in the study of intelligence would be like choosing between metabolism
and genetic replication in the study of life. Just as the metabolic
system provides a substrate in which the genetic system can work,
so an emergent system may provide a substrate in which the symbolic
system can operate.
Currently, the metabolic system of life is far too complex for
us to fully understand or reproduce. By comparison, the Mendelian
rules of genetic replication are almost trivial, and it is possible
to study them as a system unto themselves without worrying about
the details of metabolism which supports them. In the same sense,
it seems likely that symbolic thought can be fruitfully studied
and perhaps even recreated without worrying about the details of
the emergent system that supports it. So far this has been the dominant
approach in artificial intelligence and the approach that has yielded
the most progress.
The other approach is to build a model of the emergent substrate
of intelligence. This artificial substrate for thought would not
need to mimic in detail the mechanisms of the biological system,
but it would need to exhibit those emergent properties that are
necessary to support the operations of thought.
What is the minimum that we would need to understand in order to
construct such a system? For one thing, we would need to know how
big a system to build. How many bits are required to store the acquired
portion of human knowledge of a typical human? We need to know an
approximate answer in order to construct an emergent intelligence
with human-like performance. Currently the amount of information
stored by a human is not known to within even two orders of magnitude,
but it can in principle be determined by experiment. There are at
least three ways the question might be answered.
One way to estimate the storage requirements for emergent intelligence
would be from an understanding of the physical mechanisms of memory
in the human brain. If that information is stored primarily by modifications
of synapses, then it would be possible to measure the information
storage capacity of the brain by counting the number of synapses.
Elsewhere in this issue, Schwartz shows that this method leads to
an upper bound on the storage capacity of the brain of 10 to the
15th bits. Even knowing the exact amount of physical storage in
the brain would not completely answer the question of storage requirement,
since much of the potential storage might be unused or used inefficiently.
But at least this method can help establish an upper bound on the
requirements.
A second method for estimating the information in symbolic knowledge
would be to measure it by some form of statistical sampling. For
instance, it is possible to estimate the size of an individual's
vocabulary by testing specific words randomly sampled from a dictionary.
The fraction of words known by the individual is a good estimate
of the fraction of words known in the complete dictionary. The estimated
vocabulary size is this fraction times the number of words in the
dictionary. The experiment depends on having a predetermined body
of knowledge against which to measure. For example, it would be
possible to estimate how many facts in the "Encyclopedia Britannica"
were known by a given individual, but this would give no measure
of facts not contained within the encyclopedia. The method is useful
only in establishing a lower bound.
A related experiment is the game of "20 questions" in
which one player identifies an object chosen by the other by asking
a series of 20 yes-or-no questions. Since each answer provides no
more than a single bit of information, and since skillful players
generally require almost all of the 20 questions to choose correctly,
we can estimate that the number of allowable choices is on the order
of 2 to the 20th, or about one million. This gives an estimated
number of allowable objects known in common by the two players.
Of course, the measure is inaccurate since the questions are not
perfect and the choices of objects are not random. It is possible
that a refined version of the game could be developed and used to
provide another lower bound.
A third approach to measuring the amount of information of the
symbolic portion of human knowledge is to estimate the rate of acquisition
and to integrate over time. For example, experiments on memorizing
random sequences of syllables indicate that the maximum memorization
rate of this type of knowledge is about one "chunk" per
second. A "chunk" in this context can be safely assumed
to contain less than 100 bits of information, so the results suggest
that the maximum rate that a human is able to commit information
to long-term memory is significantly less than 100 bits per second.
If this is true, a 20-year-old human learning at the maximum rate
for 16 hours a day would know less than 50 gigabits of information.
I find this number surprisingly small.
A difficulty with this estimate of the rate of acquisition is that
the experiment measures only information coming through one sensory
channel under one particular set of circumstances. The visual system
sends more than a million times this rate of information to the
optic nerve, and it is conceivable that all of this information
is committed to memory. If it turns out that images are stored directly,
it will be necessary to significantly increase the 100 bit per second
limit, but there is no current evidence that this is the case. In
experiments measuring the ability of exceptional individuals to
store "eidetic" images of random dot stereograms, the
subjects are given about 5 minutes to "memorize" a 128x128
image. Memorizing only a few hundred of these bits is probably sufficient
to pass the test.
I am aware of no evidence that suggests more than a few bits per
second of any type of information can be committed to long-term
memory. Even if we accept at face value reports of extraordinary
feats of memory, such as those of Luria's showman in "Mind
of the Mnemonist", the average rate of commitment to memory
never seems to exceed a few bits per second. Experiments should
be able to refine this estimate, but even if we knew the maximum
rate exactly, the rate averaged over a lifetime would probably be
very much less. Knowing the maximum rate would establish an upper
bound on the requirements of storage.
The sketchy data cited above would suggest that an intelligent
machine would require 10 to the 9th bits of storage, plus or minus
two orders of magnitude. This assumes that the information is encoded
in such a way that it requires a minimum amount of storage, which
for the purpose of processing information would probably not be
the most practical representation. As a would-be builder of thinking
machines, I find this number encouragingly small, since it is well
within the range of current electronic computers. As a human with
an ego, I find it distressing. I do not like to think that my entire
lifetime of memories could be placed on a reel of magnetic tape.
Hopefully experimental evidence will clear this up one way or the
other.
There are a few subtleties in the question of storage requirements,
in defining the quantity of information in a way that is independent
of the representation. Defining the number of bits in the information-theoretical
sense requires a measure of the probabilities over the ensemble
of possible states. This means assigning an "a priori"
probability to each possible set of knowledge, which is the role
of inherited intelligence. Inherited intelligence provides a framework
in which the knowledge of acquired intelligence can be interpreted.
Inherited intelligence defines what is knowable; acquired intelligence
determines which of the knowable is known.
Another potential difficulty is how to count the storage of information
that can be deduced from other data. In the strict information-theoretical
sense, data that can be inferred from other data add no information
at all. An accurate measure would have to take into account the
possibility that knowledge is inconsistent, and that only limited
inferences are actually made. These are the kind of issues currently
being studied on the symbolic side of the field of artificial intelligence.
One issue that does not need to be resolved to measure storage
capacity is distributed versus localized representation. Knowing
what types of representation are used in what parts of the human
brain would be of considerable scientific interest, but it does
not have a profound impact on the amount of storage in the system,
or on our ability to measure it. Non-technical commentators have
a tendency to attribute almost mystical qualities to distributed
storage mechanisms such as those in holograms and neural networks,
but the limitations on their storage capacities are well understood.
Distributed representations with similar properties are often used
within conventional digital computers, and they are invisible to
most users except in the system's capacity to tolerate errors. The
error correcting memory used in most computers is a good example.
The system is composed of many physically separate memory chips,
but any single chip can be removed without loosing any data. This
is because the data is not stored in any one place, but in a distributed
non-local representation across all of the units. In spite of the
"holographic" representation, the information storage
capacity of the system is no greater than it would be with a conventional
representation. In fact, it is slightly less. This is typical of
distributed representations.
Storage capacity offers one measure of the requirements of a human-like
emergent intelligence. Another measure is the required rate of computation.
Here there is no agreed upon metric, and it is particularly difficult
to define a unit of measure that is completely independent of representation.
The measure suggested below is simple and the answer is certainly
important, if not sufficient.
Given an efficiently stored representation of human knowledge,
what is the rate of access to that storage (in bits per second)
required to achieve human-like performance? Here "efficiently
stored representation" means any representation requiring only
a multiplicative constant of storage over the number of bits of
information. This restriction eliminates the formal possibility
of a representation storing a pre-computed answer to every question.
Allowing storage within a multiplicative constant of the optimum
does restrict the range of possible representations, but it allows
most representations that we would regard as reasonable. In particular,
it allows both distributed and local representations.
The question of the bandwidth required for human-like performance
is accessible by experiment, along similar approaches as those outlined
for the question of storage capacity. If the "cycle time"
of human memory is limited by the firing time of a neuron, then
the ratio of this answer to the total number of bits tells the fraction
of the memory that is accessed simultaneously. This gives an indication
of the parallel or serial nature of the computation. Informed opinions
differ greatly in this matter. The bulk of the quantitative evidence
favors the serial approach. Memory retrieval times for items in
lists, for example, depend on the position and the number of items
in the list. Except for sensory processing, most successful artificial
intelligence programs have been based on serial models of computation,
although this may be a distortion caused by the availability of
serial machines.
My own guess is that the reaction time experiments are misleading
and that human-level performance will require accessing of large
fractions of the knowledge several times per second. Given a representation
of acquired intelligence with a realistic representation efficiency
of 10%, the 10 to the 9th bits of memory mentioned above would require
a memory bandwidth about 10 to the 11th bits per second. This bandwidth
seems physiologically plausible since it corresponds to about a
bit per second per neuron in the cerebral cortex.
By way of comparison, the memory bandwidth of a conventional electronic
computer is in the range of 10 to the 6th to 10 to the 8th bits
per second. This is less than 0.1% of the imagined requirement.
For parallel computers the bandwidth is considerably higher. For
example, a 65,536 processor Connection Machine can access its memory
at approximately 10 to the 11th bits per second. It is not entirely
coincidence that this fits well with the estimate above.
Another important question is: What sensory-motor functions are
necessary to sustain symbolic intelligence? An ape is a complex
sensory-motor machine, and it is possible that much of this complexity
is necessary to sustain intelligence. Large portions of the brain
seem to be devoted to visual, auditory, and motor processing, and
it is unknown how much of this machinery is needed for thought.
A person who is blind and deaf or totally paralyzed can undoubtedly
be intelligent, but this does not prove that the portion of the
brain devoted to these functions is unnecessary for thought. It
may be, for example, that a blind person takes advantage of the
visual processing apparatus of the brain for spatial reasoning.
As we begin to understand more of the functional architecture of
the brain, it should be possible to identify certain functions as
being unnecessary for thought by studying patients whose cognitive
abilities are unaffected by locally confined damage to the brain.
For example, binocular stereo fusion is known to take place in a
specific area of the cortex near the back of the head. Patients
with damage to this area of the cortex have visual handicaps, but
show no obvious impairment in their ability to think. This suggests
that stereo fusion is not necessary for thought. This is a simple
example, and the conclusion is not surprising, but it should be
possible by such experiments to establish that many sensory-motor
functions are unnecessary. One can imagine, metaphorically, whittling
away at the brain until it is reduced to its essential core. Of
course it is not quite this simple. Accidental damage rarely incapacitates
completely and exclusively a single area of the brain. Also, it
may be difficult to eliminate one function at a time since one mental
capacity may compensate for the lack of another.
It may be more productive to assume that all sensory-motor apparatus
is unnecessary until proven useful for thought, but this is contrary
to the usual point of view. Our current understanding of the philogenic
development of the nervous system suggests a point of view in which
intelligence is an elaborate refinement of the connection between
input and output. This is reinforced by the experimental convenience
of studying simple nervous systems, or studying complicated nervous
systems by concentrating on those portions most directly related
to input and output. By necessity, most everything we know about
the function of the nervous system comes from experiments on those
portions that are closely related to sensory inputs or motor outputs.
It would not be surprising if we have overestimated the importance
of these functions to intelligent thought.
Sensory-motor functions are clearly important for the application
of intelligence and for its evolution, but these are separate issues
from the question above. Intelligence would not be of much use without
an elaborate system of sensory apparatus to measure the environment
and an elaborate system motor apparatus to change it, nor would
it have been likely to have evolved. But the apparatus necessary
to exercise and evolve intelligence is probably very much more than
the apparatus necessary to sustain it. One can believe in the necessity
of the opposable thumb for the development of intelligence, without
doubting a human capacity for thumbless thought. It is quite possible
that even the meager sensor-motor capabilities that we currently
know how to provide would be sufficient for the operation of emergent
intelligence.
These questions of capacity and scope are necessary in defining
the magnitude of the task of constructing an emergent intelligence,
but the key question is one of understanding. While it is possible
that we will be able to recreate the emergent substrate of intelligence
without fully understanding the details of how it works, it seems
likely that we would at least need to understand some of its principles.
There are at least three paths by which such understanding could
be achieved. One is to study the properties of specific emergent
systems, to build a theory of their capabilities and limitations.
This kind of experimental study is currently being conducted on
several classes of promising systems including neural networks,
spin glasses, cellular automata, classifier systems and adaptive
automata. Another possible path to understanding is the study of
biological systems, which are our only real examples of intelligence,
and our only example of an emergent system which has produced intelligence.
The disciplines that have provided the most useful information of
this type so far have been neurophysiology, cognitive psychology,
and evolutionary biology. A third path would be a theoretical understanding
of the requirements of intelligence, or of the phenomena of emergence.
Examples of relevant disciplines of theories of logic and computability,
linguistics, and dynamical systems theory. Anyone who looks to emergent
systems as a way of defending human thought from the scrutiny of
science is likely to be disappointed.
One cannot conclude, however, that a reductionist understanding
is necessary for the creation of intelligence. Even a little understanding
could go a long way toward the construction of an emergent system.
A good example of this is how cellular automata have been used to
simulate the emergent behavior of fluids.
The whirlpools that form as a fluid flows past a barrier are not
well understood analytically, yet they are of great practical importance
in the design of boats and airplanes. Equations that describe the
flow of a fluid have been known for almost a century, but except
for a few simple cases they cannot be solved. In practice the flow
is generally analyzed by simulation. The most common method of simulation
is the numerical solution of the continuous equations.
On a highly parallel computer it is possible to simulate fluids
with even less understanding of the system, by simulating billions
of colliding particles that reproduce the emergent phenomena such
as vortices. Calculating the detailed molecular interactions for
so many particles would be extremely difficult, but a few simple
aspects of the system such as conservations of energy and particle
number are sufficient to reproduce the large-scale behavior. A system
of simplified particles that obey these two laws, but are otherwise
unrealistic, can reproduce the same emergent phenomena as reality.
For example, it is possible to use particles of unit mass that move
only at unit speed along a hexagonal lattice, colliding according
to the rules of billiard balls. Experiments show that this model
produces laminar flow, vortex streams, and even turbulence that
is indistinguishable from the behavior of real fluids. Although
the detailed rules of interaction are very different than the interactions
of real molecules, the emergent phenomena are the same. The emergent
phenomena can be created without understanding the details of the
forces between the molecules or the equations that describe the
flow of the fluid.
The recreation of intricate patterns of ebbs and flows within a
fluid offers an example of how it is possible to produce a phenomenon
without fully understanding it. But the model was constructed by
physicists who knew a lot about fluids. That knowledge helped to
determine which features of the physical system were important to
implement, and which were not.
Physics is an unusually exact science. Perhaps a better example
of an emergent system which we can simulate with only a limited
understanding is evolutionary biology. We understand, in a weak
sense, how creatures with Mendelian patterns of inheritance, and
different propensities for survival can evolve toward better fitness
in their environments. In certain simple situations we can even
write down equations that describe how quickly this adaptation will
take place. But there are many gaps in our understanding of the
processes of evolution. We can explain in terms of natural selection
why flying animals have light bones, but we cannot explain why certain
animals have evolved flight and others have not. We have some qualitative
understanding of the forces that cause evolutionary change, but
except in the simplest cases, we cannot explain the rate or even
the direction of that change.
In spite of these limitations, our understanding is sufficient
to write programs of simulated evolution that show interesting emergent
behaviors. For example, I have recently been using an evolutionary
simulation to evolve programs to sort numbers. In this system, the
genetic material of each simulated individual is interpreted as
a program specifying a pattern of comparisons and exchanges. The
probability of an individual survival in the system is dependent
on the efficiency and accuracy of this program in sorting numbers.
Surviving individuals produce offspring by sexual combination of
their genetic material with occasional random mutation. After tens
of thousands of generations, a population of hundreds of thousands
of such individuals will evolve very efficient programs for sorting.
Although I wrote the simulation producing these sorting programs,
I do not understand in detail how they were produced or how they
work. If the simulation had not produced working programs, I would
have had very little idea about how to fix it.
The fluid flow and simulated evolution examples suggest that it
is possible to make a great deal of use of a small amount of understanding.
The emergent behaviors exhibited by these systems are a consequence
of the simple underlying rules, which are defined by the program.
Although the systems succeed in producing the desired results, their
detailed behaviors are beyond our ability to analyze and predict.
One can imagine if a similar process produced a system of emergent
intelligence, we would have a similar lack of understanding about
how it worked.
My own guess is that such an emergent system would not be an intelligent
system itself, but rather the metabolic substrate on which intelligence
might grow. In terms of the apes and the songs, the emergent portion
of the system would play the role of the ape, or at least that part
of the ape that hosts the songs. This artificial mind would need
to be inoculated with human knowledge. I imagine this process to
be not so different from teaching a child. This would be a tricky
and uncertain procedure since, like a child, this emergent mind
would presumably be susceptible to bad ideas as well as good. The
result would be not so much an artificial intelligence, but rather
a human intelligence sustained within an artificial mind.
Of course, I understand that this is just a dream. And I will admit
that I am more propelled by hope than by the probability of success.
But if, within this artificial mind, the seed of human knowledge
begins to sustain itself and grow of its own accord, then for the
first time human thought will live free of bones and flesh, giving
this child of mind an earthly immortality denied to us.
Attempts to create emergent intelligence, at least those that are
far enough in the past for us to judge, have been disappointing.
Many computational systems, such as homeostats, perceptrons, and
cellular automata exhibit clear examples of emergent behavior, but
that behavior falls far short of intelligence. A perceptron, for
example, is a collection of artificial neurons that can recognize
simple patterns. Considerable optimism was generated in the 1960's
when it was proved that anything a perceptron could recognize, it
could learn to recognize from examples. This was followed by considerable
disappointment when it was realized that the set of things that
could be recognized at all was very limited. What appeared to be
complicated behavior of the system turned out in the final analysis
to be surprisingly simple.
In spite of such disappointments, I believe that the notion of
emergence contains an element of truth, an element that can be isolated
and put to use.
A helpful analogy is the brewing of beer. The brewmaster creates
this product by making a soup of barley and hops, and infecting
it with yeast. Chemically speaking most of the real work is done
by the yeast, which converts the starch to alcohol. The brewmaster
is responsible for creating and maintaining the conditions under
which that conversion can take place. The brewmaster does not need
to understand exactly how the yeast does its work, but does need
to understand the properties of the environment in which the yeast
will thrive. By providing the right combination of ingredients at
the right temperature in the right container, the brewmaster is
able to create the necessary conditions for the production of beer.
Something analogous to this process may be possible in the creation
of an artificial intelligence. It is unlikely that intelligence
would spontaneously appear in a random network of neurons, just
as it is unlikely that life would spontaneously appear in barley
soup. But just as carefully mixed soup can be inoculated with yeast,
it may be that a carefully constructed network of artificial neurons
can be inoculated with thought.
The approach depends on the possibility of separating human intelligence
into two parts, corresponding to the soup and the yeast. Depending
on one's point of view, these two parts can be viewed as hardware
and software, intellect and knowledge, nature and nurture, or program
and data. Each point of view carries with it a particular set of
intuitions about the nature of the split and the relative complexity
of the parts.
One way that biologists determine if a living entity is a symbiont
is to see if the individual components can be kept alive separately.
For example, biologists have tried (unsuccessfully) to prove the
oil-drop theory by sustaining metabolizing oil drops in an artificial
nutrient broth. Such an experiment for human intelligence would
have two parts. One would be a test of the human ape's ability to
live without the ideas of human culture. This experiment is occasionally
conducted in an uncontrolled form when feral children are reared
by animals. The two-part theory would predict that such children,
before human contact, would not be significantly brighter than nonhuman
primates. The complementary experiment, sustaining human ideas and
culture in an artificial broth, is the one in which we are more
specifically interested. If this were successful we would have a
thinking machine, although perhaps it would not be accurate to call
it an artificial intelligence. It would be natural intelligence
sustained within an artificial mind.
To pursue the consequences of this point of view, we will assume
that human intelligence can be cleanly divided into two portions
that we will refer to as acquired and inherited intelligence. These
correspond to the songs and to the apes, respectively, or in the
fermentation metaphor, the yeast and the barley soup. We will consider
only those features of inherited intelligence that are necessary
to support acquired intelligence, only those features of acquired
intelligence that impose requirements on inherited intelligence.
We will study the interface between the two.
Even accepting this definition of the problem, it is not obvious
that the interface is easy to understand or recreate. This leads
to a specific question about the scope of the interface that can
presumably be answered by experiment.
The functional scope of the interface between acquired and inherited
intelligence is not the only property that can be investigated.
To build a home for an animal, the first thing we would need to
know is the animal's size. This is also one of the first things
we need to know in building an artificial home for acquired intelligence.
This leads to question number two:
The guesses to answers that I have given are imprecise, but the
questions are not. In principle they can be answered by experiment.
The final question I will pose is more problematic. What I would
like to ask is "What are the organizing principles of inherited
intelligence?" but this question is vague and it is not clear
what would be an acceptable answer. I shall substitute a more specific
question that hopefully captures the same intent:
"Question IV: What quantities remain constant during the computation
of intelligence; or, equivalently, what functions of state are minimized?"
This question assumes that inheritable intelligence is some form
of homeostatic process and asks what quantity is held static. It
is the most difficult of the four questions, but historically it
has been an important question to ask in areas when there was not
yet a science to guide progress.
The study of chemistry is one example. In chemical reactions between
substances it is obvious that a great number of things change and
not so obvious what stays the same. It turns out that if the experiment
is done carefully, the weight of the reactants will always equal
the weight of the product. The total weight remains the same. This
is an important organizing principle in chemistry and understanding
it was a stepping stone to the understanding of an even more important
principle: the conservation of the weights of the individual elements.
The technical difficulty of defining and creating a truly closed
experiment, in particular eliminating the inflow and outflow of
gases, explains why chemists did not fully appreciate these principles
until the middle of the 19th century.
Another very different example of a system that can be understood
in terms of what is held constant is the system of formal logic.
This is a set of rules under which sentences may be changed without
changing their truth. A similar example, which has also been important
to artificial intelligence, is the lambda calculus, which is the
basis of the language Lisp. This is a system of transforming expressions
in such a way that their "values" do not change, where
the values are those forms of the expression which are not changed
by the transformations. (This sounds circular because it is. A more
detailed explanation would show it to be more so.) These formal
systems are conceptually organized around that which is held constant.
In physics there are many examples of how conservations have been
used successfully to organize our conception of reality, but while
conservations of energy, momentum, mass, and charge are certainly
important, I do not wish to make too much of them in this context.
In this sense the principles of conservation will more likely resemble
those of biology than physics.
One of the most useful conservation principles in biology appears
in the notion of a gene. This is the unit of character determination
that is conserved during reproduction. In sexual reproduction this
can get complicated since an individual receives a set of genes
from each of two parents. A gene that affects a given trait may
not be expressed if it is masked by another, and there is not a
simple correspondence between genes and measurable traits. The notion
that atomic units of inheritance are always present, even when they
are not expressed, was hard to accept and it was not widely believed
almost a century after Mendel's initial experiments. In fact the
conservation is not perfect, but it is still one of the most important
organizing principles in the study of living organisms.
In biology, the rules of conservation are often expressed as minimum
principles. The two forms are equivalent. For instance, the minimum
principle corresponding to the physical conservation of momentum
is the principle of least action. A biological example is the principle
of optimal adaptation, which states that species will evolve toward
fitness to their environments. The distance to the ideal is minimized.
A conservation principle associated with this is the Fischer Theorem
of Natural Selection, which states that the rate of change in fitness
is equal to the genetic variance. In cases where this minimum principle
can be applied, it allows biologists to quantitatively predict the
values of various biological parameters.
For example, sickle-cell anemia is a congenital disease controlled
by a recessive gene. Individuals who inherit the gene from both
parents are likely to die without reproducing, but individuals who
inherit the gene from a single parent are resistant to malaria.
In certain regions of West Africa 40% of the population carries
the gene. From this fact and the principle of optimal fitness, it
is possible to predict that the survival advantage of resistance
to malaria is about 25% in these regions. This estimate fits well
with measured data. Similar methods have been used to estimate the
number of eggs laid by a bird, the shape of sponges, and the gait
of animals at different speeds. But these examples of applying a
minimum principle are not so crisp as those of physics. Why, for
example, do we not evolve a non-lethal gene that protects against
malaria? The answer is complicated, and the principle of fitness
offers no help. It is useful in aiding our understanding, but it
does not explain all. This is probably the kind of answer to Question
IV for which we will have to settle.
Even in physics, knowledge of the exact law does not really explain
all behaviors. The snowflakes and whirlpools of water are examples.
The forces that govern the interaction of water molecules are understood
in some detail, but there is no analytical understanding of the
connection between these forces and their emergent behaviors of
water.
On the other hand, our goal is not necessarily to understand, but
to recreate. In both of the examples mentioned, conservation principles
give us sufficient understanding to recreate the phenomena.
In order to achieve this kind of understanding for intelligence
it will be necessary to ask and answer the kinds of questions that
are mentioned above.
I do not know the answer to Question IV. It is possible that it
will be very complicated and the interface between acquired and
inherited intelligence will be difficult to reproduce. But it is
also possible that it will be simple. One can imagine this would
be the artificial substrate for thought.
Once this is achieved it will still remain to inoculate the artificial
mind with the seed of knowledge. I imagine this to be not so different
from the process of teaching a child. It will be a tricky and uncertain
process since, like a child, this mind will presumably be susceptible
to bad ideas as well as good. The first steps will be the most delicate.
If we have prepared well, it will reach a point where it can sustain
itself and grow of its own accord.
For the first time human thought will live free of bones and flesh,
giving this child of mind an earthly immortality denied to us.
References
Dyson, Freeman. "The Origins of Life", Cambridge University
Press, 1985.
Haldane, J. B. S. "The Causes of Evolution", Harper &
Brothers, 1932.
Hillis, W. Daniel. "The Connection Machine", The MIT
Press, 1985.
Luria, A. R. "Mind of the Mnemonist", Basic Books, 1968.
Newell, Allen. "Human Problem Solving", Prentice Hall,
1972.
Wolfram, Stephen. "Theory of Applications of Cellular Automata",
World Scientific, 1986.
"Intelligence as an Emergent Behavior or, The Songs of
Eden" reprinted by permission of Daedalus, Journal of the
American Academy of Arts and Sciences, from the issue entitled,
"Artificial Intelligence," Winter 1988, Vol. 117, No.
1.
| 
