From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Jan 04 2004 - 21:40:26 MST
Perry E. Metzger wrote:
>
>>THEN you have a noticeable amount of selection pressure
>
> Before we continue, might I ask you why you have such an astounding
> vitriolic reaction to the idea that evolution might continue?
For much the same reason that Drexler has such a vitriolic reaction to the
idea that nanotechnology requires Smalley fingers; I've been arguing it
too damn long.
People all and for their individual stupid reasons really really want to
believe that AIs are just like humans in some facet or another. Sometimes
it's a technophobe who wants to argue that all generic optimization
processes possess the emotion of resentment and will make self-justifying
speeches to human captives, just like in "The Matrix". Sometimes it's a
technophile who wants to argue that all AIs are necessarily helpful.
Sometimes the motive doesn't have anything to do with it; it's just a
question of people making up interesting-sounding tall tales about the
future, predicting the actions of generic intelligences using their human
empathy brainware to imagine what they would do "in those shoes".
Anthropomorphism. Just about every damn error I have ever seen made about
AI amounts to anthropomorphism. Even the people who think they're
inventing aliens just invent emotionally repressed humans, or humans who
are more strictly selfish, and so on. Sometimes it's because people want
to believe, and sometimes it's because people don't have the technical
knowledge to see the strict assumptions underlying their own minds,
choices, all the systemic behaviors of the world as we know it.
And when it comes time to challenge that anthropomorphism, it usually
consists of pointing out how the human case of something is a particular
instance of a particular kind of natural selection at work.
And then the person says, "Oh, but AIs will be designed by natural selection".
Now natural selection is a deep deep deep topic. This is not widely
appreciated. Scientifically literate people spend too much time arguing
with creationists about whether evolution happens at all; a very simple
argument which can be resolved through very easy, intuitive evidence. But
there is more depth to evolutionary theory than this, far more depth, and
yes math as well, and predictions exceedingly precise, beautifully and
sharply matched by observation. It is not just that humans evolved as
proven by the fossil evidence, but that when I look at the human being, I
see a pattern formed and created, in every shape and element constrained,
not by some generic vague force called "evolution", but by evolution under
particular circumstances, natural selection with particular dynamics, so
that if a wise alien came across a human being floating alone in space,
that human would be known as an evolved thing, and a most particular
evolved thing.
Human beings are one particular case of natural selection. If we ran the
entire process over again, keeping all the dynamics exactly the same,
there's no guarantee things would come out the same. Whatever happened
would be tightly constrained to bear the signature of our particular
dynamics, but that itself is a large area of configuration space... not
all the variables are constrained. You might think of a really huge
volume of possibilities, and natural selection constraining the answer to
be within one very unusual and memorable and distinctly identifiable
bubble, but there still being room for variation inside the bubble.
If we changed the dynamics things would definitely not come out the same.
They couldn't. There'd be room inside the constrained bubble, but it'd
be a different bubble.
There is not one AI scenario I have ever seen proposed that would not
DEEPLY change practically ALL of the dynamics from the human ancestral
environment.
People don't see the dependencies.
But then they still want to infer all the warm and fuzzy, or occasionally,
evil and cunning, but at any rate blatantly anthropomorphic, properties
that started the argument in the first place.
So they try to deductively infer evolution from anything that can be
described as reproduction, then infer absurdly exact similarities to
humans from the supposed presence of "evolution", for a grand double fallacy.
Now, Belldandy knows that these are just my innocent fellow humans, trying
to get through the day. Belldandy knows I've committed much worse
theoretical errors in the past, if formulating an intention that would
have wiped out the human species if carried through is considered "bad",
so they're ahead of me on points no matter how you count them.
But I'm still annoyed, because I'm not Belldandy, and I've been arguing
this too long.
Now maybe you, Perry, do not commit these particular fallacies - though,
down below, you commit one of the classics. And perhaps there are people
who use the word "Friendly AI" correctly and rigorously, who do not happen
to be posting on that topic at the present time, but if you see it misused
enough, you will eventually come to dislike the concept itself, yes? The
strength of an idea is the strength of its strongest proponents; it may be
fun to go around attacking weaker proponents, or strategically wise in
politics, but it is not part of the process of social rationality.
And perhaps one day someone will come along who makes a rigorous argument
for natural selection applying to SI scenarios, and who derives from this
an argument about the properties of most SIs. I am not licensed to rule
this out from hearing it argued incorrectly so many times. The world's
greatest fool may say the sun is shining; that doesn't make it dark outside.
But that day has not yet come.
>>No, so long as you have limited resources
>>AND frequent death to free up resources
>
> Nope. Many species of single celled organisms don't have appreciably
> limited life spans, and yet they still evolve. Besides, though, one
> assumes that death will continue to be frequent in the future,
> especially when less prepared entities meet more prepared ones.
Perry, something has to die, frequently, to free up resources, or there is
no selection pressure. It doesn't mean there's an upper limit on
individual lifespans. It means that there's frequent death or there is no
appreciable selection pressure. It is required by the math, so you can be
sure that if those single celled organisms are evolving, then they, or
something that they eat, is dying. Frequently.
>>AND multiple competing phenotypes with heritable characteristics
>
> As soon as you get a change of any sort, you have multiple classes of
> entities competing.
The heritable characteristics must be preserved with fidelity from
generation to generation; any loss of fidelity is factored into the math
and reduces the selection pressure, perhaps sharply. For selection
pressure to add up over 20 generations there must be fidelity of
inheritance over 20 generations; to add up over 1000 generations there
must be fidelity over 1000 generations.
>>AND substantial variation in those characteristics
>
> That will arise with time.
Pardon me, but this appears to be a statement of pure faith, or of blind
generalization from DNA-based organisms. I have already spoken about the
advantage to generic utilitarian optimizing processes to pay the cost (not
that it even looks all that costly) to exactly preserve their utility
function, both locally and in offspring.
> You only need a little variation at the start
What? This is untrue even of DNA-based organisms, which require a
continuing source of variation.
> -- and even the variation of experience histories of multiple
> entities can be sufficient to provide for advantages
How much of an advantage? A substantial variation in reproductive fitness?
>>AND substantial variation in reproductive fitness
>
> It doesn't need to be substantial. Even slight advantages lead to big
> long term changes. Call it the "compound interest effect".
Slight advantages can be fixed in a population pool *if* they are
persistent and *if* they are iterated over hundreds or thousands of
generations. The slighter the advantage, the more iterated generations
you need.
>>AND correlation between heritable characteristics and fitness
>
> In the DNA world, that's crucial. In the pseudo-lamarkian world of the
> future, it is unlikely to be nearly as obvious an issue. Presumably,
> entities will construct descendents using design rather than blind
> mutation of a blueprint.
And at this point you have tossed away your entire argument without
realizing it, like a theist remarking, "Well, at this point you have to
take the argument on faith; now, the next point is that..."
The patterns we have learned from natural selection as it exists in the
world today are the results of particular dynamics, like the distribution
of heat in an iron plate is the result of particular physics equations,
and if you toss away the dynamics the result does not hang around.
What you have just said is:
"The dynamics of the future are so absolutely different from those of
natural selection that no generalization of any kind can be made from
natural selection as we know it, unless the arguments are re-derived from
scratch."
>>AND this is iterated for many generations
>
> What are "many generations"? People have been able to artificially
> induce astounding changes in animals in a half dozen to dozen
> generations. See, for example, the Russian experiments on
> domesticating wild animals during the middle of the last century.
If you use much sharper selection pressures - artificially inducing huge
variations in reproductive success - then you can get by with fewer
generations. And also generations are not always integers; they are
measured in the percentage of resources freed up and retaken per unit
time. So, like I said before, you require a large global death rate,
inherent limitations on individual lifespan or not.
>>Natural selection feeds on variation and, in feeding on it, uses it
>>up.
>
> "Feeding on it"? "Uses it up"? I don't see that in the real world. I
> see billions of species.
One, this is a standard proverb in evolutionary biology, though I have
paraphrased it.
Two, mutation, copying errors, strand-swapping and sexual recombination
are continually producing *new* variation in the real world. The point is
that once a particular lucky allele becomes fixed in the population as the
result of selection, it doesn't vary anymore.
>>Intelligence is a vastly faster optimization process -
>
> Evolution doesn't require genetic material and literal inheritance to
> work. Companies evolve, societies evolve, investment strategies
> evolve.
Again, you've just tossed away your entire argument without realizing it.
It's like saying, "Well, planets move in ellipses because of physics...
and there are all kinds of physics, quantum physics, relativistic
physics... but as long as things move because of physics they will go in
ellipses."
> Intelligence will doubtless guide much of the design change of
> the future -- but it will also doubtless be tempered with the
> selection pressure that competition and limited resources bring. I see
> no reason to expect that in this regard the future will be any
> different from the present.
It looks to me like intelligence can easily chew up all the variation
available to evolution before evolution gets a chance to feed on it.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT