Re: "friendly" humans?

From: Eliezer S. Yudkowsky (
Date: Thu Jan 01 2004 - 13:47:50 MST

Wei Dai wrote:
> On Thu, Jan 01, 2004 at 06:13:19PM +1300, Nick Hay wrote:
>> [...] Were evolution to have honestly represented its intentions and
>> made us, simply and directly, want more children (or, rather, want to
>> become ancestors), then derived everything from that as a subgoal,
>> it'd have had better luck when our environment changed. [...]
> I think you missed one of my points. Evolution *did* directly make us
> want more children, not through genes, but through co-evolved symbiotic
> memes.

What? If evolution acts on you, it is through the intermediary of a gene,
gene complex, or other heritable genetic information; at some point in the
past, a heritable gene must have arisen which produced heritable variation
which covaried positively with reproductive success and thereby became
fixed in the population. Evolution is not a pro-child puppet master that
can employ genes on one occasion and memes on another.

> Hence my reference to the "be fruitful and multiply" passage in
> Genesis 1.

If a meme that urges childbearing from a philosophical standpoint is
prevalent over evolutionary time, and there is a gene whose heritable
effect is to make the bearer more susceptible to that meme, and this
gene's other effects do not have negative reproductive effects that
outweigh this (for example by making the bearer susceptible to other memes
that urge celibacy), and the covariance (not the correlation, the
covariance) between the gene's heritable variation in susceptibility and
the meme-bearer's variation in actual reproductive output is large
(meaning that the gene must make people noticeably more susceptible to the
meme, and people who bear the gene must have noticeably more children, so
that the covariance and not just the correlation is large), then the
darwinian dynamic may fix the gene in a population.

It's not clear why memes that particularly urge childbearing would be
around reliably, over evolutionary time, in a population that did not
already have an emotional urge to do so; nor is it clear that this has
actually happened, given that in the West, at least, many major religions
tend to be somewhat down about sex.

> I think evolution was quite clever here, and came to the
> same conclusion that Eliezer reached: the supergoal needs philosophical
> support. It could not be provided in the genes and had to come from
> symbiotic memes because genes can't evolve fast enough to defend
> against parasitic memes.

I think you mean to hypothesize that the specific information was too
complex to arise in the genes as a natural mutation, and so arose in the
form of philosophical susceptibility to memes in general, which in the
ancestral environment tended to urge childbearing.

But humans *do* have an urge for children, as such. Women certainly do.
Men are certainly fond of children once they've been born, and possibly
(I'm not sure) experience a desire to be fathers.

> Unfortunately for the genes, the parasitic
> memes are now evolving faster because of greater communications
> bandwidth between non-relatives so that even the symbiotic memes can't
> catch up.

This sounds confused. Sorry.

The reason humans are escaping the "control" of evolution, in the sense
that the effects now produced by evolved brainware frequently depart from
the original optimization criterion of the darwinian dynamic, is that the
darwinian dynamic cares only about who did in fact reproduce, and weights
selection pressures by the covariance (not correlation, covariance) of
heritable capacities with their reproductive effects *in the ancestral
domain*. Anything that has not yet happened is absolutely invisible to
evolution except insofar as it covaries (not correlates) with a situation
that our ancestors tended to already encounter.

Natural selection has absolutely no foresight except by luck, and humans
are getting steadily farther from the domain in which that luck applies.
The situation in which humans gain control of their own source code did
not occur ancestrally; therefore the darwinian dynamic determines that
outcome only as a side effect of the ancestral variables it did optimize.

Evolution simply didn't think of it. Evolution didn't even try to think
of it. That is just not how natural selection works. What we call
"evolution" is just a statistical regularity in the record of what did in
fact happen.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT