Re: Darwinian dynamics unlikely to apply to superintelligence

From: Eliezer S. Yudkowsky (
Date: Fri Jan 02 2004 - 23:30:25 MST

Perry E. Metzger wrote:
> That's like trying to get rid of gravitation. So long as there are
> limited resources and multiple competing actors capable of passing on
> characteristics, you have selection pressure...

No, so long as you have limited resources
AND frequent death to free up resources
AND multiple competing phenotypes with heritable characteristics
AND substantial variation in those characteristics
AND substantial variation in reproductive fitness
AND correlation between heritable characteristics and fitness
AND this is iterated for many generations
THEN you have a noticeable amount of selection pressure

Natural selection feeds on variation and, in feeding on it, uses it up.
Intelligence is a vastly faster optimization process - capable of applying
more than 2 bits of optimization per generation, for starters - and
intelligent optimization would rapidly use up all the variation that
natural selection ordinarily feeds on.

If there are different configurations A and B, and B beats A, then a
*powerful* optimization process (unlike evolution) does not need to choose
A or B at random, and then wait to be selected on; an optimization process
calculates the expected utility of A and B, and chooses B. So from the
perspective of natural selection, there's just a flat buffet of B, with no
heritable variation *left over* from the optimizing effect of
intelligence, or trivial variation.

Natural selection is a very meager optimization process, a poor scavenger
of mutations. Recursive self-improvement would eat up all the variation
before evolution has the chance to even look at it.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT