From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Jul 14 2005 - 20:11:25 MDT
Russell Wallace wrote:
> On 7/15/05, Eliezer S. Yudkowsky <sentience@pobox.com> wrote:
>
>>Your argument fails to carry even given its assumptions; it addresses only a
>>small portion of the necessary conditions for natural selection. In
>>particular, you did not address my points about extreme-fidelity replication,
>>insufficient frequency of death to ensure multiple generations, and
>>insignificant covariance of goal system content with replication speed. Any
>>one of these points is sufficient to rule out natural selection as a
>>significant shaper of post-Singularity order, even if we grant slow takeoff,
>>late development, and force of numbers as an even balance to greater intelligence.
>
> Extreme-fidelity replication is easily disposed of, since many members
> of the population will be _deliberately_ engaging in low-fidelity
> replication - namely, self-improvement. Sure, it may be possible to
> self-improve with high assurance of preserving one's goal system, but
> there are going to be individuals for whom that will not be a
> priority, that will accept nonpreservation of goal system for the sake
> of faster replication.
See this post: http://sl4.org/archive/0401/7513.html regarding the observed
evolutionary imperative toward the development of monoliths and the prevention
of, e.g., meiotic competition between genes. Replicating hypercycles were
assimilated into cells, cells assimilated into organisms, etc. In Earth's
evolutionary history there was a tremendous advantage associated with
suppressing internal competition in order to externally compete more
effectively; under your postulates I would expect high-fidelity goal-system
expanding monoliths to eat any individual replicators, much as fish eat algae.
> Frequency of death is an interesting one: it comes down in large part
> to whether offense ultimately beats defense. Unfortunately, the
> potential existence of a weapon called the Nicoll-Dyson laser (see
> Google Groups for references) appears to suggest that it does and
> therefore that this condition is met.
See http://sl4.org/archive/0401/7611.html about the fixed costs of combat and
the possible preference this creates for negotiations, and the motive other
SIs have to punish defectors.
> Covariance of goal system content with replication speed... I think
> this is the one that offers most hope (if not met, it could mean
> survival even of a botched Singularity); but it strikes me as likely
> to be met. In the previous conversation, you suggested that a
> replicator spending 1% of its resources on carrying a payload of
> sentient life might still replicate 99% as fast as one that did not
> carry such a payload... which suffices if each seed starts at a
> different location.
>
> However, if they all start at the same location, then by the time the
> sphere is a million light years across, optimal self-replicators have
> tens of thousands of years to gobble all new resources before sentient
> beings arrive.
Even granting the assumptions, these are not creatures optimized *for*
self-replication. They are merely nonevolved goal systems whose utility
functions do not happen to impede rapid expansion and assimilation of
territory, which is very different from natural selection exerting design
pressure sufficient to create complex adaptations.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT