Re: Adaptation brings unFriendliness

From: Gordon Worley (
Date: Wed Nov 15 2006 - 15:04:31 MST

On Nov 13, 2006, at 3:33 PM, Eliezer S. Yudkowsky wrote:

> Joshua Fox wrote:
>> Here's a question I have been wondering about. I'm sure that it
>> has been addressed somewhere. If someone can point me to a URL,
>> I'd appreciate it.
>> If multiple near-AGIs emerge, then basic Darwinian arguments show
>> that the one that reproduces itself the best will have the most
>> copies; and mutations favoring survival will spread.
> See the thread "Darwinian dynamics unlikely to apply to
> superintelligence" in January 2004:

That discussion is a little long, so to sum it up in the interest
that people learn rather than not click the link and not learn:

In order to invoke Darwin, you must meet his conditions: inheritable
characteristics and differential reproduction over sufficient
generations. In the case of AGIs, there's no reason to suppose
reproduction, let alone differential reproduction, will occur, hence
Darwin does not apply.

Even if you do suppose that AGIs will reproduce, it's unlikely that
there will be sufficient generations for Darwin to work his magic,
since the AGIs will surpass the critical point at which they tile the
universe with paperclips or lead us off to hug fluffy bunnies well
before a sufficient number of generations passes.

And if the AGIs don't reach that critical point quickly enough to not
pass a sufficient number of generations, then we will most likely
have ended in an existential stall.

P.S. Speaking of fluffy bunnies, I now have one. I could definitely
hug him forever.

-- -- -- -- -- -- -- -- -- -- -- -- -- --
                Gordon Worley
             Phone: 352-875-5808
e-mail: PGP: 0xBBD3B003

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT