From: Joshua Fox (joshua@joshuafox.com)
Date: Mon Nov 13 2006 - 12:06:54 MST
Here's a question I have been wondering about. I'm sure that it has been
addressed somewhere. If someone can point me to a URL, I'd appreciate it.
If multiple near-AGIs emerge, then basic Darwinian arguments show that the
one that reproduces itself the best will have the most copies; and
mutations favoring survival will spread. (Reproduction here means building
the next generation of technology, based on the previous system and perhaps
with its help.)
Yet clearly mutations that involve destroying and/or using the resources of
(potential) competitors are often adaptive. Thus, AGIs that are not only
unFriendly but downright aggressive will emerge.
Note that this has nothing to do with evolutionary algorithms for AI
development, nor is this an argument that aggression or any sort of goal is
necessarily built into intelligence. It is just a straightforward Darwinian
argument that adaptive systems tend to produce "redness in tooth and claw."
I suppose that in a hard takeoff, the leading AGI could gain so much
(possibly Friendly) power to make all questions moot. But otherwise, doesn't
the above suggest that we do have some idea of the direction in which AGIs
will tend to develop within the space of possible intelligences, and that
it's not a good one? This would be yet another strong reason to be concerned
for AGI Friendliness.
Joshua
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT