From: Philip Goetz (philgoetz@gmail.com)
Date: Mon Nov 20 2006 - 13:52:03 MST
On 11/15/06, Gordon Worley <redbird@mac.com> wrote:
>
> On Nov 13, 2006, at 3:33 PM, Eliezer S. Yudkowsky wrote:
>
> > Joshua Fox wrote:
> >> Here's a question I have been wondering about. I'm sure that it
> >> has been addressed somewhere. If someone can point me to a URL,
> >> I'd appreciate it.
> >> If multiple near-AGIs emerge, then basic Darwinian arguments show
> >> that the one that reproduces itself the best will have the most
> >> copies; and mutations favoring survival will spread.
> >
> > See the thread "Darwinian dynamics unlikely to apply to
> > superintelligence" in January 2004:
> >
> > http://www.sl4.org/archive/0401/index.html#7506
>
> That discussion is a little long, so to sum it up in the interest
> that people learn rather than not click the link and not learn:
>
> In order to invoke Darwin, you must meet his conditions: inheritable
> characteristics and differential reproduction over sufficient
> generations. In the case of AGIs, there's no reason to suppose
> reproduction, let alone differential reproduction, will occur, hence
> Darwin does not apply.
Darwinian evolution is only one subclass of the competitive systems
that lead to self-preservation and greediness. We can expect this
outcome when there is competition for limited resources, and expansion
of the use of resources. It doesn't matter whether the competition
takes the form of a colony of organisms that increase their resource
share by reproduction, and adapt by evolution; or a few superorganisms
that increase their resource share by growing, and adapt largely by
introspection.
This is why I think the larger goals of friendless will be met not by
trying to control AIs, but by trying to design them to have those
attributes that we value in humans, and in trying to set up the
initial conditions so that the ecosystem of AIs will encourage
cooperation, or at least something better than winner-takes-all.
Most ancient civilizations could not conceive of any government other
than by a single ruler. We seem to be suffering a similar deficiency
of imagination.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT