From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Tue Feb 27 2001 - 16:38:30 MST
John Smart wrote:
>
> It seems all complex systems merge symbiotically with the complex systems
> around them, in the biological record, and the more complex they are, the
> more extensive the merging. (Margulis, Symbiotic Planet). The more complex
> systems never remain subordinate, so this argues against the "serving"
> supergoal. Is there merging friendly? Again symbiosis as a mechanism
> produces ever greater local computational complexity, so we might call it
> collectively algorithmically friendly, as an uninterrupted supergoal. Is it
> friendly to the individual systems involved? I think again the record shows
> this, as a function of complexity, but I wish to make that case in detail at
> a later date.
Perhaps ecological systems become friendlier with increasing complexity
(they don't), or humans become friendlier with increasing intelligence
(they do), but that doesn't necessarily tell you how much work is needed
to make the same hold true of AIs. In humans, intelligence is only one
prominent variable in a network of mutually interacting causes, and the
fact that increased intelligence skews the result towards friendliness
doesn't necessarily mean that friendliness is an automatic output of
increased intelligence for minds in general regardless of initial
conditions. The initial conditions for humans are too complex and too
constant for us to draw that conclusion about minds in general. That's
the mistake I made years ago.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT