From: Matt Mahoney (matmahoney@yahoo.com)
Date: Fri Mar 14 2008 - 15:55:18 MDT
If I understand Mark's proposal (which he claims I don't), it is that a group
of agents that cooperate with each other have greater fitness than a group of
agents that fight among themselves. On this point I agree, but it defines
Friendliness very broadly. Humans have no special status. I described on the
AGI list a possible outcome where the Earth is transformed into a Dyson sphere
of gray goo devoid of DNA based life. According to Mark's definition, this
outcome is Friendly because the nanobots are cooperating. He at least did not
post any objection.
This is not my objection either. (The nanobots are achieving their goals, and
since nothing else is alive, their opinions are the only ones that matter).
My objection is that evolution is not a stable process. It lies on the border
between stability and chaos. It is punctuated by mass extinctions, plagues,
and other catastrophes. A state where there is no competition is the worst
kind of catastrophe. Without competition between groups (e.g. wars), there is
no evolutionary pressure to maintain goals that promote group survival.
But maybe that's just my own evolutionary programmed goals telling me that
survival is desirable.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT