From: Matt Mahoney (matmahoney@yahoo.com)
Date: Sat Mar 15 2008 - 18:49:55 MDT
--- Mark Waser <mwaser@cox.net> wrote:
> > If I understand Mark's proposal (which he claims I don't), it is that a
> > group of agents that cooperate with each other have greater fitness
> > than a group of agents that fight among themselves. On this point I
> > agree, but it defines Friendliness very broadly. Humans have no
> > special status. I described on the AGI list a possible outcome where
> > the Earth is transformed into a Dyson sphere of gray goo devoid of
> > DNA based life. According to Mark's definition, this outcome is
> > Friendly because the nanobots are cooperating. He at least did
> > not post any objection.
>
> I'm sorry. I didn't see this before and I'm posting an objection now. The
> nanobots *are* cooperating with each other but they are stomping all over
> humanity. That is UnFriendly in exactly the same sense that humans
> terraforming another world with a native sentient species would be extremely
> unFriendly on the part of the humans. Humans have no special status. They
> should not be stomped on. They should also be not be stomping on others.
There are a number of ways in which humans could become extinct without our
goals being stomped on. Human goals are appropriate for survival in a
primitive world, not a world where we can have everything we want. If you
want 1000 permanent orgasms or a simulated fantasy world with a magic genie,
then the nanobots go into your brain and your wishes are granted. What
difference does it make to you if your brain is re-implemented more
efficiently as gray goo and your body and world are simulated? You're not
going to know. Does this count as extinction?
> > Without competition between groups (e.g. wars), there is
> > no evolutionary pressure to maintain goals that promote group survival.
>
> And this is where we currently have a fundamental disagreement, I am
> contending that the competition at lower levels or the competition against
> the environment (and, as long as your goals aren't fulfilled, you *ARE*
> competing against your environment to fulfilll them) *IS* sufficient
> pressure to maintain the goal of survival for the competing lower-level
> entity which then trickles up to group survival. Also, if the group is
> self-improving and has any goals, it will then the Omohundro drive for
> self-preservation.
Well, you're right that competition doesn't necessarily imply agents killing
each other. Rather, it is indirect. When an agent acquires a limited
resource needed for its survival or reproduction, it reduces the survival or
reproduction probability of other agents. Also, agents that cooperate (are
friendly) can acquire more resources on average than those that don't. We
agree that evolution favors friendly groups over unfriendly groups.
But we don't really have a choice over whether there is competition between
groups or not. My bigger concern is the instability of evolution, like a
plague or population explosion that drastically changes the environment and
reduces the diversity of life. Some of the proposals for controlling the
outcome of a singularity depend on a controlled catastrophe by setting the
initial dynamic in the right direction. This is risky because catastrophes
are extremely sensitive to initial conditions. But of course we are in the
midst of one now, a mass extinction larger than any other in the last 3.5
billion years. We lack the computing power to model it, and there is no way
to acquire it in time because the process itself is needed to produce it. So
it always stays a step ahead. Sorry for the bad news.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT