Re: Investing in FAI research: now vs. later

From: Matt Mahoney (matmahoney@yahoo.com)
Date: Wed Feb 20 2008 - 19:07:56 MST


--- Nick Tarleton <nickptar@gmail.com> wrote:

> On Wed, Feb 20, 2008 at 5:54 PM, Matt Mahoney <matmahoney@yahoo.com> wrote:
> > The question is whether we become the machine that turns the earth into
> > paperclips, assuming it has some approximation of a copy of our memories,
> or
> > do we just die?
>
> Why would a paperclip maker, or a computronium maker, retain our memories?

The cost of doing so would be negligible, and it would benefit by having more
knowledge. Even if it didn't, it could still generate a plausible script and
you wouldn't know the difference.

> (Even if it did, I would find it little consolation.)

That is your present self talking. Knowing more, you might feel differently.

> > I don't believe the earth would be turned into paperclips because
> paperclip
> > makers would have less reproductive fitness than computronium makers.
> That
> > doesn't affect the basic issue, though.
>
> If the paperclip-maker takes off first and hard enough, fitness
> shouldn't matter. See http://sl4.org/archive/0401/7506.html

Thanks for the link. Yes, you're right. Evolution could be subverted by
cooperation. I would view that as a great disaster (although knowing more, I
might feel differently).

-- Matt Mahoney, matmahoney@yahoo.com



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT