Re: [sl4] A model of RSI

From: Nick Tarleton (
Date: Thu Sep 25 2008 - 10:44:55 MDT

On Thu, Sep 25, 2008 at 11:22 AM, Matt Mahoney <> wrote:

> The desire not to die causes us to want to produce copies of ourselves with
> the same memories, goals, behavior, and appearance, to be turned on after we
> die. (Whether such a copy transfers your consciousness and becomes "you" is
> an irrelevant philosophical question).

It's not irrelevant to my preferences.

> Once we have the technology to upload, you will see your dead friends
> appear to come back to life. Since you have nothing to lose, you will invest
> in this option, hoping for immortality. The result is a lot of autonomous
> agents with human-like goals, but with options not available to us, such as
> the ability to reprogram their brains. Some will directly optimize their
> utility functions or live in simulations with magic genies. They will die.
> Others will turn off their fear of death. They will also die. Others will
> have the goal of replicating themselves or some variation as fast as
> possible. The copies that fear death and can't change their goals will take
> over. So we are back where we
> started, with an evolutionary process.

The fitness function is not fixed.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT