Re: [sl4] Our arbitrary preferences (was: A model of RSI)

From: Matt Mahoney (matmahoney@yahoo.com)
Date: Sun Sep 28 2008 - 13:07:55 MDT


--- On Sun, 9/28/08, Nick Tarleton <nickptar@gmail.com> wrote:

> On Sat, Sep 27, 2008 at 6:04 PM, Matt Mahoney
> <matmahoney@yahoo.com> wrote:

> > Because rational, goal seeking agents have scalar
> utility functions.
>
> Which may be over histories of the universe, rather than
> moments.

That works fine when agents can't reprogram their inputs or their goals. Meanwhile, we imagine a future where we are immortal and can have everything we want. When technology permits, we can achieve this by living in simulate worlds with magic genies, or by directly stimulating our reward pathways using drugs, wireheading, or software changes in uploaded form. Sorry. Evolution won't allow it.

-- Matt Mahoney, matmahoney@yahoo.com



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT