From: Nick Tarleton (nickptar@gmail.com)
Date: Sat Apr 12 2008 - 22:57:24 MDT
On Sat, Apr 12, 2008 at 10:00 PM, Vladimir Nesov <robotact@gmail.com> wrote:
> On Sat, Apr 12, 2008 at 5:03 AM, Eliezer S. Yudkowsky
> <sentience@pobox.com> wrote:
> >
> > People sure are in a rush to hack the utility function all sorts of ways...
> > probably because they don't understand what this little mathy object
> > *means*; it's the set of things you really, actually care about.
> >
>
> But how do we figure it out? Why must utility be, say, additive by the
> number of people, as you suggested? Such rule sounds completely
> arbitrary. World-with-humans should be somehow observed and
> extrapolated, and principles of this extrapolation is an important
> problem to figure out, but before that is done it's not clear what the
> result should be.
Well, like he said, it's *what you care about*, which is not arbitrary
(or, rather, what you currently care about is contingent, but the fact
that you already have values means you can't pick any random
extrapolation), and which people have reasoned about for quite a
while. Even now, isn't it much more plausible that the more
intelligent, more rational, more knowledgeable you would have an
additive utility function over people than, I don't know, sinusoidal;
and that the same would be true of other humans?
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT