From: Peter C. McCluskey (pcm@rahul.net)
Date: Mon Oct 04 2004 - 14:51:35 MDT
chealey@unicom-inc.com (Chris Healey) writes:
>The situation you mention is also a limited wireheading event, since
>happiness is effectively an aggregate metric, contributed to by a
>myriad of mental sub-modules indirectly suporting maximum reproductory
>success. By elevating to a super-goal priority the maximization of
>such a metric, you've introduced a strange loop into your goal system.
I only said that happiness was a more important goal than most. I didn't
claim that it should be treated as a super-goal, and I don't think I
understand myself well enough to know how to describe my super-goal.
Resolving conflicts such as those between happiness and a long life is
a normal (even if unpleasant) task.
>That might not be a good idea if you consider consistency to be a
>desireable thing.
I consider consistency to be a good subgoal.
mwdestinystar@yahoo.co.uk (Michael Wilson) writes:
>'Fully-informed' isn't as simple as it sounds. To be fully informed about
>issues too complex for you to work out for yourself, something more
>intelligent
>than you has to inform you. If they're too complex for you even to
>comprehend,
I didn't intend to imply that becoming fully informed was simple. Omniscience
is sufficiently unusual that bemoaning its absence seems odd.
>something more intelligent than you has to take your preferences and apply
>them to generate an opinion on the complex issue. CV works by extrapolating
>you forward to get the more intelligent entity that advises the earlier
>versions. The problem is that the extrapolation might snuff out something
>you would consider important if you knew about it in a way that the
>non-extrapolated version of you can't recognise and the extrapolated version
>of you doesn't report. My conclusion is that to avoid this the extrapolated
>volitional entities in CV need to interact with the (Power-class) AGI running
>the process as an effective 'neutral party' to detect these sort of
>discontinuities, but that further complicates the problem of not letting the
>knowledge that the CV process is a simulation affect the result.
Ok, that's a clearer explanation of a possible problem. If that's the
biggest problem with CV, I'm not terribly scared of CV (although I'm still
thinking about the alternatives).
-- ------------------------------------------------------------------------------ Peter McCluskey | Read FactCheck.org often to see which presidential www.bayesianinvestor.com | campaign has distorted the truth more.
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:46 MST