From: Peter C. McCluskey (pcm@rahul.net)
Date: Tue Sep 28 2004 - 14:09:45 MDT
mwdestinystar@yahoo.co.uk (Michael Wilson) writes:
>and computing power. If there's any sort of competition going on, turning
>yourself into a RPOP is the way to win. Unfortunately it also appears to be
>the end of most of the stuff we place moral value on. A universe full of
>perfect rationalists is a universe where all diversity resides solely in
>people's goal systems (which may or may not converge); the qualities of
>'insight', 'creativeness', 'willpower' etc will all dissappear as they are
>defined against flaws, and goal-system properties such as 'compassion' will
>revert to 'did this person have an initial utility function that was
>compassionate under renomralisation'? This is on top of the already known
I'm having trouble figuring out why we should worry about this kind of
problem. We have goal-systems which are sometimes inconsistent, and when
we need to choose between conflicting goals, we figure out which goal is
least important and discard it as an obsolete sub-goal. The qualities you
mention above seem like sub-goals that I would be willing to discard if
they conflicted with more important goals such as happiness.
>The CV question could be glibly summarised as 'is there a likely incremental
>self-improvement path from me to a paperclip optimiser?'. While few people
>like paperclips /that/ much, it seems likely that many people would choose
>to become perfect rationalists without appreciating what they're losing. If
If better information would cause people to choose a different path, then
that suggests that better information would be desirable. But that's a
normal problem with most decisions, and I don't see how the CV creates
a different type of problem in that regard.
Or were you suggesting that we should be upset with the end result if
fully informed people would decide to follow that path?
-- ------------------------------------------------------------------------------ Peter McCluskey | Asimov's 3 laws of robotics considered harmful; www.bayesianinvestor.com | see http://www.asimovlaws.com
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:46 MST