From: Michael Vassar (michaelvassar@hotmail.com)
Date: Wed Feb 15 2006 - 19:31:46 MST
Recommendation, seriously study some game theory and decision theory (you
should anyway). The first point is a well known discovery. Lots of logic
puzzles/games are based on it.
It's best that the belief that knowledge is inherently desirable be
a) probablistic
b) non-exhaustive, e.g. other values that might sometimes be desirable can
sometimes conflict with knowledge
c) dependent on the relevant definition of knowledge
and
d) recognized as dependent on the relevant utility function
In fact, for most common situations and most common utility functions,
knowledge when defined in certain ways is extrinsically desirable. For a
much smaller set of utility functions it is intrinsically desirable. Since
humans don't have utility functions, the actual situation is very
problematic.
>So, if I understand you correctly, you're saying that you would consider
>the
>ability to detach our happiness from outcomes to others Unfriendly. This is
>to say, that breaking of empathy, or conferring control to a subject of
>what
>makes them happy, or their emotional attachments, would be Unfriendly. I
>disagree here.
I don't say that outright. I say that I would be very skeptical of a FAI
dynamic that did this in a rapid and thorough fashion. It's not proof of
Unfriendlyness, but is evidence for subtle failure of friendlyness. Making
all humans FAIs is almost definitely Unfriendly.
>As I said before, though, you've raised good points, and I'll think further
>on this.
Thanks. I hope it leads to something useful.
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:23:30 MST