From: Rafal Smigrodzki (rafal@smigrodzki.org)
Date: Fri May 21 2004 - 10:32:06 MDT
Eliezer Yudkowsky wrote:
> And finally, what about all the consequences, and categories of
> consequences, that you haven't foreseen? When you imagine the act of
> self-modification, you will imagine only the easily mentally accessible
> consequences of the act, not the actual consequences. Just because you
> can't see the doom, doesn't mean the doom isn't there.
>
### While I acknowledge your points, I am much in favor of
self-modification, or autopsychoengineering (APE). Some of the same
disaster-management techniques that are envisioned for AI, could be
applied to APE-ing. Copies of old structures could be left, with
provisions for division of resources between the new version and the old
version at some time, if neither is willing to merge with the other. The
ultimate judgment of the quality of a given type of modification could
be given by a combination of the economic efficiency of the resulting
mind and the first-person qualia transferred to others - observers
contemplating a modification already tried by intrepid pioneers could
see if they are likely to succeed financially and enjoy it too, which
seems to me to be the ultimate in social goodness (crime bosses aside).
Rafal
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:36 MST