From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Thu Jan 17 2002 - 17:20:40 MST
gabriel C wrote:
>
> Actually, I'd rather not have a choice as to whether or not I'm uploaded. My
> stupid anthropomorphic sentiment might just kick in at the wrong moment and
> say "nah, I'll stay normal"...
>
> I don't trust my own meat brain to do the right thing with such a huge
> decision as that, when there is a brain many orders of magnitude more
> intelligent than mine making the offer.
Hm. This is actually a very sensible way of looking at things. I would
make one alteration, though, which is that I'd rely on the
superintelligence to decide whether or not I should be consulted before
being uploaded. That is, rather than "just upload me right away, please",
my own decision reads "I trust you; you can either upload me immediately,
or ask me". Or to make it even finer-grained, Gabriel's decision should
be phrased as "The possibility that I will incorrectly decide not to be
uploaded, if analysis of my mind shows this is a significant probability,
is in my consensual moral judgement a valid reason for uploading me
immediately".
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT