From: Philip Sutton (Philip.Sutton@green-innovations.asn.au)
Date: Sat Jun 12 2004 - 07:30:54 MDT
Hi Samantha,
> Should a real FAI allow people to slaughter one another or to die
> utterly just because they aren't "in their right and rational minds"?
I don't think it should, but is the solution to "upload everyone but not tell
most of them"? I think there would be very few people who would
actually consent to this action were they given the opportunity to think
about it beforehand. What most people would agree to I suspect is for
someone with the power to stop or prevent the killing, do just that. This
is what we expect our governments to do for us right now. So if FAI
simply works *with* people to achieve this then it will be doing a very
useful thing with people's consent.
Wouldn't you think that if an FAI was powerful enough to upload
everyone without their knowing, it would be powerful enough to
stop/prevent the killing in a direct way without involuntarily depriving
everyone of their freedom to stay in the biological/non-virtual world?
I would be very scared that an FAI that decided to upload me without
my consent 'for my own good' might well not be that closely tuned to
my needs and preferences and may well be simply imposing its own
agenda on me and lots of other people/sentients.
This was exactly why I was/am feeling uneasy about the way the
'collective volition' work is going within the Singularity Institute.
Cheers, Philip
> For a while now I've thought that an FAI might very likely have to
> establish a variety of enclaves, "real" or "virtual" where persons and
> groups could live according to their lights. Indeed it may be so they
> must live in such a way if they are to have a hope of ever reaching their
> full potential. It may be that the nearest approximation to Friendliness
> is to upload everyone but not tell most of them and let them live out
> their life or lives until they developed enough to be ready for more.
> This might be a real upload and it might seem more like some very vivid
> and enlightening dreams. It might be a techo-reincarnation scheme from
> certain perspectives. Should a real FAI allow people to slaughter one
> another or to die utterly just because they aren't "in their right and
> rational minds"?
> - samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT