From: Michael Vassar (michaelvassar@hotmail.com)
Date: Tue Jun 06 2006 - 23:37:58 MDT
Yes analogies are always suspect...
And existance proofs really do always demonstrate possibility...
So so long as the existance proof is valid and people really *can* want to
remain moral and become more intelligent the analogy is simply a waste of
our time.
If you want to get serious, at the very least you have to make a serious
case that the robustness of human morality under carefully considered and
bias corrected self-modification has not been adequately demonstrated (or
that human morality is not robust under bias correction) and give us some
reason for seriously doubting it. Succeed in that and you still won't have
prooved that Friendlyness is impossible, you will just be in a position
analogous to those who claimed that heavier than air flying machines capable
of lifting humans are impossible rather than that of those who claim that NO
object of any size or with any other characteristics can fly.
>From: Keith Henson <hkhenson@rogers.com>
>Reply-To: sl4@sl4.org
>To: sl4@sl4.org
>Subject: Re: I am a moral, intelligent being (was Re: Two draft papers: AI
>and existential risk; heuristics and biases)
>Date: Tue, 06 Jun 2006 21:53:24 -0400
>
>At 09:46 AM 6/6/2006 -0700, robin wrote:
>
>snip
>
>>It blows my mind that any intelligent and relevantly-knowledgeable
>>person would have failed to perform this thought experiment on
>>themselves to validate, as proof-by-existence, that an intelligent
>>being that both wants to become more intelligent *and* wants to
>>remain kind and moral is possible.
>
>>Really bizarre and, as I said, starting to become offensive to me,
>>because it seems to imply that my morality is fragile.
>
>Analogy is always suspect, but consider cats. We treat them as morally as
>we can.
>
>The dire reality is that reproduction cannot be unlimited in a limited
>world--so we go *SNIP* to cat gonads. This is good from the moral
>viewpoint of a substantial majority of humans.
>
>But I have my doubts about how the cats feel about it. At least it is my
>observation that intact cats have more interesting personalities.
>
>I "solved" this problem in the fiction I have been writing by putting rules
>on the AIs that they would analyze as being such a good idea they would not
>want to do otherwise. Namely, no reproduction inside uploaded simulations
>and no food production by the AIs outside the simulations.
>
>And the simulations were so attractive compared to the real world that the
>big problem was getting enough people to have children in the physical
>world to keep up a remnant population.
>
>(The AIs were constructed without the desire to reproduce and were only
>brought into existence by physical state humans.)
>
>Keith Henson
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT