From: Keith Henson (hkhenson@rogers.com)
Date: Wed Jun 07 2006 - 03:06:04 MDT
At 01:37 AM 6/7/2006 -0400, Michael Vassar wrote:
>Yes analogies are always suspect...
>And existance proofs really do always demonstrate possibility...
>So so long as the existance proof is valid and people really *can* want to
>remain moral and become more intelligent the analogy is simply a waste of
>our time.
>
>If you want to get serious, at the very least you have to make a serious
>case that the robustness of human morality under carefully considered and
>bias corrected self-modification has not been adequately demonstrated (or
>that human morality is not robust under bias correction) and give us some
>reason for seriously doubting it. Succeed in that and you still won't
>have prooved that Friendlyness is impossible, you will just be in a
>position analogous to those who claimed that heavier than air flying
>machines capable of lifting humans are impossible rather than that of
>those who claim that NO object of any size or with any other
>characteristics can fly.
I am sorry to say I don't understand the basis of your complaint against my
post.
"The dire reality is that reproduction cannot be unlimited in a limited
world--so we go *SNIP* to cat gonads. This is good from the moral
viewpoint of a substantial majority of humans."
(I should correct the above to a substantial fraction of western culture
members.)
My point was that more intelligent AIs or upgraded humans may have a
different view of what is moral as we have a different view of what is
moral compared to cats. If you upgraded a cat to human level intelligence
would it think controlling the population of regular cats the way we do was
moral? (I have no idea.)
I certainly had no intent to prove Friendlyness impossible either. I would
be more inclined to attempt to prove it possible or even likely. But I do
think Friendly AIs will have to make some hard decisions if unmodified
humans remain in the world--analogous to the decisions we make about cats
and for the same reason.
I think you may be objecting to my word use. The word "solved" (in
quotes) was in the literary device sense i.e., for the story. I make no
claims at all about this being a solution for the real world outside the story.
Though it does have the flavor of what we do with cats . . . .
Keith Henson
PS. If the list snippers clip it, we should move this to the extopy-chat
list. In preparation I will cc that list.
>>From: Keith Henson <hkhenson@rogers.com>
>>Reply-To: sl4@sl4.org
>>To: sl4@sl4.org
>>Subject: Re: I am a moral, intelligent being (was Re: Two draft
>>papers: AI and existential risk; heuristics and biases)
>>Date: Tue, 06 Jun 2006 21:53:24 -0400
>>
>>At 09:46 AM 6/6/2006 -0700, robin wrote:
>>
>>snip
>>
>>>It blows my mind that any intelligent and relevantly-knowledgeable
>>>person would have failed to perform this thought experiment on
>>>themselves to validate, as proof-by-existence, that an intelligent
>>>being that both wants to become more intelligent *and* wants to
>>>remain kind and moral is possible.
>>
>>>Really bizarre and, as I said, starting to become offensive to me,
>>>because it seems to imply that my morality is fragile.
>>
>>Analogy is always suspect, but consider cats. We treat them as morally
>>as we can.
>>
>>The dire reality is that reproduction cannot be unlimited in a limited
>>world--so we go *SNIP* to cat gonads. This is good from the moral
>>viewpoint of a substantial majority of humans.
>>
>>But I have my doubts about how the cats feel about it. At least it is my
>>observation that intact cats have more interesting personalities.
>>
>>I "solved" this problem in the fiction I have been writing by putting
>>rules on the AIs that they would analyze as being such a good idea they
>>would not want to do otherwise. Namely, no reproduction inside uploaded
>>simulations and no food production by the AIs outside the simulations.
>>
>>And the simulations were so attractive compared to the real world that
>>the big problem was getting enough people to have children in the
>>physical world to keep up a remnant population.
>>
>>(The AIs were constructed without the desire to reproduce and were only
>>brought into existence by physical state humans.)
>>
>>Keith Henson
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT