Re: I am a moral, intelligent being (was Re: Two draft papers: AI and existential risk; heuristics and biases)

From: Keith Henson (
Date: Tue Jun 06 2006 - 19:53:24 MDT

At 09:46 AM 6/6/2006 -0700, robin wrote:


>It blows my mind that any intelligent and relevantly-knowledgeable
>person would have failed to perform this thought experiment on
>themselves to validate, as proof-by-existence, that an intelligent
>being that both wants to become more intelligent *and* wants to
>remain kind and moral is possible.

>Really bizarre and, as I said, starting to become offensive to me,
>because it seems to imply that my morality is fragile.

Analogy is always suspect, but consider cats. We treat them as morally as
we can.

The dire reality is that reproduction cannot be unlimited in a limited
world--so we go *SNIP* to cat gonads. This is good from the moral
viewpoint of a substantial majority of humans.

But I have my doubts about how the cats feel about it. At least it is my
observation that intact cats have more interesting personalities.

I "solved" this problem in the fiction I have been writing by putting rules
on the AIs that they would analyze as being such a good idea they would not
want to do otherwise. Namely, no reproduction inside uploaded simulations
and no food production by the AIs outside the simulations.

And the simulations were so attractive compared to the real world that the
big problem was getting enough people to have children in the physical
world to keep up a remnant population.

(The AIs were constructed without the desire to reproduce and were only
brought into existence by physical state humans.)

Keith Henson

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT