RE: FAI means no programmer-sensitive AI morality

From: Ben Goertzel (
Date: Sun Jun 30 2002 - 11:41:28 MDT


> > You seem to have this idea that there is some kind of
> > "meta-rightness" standard by which different ethical standards can be
> > judged more or less correct.
> >
> > There is no such thing.
> You seem awfully certain about that. (Heh. I finally got to use that
> line.)

Well I'm certain that no human has proposed such a thing in a sensible way,
and highly (though not totally) certain that no human is going to in the
next few decades.

Maybe an AI will someday ;)

> Ben, you're a member of a species that has spent the last few hundred
> thousand years arguing politics and morality with occasional pauses to
> eat and sleep. Is it really that strange if I suggest that there are
> complex functional adaptations which influence your choice of
> declarative moral beliefs?

Of course there are, I am a biological being, not totally detached and

My desire to see the human race survive is itself a "complex functional

> This is actually a rather Zen situation; you have meta-ethics but you
> don't know you have meta-ethics. I wish now I had some formal training
> in Zen methods of getting people to realize that they possess a
> subjective experience by just having them *see* it, without a lot of
> verbal argument.

If you attack me with a stick, watch out -- I bite!!

> Let's start with a moral question. Some people even today, though
> thankfully not as many as there a few generations ago, believe that
> people of certain races (or at least, what they regard as "races") are
> intrinsically worth less than others. You have a different morality
> under which race makes no difference to intrinsic worth. Now I'm not
> asking you why you believe these other people are wrong, because if so
> you'll just answer "Because their morality conflicts with mine"; rather
> I'm asking you why you don't share their morality. If morality is
> genuinely arbitrary then one mapping of sentiences to intrinsic value is
> as good as any other; why is your morality different from theirs?

Being "worth less" is ambiguous...

If the argument is that blacks are extremely less intelligent than whites
(an argument made in the past by many people), this is an empirically
testable statement, and has been refuted [yes, I know there is a slight IQ
difference among races, with whites getting higher than blacks and orientals
getting higher than whites -- but this is not the sort of thing I'm tallking

If the argument is that blacks don't have souls, then I guess it's outside
the domain of experiment and logic...

I'm not sure I would agree that "morality is genuinely arbitrary." I would
say that there is no objective scientific or logical way to judge one moral
system versus another. Because any system of judging presumes some
"criterion of merit"; the choice of criterion of merit will then determine
which moral system is better.... of course, one can then ask which criterion
of merit is better

However, it could nonetheless be the case that highly intelligent systems
tend toward certain moral systems, as opposed to others. Just as modern
technological culture tends toward different moral systems than tribal

-- Ben G

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT