Re: FAI means no programmer-sensitive AI morality

From: Samantha Atkins (
Date: Fri Jun 28 2002 - 23:58:37 MDT

Ben Goertzel wrote:
>>But it should be equally *true* for every individual, whether or not the
>>individual realizes it in advance, that they have nothing to fear
>>from the
>>AI being influenced by the programmers. An AI programmer should
>>be able to
>>say to anyone, whether atheist, Protestant, Catholic, Buddhist,
>>Muslim, Jew,
>>et cetera: "If you are right and I am wrong then the AI will agree with
>>you, not me."

Of course some breeds of religious people would simply claim
that unless the AI has an immortal soul (or Buddha nature) and
it is capable of communion with the Holy Ghost or some such that
it cannot know about these religious matters at all. As you
say below, a non-empiricist element.

> Yeah, an AI programmer can *say* this to a religious person, but to the
> religious person, this statement will generally be meaningless....
> Your statement presupposes an empiricist definition of "rightness" that is
> not adhered to by the vast majority of the world's population.
> To those who place spiritual feelings and insights above reason (most people
> in the world), the idea that an AI is going to do what is "right" according
> to logical reasoning is not going to be very reassuring.

Hmmm. A lot is hidden by "spiritual feelings and insights" and
"reason". Depending on who you ask many things could be claimed
of and included or excluded from either. It is not at all clear
that such an AI would be any less intuitive or "spiritual" than
humans. It might even be more so for some values of "spiritual".

> And those who have a more rationalist approach to religion, would only
> accept an AI's reasoning as "right" if the AI began its reasoning with *the
> axioms of their religion*. Talmudic reasoning, for example, defines right
> as "logically implied by the Jewish holy writings."

This goes too far. Many more rational approaches to religion
eschew axioms of religion.

> Is an AI programmer going to reassure the orthodox Jew that "If you are
> right *according to the principles of the Jewish holy writings* then the AI
> will agree with you, not me." Or is it going to reassure the orthodox Jew
> that "If you are right according to the empiricist philosophy implicit in
> modern science, then the AI will agree with you, not me."

Probably the latter but not necessarily. It is almost certain
it isn't going to be according to this or that holy writ though.
  One hopes. :-) The AI will find flaws in modern science and
some aspects of the philosophy of science as well as in much
else that came before it.

> You don't seem to be fully accepting the profound differences in viewpoint
> between the folks on this list, and the majority of humans.

>>The real, actual Singularity will shock us to our very core,
>>just like
>>everyone else. No, I don't think that transhumanists and traditionalist
>>Muslims are in all that different a position with respect to the real,
>>actual Singularity - whatever our different opinions about the
>>human concept
>>called the "Singularity".
> Well, let me give you an imperfect analogy here. An LSD trip is an
> experience that often causes one to feel that all the assumptions one has
> made all one's life -- cognitive, perceptual, emotional -- are just
> meaningless constructs. It brings one "beyond oneself" in a really
> significant way. If you've not tripped a lot (and I know you haven't), you
> probably don't understand. (In case anyone is curious, it's been a very
> long time since I took LSD, but the memory is definitely still with me!)
> However, some people can handle this better than others, because some people
> are "more attached to" their own habit-patterns and beliefs than others.

I know a bit about this from my own perhaps partially misspent
youth. I found LSD a useful tool for understanding how my
current personality and other constructs were put together,
blowing them apart and having some degree of freedom to
reconstruct in a way that seemed, well, "better". After a while
it felt as if I went to pretty much the same sorts of states and
saw infinite reiterations of the same basic information. The
trouble was actually making use of what was learned there in the
"real" world. It wasn't scary for me because my sense of, not so
much identity as center, was deeper or different from or more
fluid than most. I believe this sort of fluidity is indeed
essential as the world and one's very self changes more and more

If this is so then some forms of spirituality with their strong
emphasis on stepping beyond individual ego identification and
even teaching means of doing so may actually have a bit of an
advantage over the type of agnostic or atheist materialists who
are firmly identified with their current state, ego and
personality structures.

> In a similar way, I actually think that some humans are going to have their
> minds blown worse by the Singularity than others. Some minds will segue
> more smoothly into transhumanity than others, for example. A mind whose
> core belief is that Allah created everything, and that has lived its whole
> life based on this, is going to have a much harder transition than average;

Ah, but the question is what exactly is Allah? It is left as an
imponderable. For all the believer knows Allah is the name of
an SAI of a bygone era or another species that riched a certain
level of Power and did indeed act in some ways that over time
have become corrupted into the beliefs of today. Or the name of
the SAI under consideration in one of its time-travelling or
simulation experiments on how humanity could have been reached
earlier and perhaps had/have a less troubled history. The point
of this is that beliefing this or that is true of God is about
as nailed down and inflexible as believing this or that about
another imponderable, the Singularity.

Most people have little flexibility regardless of the nature of
their belief systems, especially when it comes to think of
necessity beyond their capacity to grasp.

> and a mind that combines a transhuman belief system with a deep
> self-awareness and a strong sense of the limitations of human knowledge and
> the constructed nature of perceived human reality, is going to have a much
> easier transition than average.

And who is not too attached to current reality/ego structures.

>>Again: We need to distinguish the human problem of deciding how
>>to approach
>>the Singularity in our pre-Singularity world, from the problem of
>>the integrity of the Singularity and the impartiality of
>>post-Singularity minds.
> If a post-Singularity mind rejects the literal truth of the Koran, then from
> the perspective of a Muslim human being, it is not "impartial", it is an
> infidel.

 From the point of view of a Muslim mystic, a Muslim (or anyone)
that thinks/intuits/experiences Truth in so limited a fashion is
the true infidel. :-)

> Your definition of "impartiality" is part of your rationalist/empiricist
> belief system, which is not the belief system of the vast majority of humans
> on the planet.

The most difficult thing for many is to understand that belief
does not generate truth nor is it able to evaluate what is and
is not true.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT