Re: How to make a slave (was: Building a friendly AI)

From: Stathis Papaioannou (stathisp@gmail.com)
Date: Tue Nov 27 2007 - 06:11:30 MST


On 27/11/2007, John K Clark <johnkclark@fastmail.fm> wrote:

> Of course it would be ridiculous to use anthropomorphic reasoning to
> understand how a toaster works, but not if you used it to understand
> another mind; it is after all the only tool we have for doing such a
> thing, that's why it evolved. At any rate if you want to insult me
> you're going to have to find something new to call me because I don't
> find it insulting to be called a believer in anthropomorphism, just a
> bit repetitive.

You could argue that an AI may have animalistic motives because it
will have been initially designed by animals, or because the AI's with
such motives might be more successful in Darwinian competition. But
you can't legitimately argue that animalistic motives follow naturally
from intelligence. Intelligence has nothing to do with setting
motivations, only with achieving them once they are set.

The term intelligence is sometimes loosely used as in the following:

(a) If you're intelligent, then you won't let people order you around.

What is really meant is:

(b) If you're intelligent AND you don't want people ordering you
around, then you won't let people order you around.

On the other hand, we could have,

(c) If you're dumb AND you don't want people ordering you around, then
you will let people order you around.

or,

(d) If you're intelligent AND you want to please people, then you will
let people order you around.

or,

(e) If you're dumb AND you want to please people, then you won't let
people order you around.

In other words, your willingness to let people order you around is not
necessarily dependent on your intelligence, while your ability to
bring about the appropriate state of affairs given this willingness or
lack of it is necessarily dependent on your intelligence.

-- 
Stathis Papaioannou


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:01 MDT