Re: "friendly" humans?

From: Samantha Atkins (
Date: Fri Jan 02 2004 - 19:21:48 MST

On Fri, 2 Jan 2004 14:37:44 -0800 (PST)
Lawrence Foard <> wrote:

> On Fri, 2 Jan 2004, Perry E. Metzger wrote:
> > There is no definition of "The Good", so it would be rather difficult
> > to transfer the desire to do it to a creature when we can't really
> > determine what "it" is ourselves.
> >
> > (I strongly argue that there is no such thing as "the good" outside of
> > the context of a subjective set of desires on the part of one desirer,
> > but that's another story.)
> Actually the second paragraph is very important. A 'Good' AI would have to
> be one thats sufficiently human to share human desires. What we consider
> pain and suffering is just a certain set of nerve impulses in a certain
> configuration. There is nothing universal about this that implies that
> this pattern of nerve firing is 'Bad'.
I disagree. I also disagree that the "good" is more or less synonymous with what we consider pleasurable (versus painful) or even desirable. A sufficiently intelligent non-human should be able to come to a reasonable grasp of what is "desirable" by humans though.

> You might be able to make a general statement. That which an organism
> acts so as to continue experiencing is good, that which an organism seeks
> to avoid is bad. But it has some problems already.

Starting with "organism", continuing to habitual behavior and leaving out the goals, dreams and aspirations, highest potential and so on that get all wrapped up in the word "good".

> Of course that tells you that heroin is good and work is bad. So your AI
> goes and ships everyone off to the "work free drug place" after making
> some adjustments to there opiate systems so they never gain a tolerance,
> or die from massive doses of all the best drugs a super smart being can
> whip up ]:)

Well, if we were designing Friendly AI by your notion of "good" above that would be a danger. :-)

- samantha

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:43 MDT