RE: Why FAI Theory is both Necessary and Hard (was Re: SIAI's flawed friendliness analysis)

From: Christian Rovner (
Date: Sun May 11 2003 - 17:02:11 MDT

Gary wrote:
> It's not that I don't trust the people now doing the work, it's just
> that if it's publicized and demonstrated as doable, what's to keep
> parties with less honorable motives from coming in and taking over the
> project?

Cliff wrote:
> *Understanding* smallness and meanness, although probably necessary,
> is not sufficient to *overcome* it. I know at least one person
> capable of acting in ways they know, even as they so act, are "wrong"
> (or less moralistically -- are ultimately unproductive). And I'm
> reasonably certain I'm not the only one.

Let me put it this way. Imagine a *succesful* Friendly AI programmer. Someone
who really understands the process by which a Friendly seed AI will change the
entire universe forever, making it a much better place for everyone. Can you
expect them to get involved in the (relatively) pathetic little quarrels that
are presently taught to us as human History?

-- Christian Rovner
Singularity Institute for Artificial Intelligence,

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT