Re: Why friendly AI (FAI) won't work

From: Robin Lee Powell (
Date: Wed Nov 28 2007 - 11:30:09 MST

On Wed, Nov 28, 2007 at 08:49:39AM -0800, Harry Chesley wrote:
> First, to be useful, FAI needs to be bullet-proof, with no way for the AI
> to circumvent it.

If you're talking about circumvention, you've already missed the
point. An FA no more tries to circumvent its friendliness then you
have a deep-seated desire to want to slaughter babxes.

> This equates to writing a bug-free program, which we all know is
> next to impossible.

I don't know who "we all" is there, but they are wrong.

It's hard, and requires concerted effort, but when was the last time
you hard of a bug in an air traffic control program? It happens,
but it's an extremely rare thing isolated to *particular* ATC
programs; most of them are basically bug free. Same with the space
shuttle. Same with most hospital equipment.

> Second, I believe there are other ways to achieve the same goal,
> rendering FAI an unnecessary and onerous burden. These include
> separating input from output, and separating intellect from
> motivation. In the former, you just don't supply any output
> channels except ones that can be monitored and edited.

OMFG has that topic been done to death. Read the archives on AI

> This slows things down tremendously, but is much safer.

No, it's not.

> To save everyone from having to read more postings, let me
> pre-supply some of the replies I'm sure to get to this message:
> * Read the literature!
> I don't disagree with any of those, actually, but I'm only likely
> to be convinced I'm wrong by arguments that address my points
> directly.

Why should we go to the effort of doing your research for you? How
arrogant is *that*?


Lojban Reason #17:
Proud Supporter of the Singularity Institute - ***

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:01 MDT