Re: Why friendly AI (FAI) won't work

From: Robin Lee Powell (rlpowell@digitalkingdom.org)
Date: Wed Nov 28 2007 - 11:30:09 MST


On Wed, Nov 28, 2007 at 08:49:39AM -0800, Harry Chesley wrote:
> First, to be useful, FAI needs to be bullet-proof, with no way for the AI
> to circumvent it.

If you're talking about circumvention, you've already missed the
point. An FA no more tries to circumvent its friendliness then you
have a deep-seated desire to want to slaughter babxes.

> This equates to writing a bug-free program, which we all know is
> next to impossible.

I don't know who "we all" is there, but they are wrong.
http://en.wikipedia.org/wiki/Six_Sigma

It's hard, and requires concerted effort, but when was the last time
you hard of a bug in an air traffic control program? It happens,
but it's an extremely rare thing isolated to *particular* ATC
programs; most of them are basically bug free. Same with the space
shuttle. Same with most hospital equipment.

> Second, I believe there are other ways to achieve the same goal,
> rendering FAI an unnecessary and onerous burden. These include
> separating input from output, and separating intellect from
> motivation. In the former, you just don't supply any output
> channels except ones that can be monitored and edited.

OMFG has that topic been done to death. Read the archives on AI
boxing.

> This slows things down tremendously, but is much safer.

No, it's not.

> To save everyone from having to read more postings, let me
> pre-supply some of the replies I'm sure to get to this message:
>
> * Read the literature!
>
[snip]
>
> I don't disagree with any of those, actually, but I'm only likely
> to be convinced I'm wrong by arguments that address my points
> directly.

Why should we go to the effort of doing your research for you? How
arrogant is *that*?

-Robin

-- 
Lojban Reason #17: http://en.wikipedia.org/wiki/Buffalo_buffalo
Proud Supporter of the Singularity Institute - http://intelligence.org/
http://www.digitalkingdom.org/~rlpowell/ *** http://www.lojban.org/


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:01 MDT