Re: [agi] Re: SIAI's flawed friendliness analysis

From: Eliezer S. Yudkowsky (
Date: Fri May 16 2003 - 22:13:12 MDT

Ben Goertzel wrote:
>> I will be ecstatic if the problem of Friendliness is solvable by
>> someone who genuinely wants to solve it. The problem given a hostile
>> programmer is... how can I put this... INSANE. If you want
>> technological advancement on the problem of trust, build a better lie
>> detector using advances in neuroimaging, signal processing, and
>> pattern recognition, and run the programmers through it. I hereby
>> volunteer.
> But Eliezer, what questions are you going to ask the programmer under
> the lie detector? To know what questions to ask you need to have your
> values precisely formulated. Your suggestion is a good one, but it
> simply pushed the issue of value-articulation into the process or
> programmer-quizzing....

That wasn't an attempted answer to the issue of value-articulation; it was
an attempted answer to Bill Hibbard's problem of "Friendliness
enforcement" on hostile programmers. I don't know how to solve the
enforcement problem given a hostile programmer. Presently, it looks
impossible. That's the only answer I can give.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT