Re: Changing the value system of FAI

From: Eliezer S. Yudkowsky (
Date: Wed May 10 2006 - 20:34:15 MDT

Ben Goertzel wrote:
> I suspect that there are, however. For instance, I *suspect* (but
> have not proved) that Godel-type restrictions (appealing to Chaitin's
> algorithmic information based variant of Godel's Theorem) can be shown
> to imply that a system with memory capacity M cannot prove or disprove
> the Friendliness of most AI systems with memory capacity > N.

This doesn't necessarily impose important real-world limitations
because, as I remarked earlier with the bridge analogy, an engineer
operates within the subspace of systems that knowably work.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT