From: Eliezer Yudkowsky (sentience@pobox.com)
Date: Thu Oct 21 2004 - 13:19:06 MDT
Michael Roy Ames wrote:
> Eliezer: please give us your definition of the phrase "wrap-around
> reflectivity".
Schimdhuber's original claim, that the Godel Machine could rewrite every
part of its own code including tossing out the need for a theorem-prover, I
would consider wrap-around reflectivity. A human's ability to have this
kind of email conversation - reflect on different possible architectures
for reflectivity - I would consider an even higher kind of wrap-around, and
the kind I'm most interested in. But I prefer to have somewhere to ground
the discussion, like reflective formal systems, otherwise people start
giving verbal answers.
> That is an adamant statement and one that I would agree with (except perhaps
> for the 'ever' part). The ability of humans to perform wrap-around
> reflectivity (as I understand it) is an illusiary one. We perform the same
> algorithmic techniques when working through proofs that a computer would
> use, but fall back on heuristics quickly when we detect our own tower of
> meta becoming absurd.
We are algorithms, but I don't think we're doing the same sort of thing
that a reflective theorem-prover would do. For example, humans can
actually be temporarily confused by "This statement is false", rather than
using a language which refuses to form the sentence. We don't run on
towers of meta. We're doing something else.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:49 MDT