From: Eliezer Yudkowsky (sentience@pobox.com)
Date: Sat Oct 09 2004 - 11:57:40 MDT
Ben Goertzel wrote:
> Eliezer wrote:
>
>> It's quite easy to prove that his premise 2 is false. You cannot
>> correctly solve the problem "Will Ben Goertzel solve this problem by
>> choosing the answer 'no'?" Feel free to use an appropriate
>> diagonalization lemma if you object to the self-reference.
>
> Eliezer, what I object to is your using "Ben Goertzel" in your problem
> definition without fully formalizing the concept of "Ben Goertzel."
You are a reflective system constructed in such a way as to allow explicit
self-reference, permitting me to name you to yourself without providing you
a copy of your data. Or do you seriously mean to tell me that you don't
know what I refer to by "Ben Goertzel"?
> If you try to formalize "Ben Goertzel" using computation theory, then
> you're already assuming the thing Penrose is trying to prove -- so the
> most you can do is give a Penrosean "reduction ad absurdum" of the
> assumption that "Ben Goertzel" can be formalized using computation
> theory.
> Diagonalization is odd here here because we don't know anything about
> the cardinality of the set of possible statements, since we don't know
> the cardinality of the set of possible beings such as "Ben Goertzel",
> who are postulated to live in some huge space of uncomputably complex
> entities.
I just meant diagonalization in the sense of, "When Ben Goertzel is asked
the question produced by substituting 'When Ben Goertzel is asked the
question produced by substituting X for "X" in X, will his answer be no?'
for 'X' in 'When Ben Goertzel is asked the question produced by
substituting X for "X" in X, will his answer be no?', will his answer be no?"
> Penrose would say that the correct formalization of "Ben Goertzel"
> involves some unknown physics, and plausibly involves uncomputably large
> sets of propositions...
Penrose requires that there be no way to describe a human with any kind of
physically realizable map such that I can isolate a variable X in the map,
copy the map and perform the diagonalization trick. Otherwise I can still
present you with the question.
Penrose thinks we know that Godel's Statement is true when we don't, and
that's a definite error on his part, and my reason for saying that Penrose
misunderstands Godel's Theorem.
Penrose also asserts with no proof whatsoever that humans are not subject
to diagonalization, and backs it up by asserting that humans are
uncomputable (and presumably physically indescribable); he gives us
absolutely no reason to believe this assertion, but uses it as his argument
that humans have a mystical ability (why? he hasn't shown that we can
*solve* our diagonalization, just argued that we somehow physically don't
have one), which in turn is his sole basis for asserting humans to be
uncomputable.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:46 MST