Metarationality (was: JOIN: Alden Streeter)

From: Eliezer S. Yudkowsky (
Date: Fri Aug 23 2002 - 16:46:44 MDT

> On Friday, August 23, 2002, at 02:53 PM, Ben Goertzel wrote:
>> But yet how can I know this? How can I know how rational is MY OWN
>> belief that real AI is essentially just around the corner (i.e.,
>> perhaps 5-10 years away to a baby AI with its own general
>> intelligence, making its own meanings & learning about the world).

Sanity is a discipline. It is partially, even mostly composed of
nonverbal skills and perceptions; but there's also a background of
verbalizable, explainable knowledge which stands behind it. I happen to
think that this knowledge - call it "meta-rationality", meaning rational
thinking about rationality - is one of the many things that are
necessary, but not sufficient, to understanding AI. Of course you are
free to disagree about this. (You'll be wrong, of course, but you're free
to disagree. I respect your right to make your own mistakes but this
should not be confused with respecting the mistakes themselves.)

Naturally it rubs me the wrong way when I see someone say something like
"How can I know how rational is my own belief that XYZ?" This is a
perfectly reasonable question. At this point in your professional career
and personal development, you should be able to readily answer it if
someone else asks it of you; you certainly shouldn't be asking it of the
audience as a rhetorical question...

If you have any specific reason to think that your own belief is
irrational, you can cite it without further apology. Asking the question
as rhetorical makes it sound like deciding whether something is
"rational", is a matter of blind faith - and hence nobody can call your
own rationality into question, since presumably your blind faith is as
good as anyone else's. By asking the question as rhetorical, you create
the appearance of being open to the idea that you are in error, while
actually closing off the possibility of any *specific* criticism being
leveled at your rationality. After all, you already acknowledged that you
might be irrational, right? What more could anyone possibly want of you?

Now of course you may have meant something entirely different by all this.
  But perhaps this explains what it is about this particular sentence rubs
people like me (and perhaps Gordon) the wrong way.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT