Re: Metarationality (was: JOIN: Alden Streeter)

From: Eliezer S. Yudkowsky (
Date: Mon Aug 26 2002 - 03:32:11 MDT

Samantha Atkins wrote:
> Can you directly support the implied claim that all real reasoning
> (substitute your doubtless better phraises here) is bayesian or covered
> by BPT? I see you asserting this but it does not strike me as
> obviously true or yet directly argued for much less proved.

Well, learning to see the BPT beneath the surface of all nonaccidental
successes is a skill that takes some work. Generally, you look for the
reason why something works, say "X is evidence about Y for reason Z", or
"X correlates to Y for reason Z", and then, poof, it's governed by the
BPT. In that sense you might call it a tautology... but it's a very
powerful, very useful tautology, especially if you've previously been
thinking of all the surface manifestations of the BPT as different things.

If you like, consider it a challenge: Name a nonaccidental success, and
I'll show you how it's rational or why rationality lies beneath its surface.

>> Yes. And because you use "rational" to denote only this kind of
>> thought, you can get away with telling yourself that "both
>> 'rationality' and 'irrationality' are necessary to thought", thereby
>> avoiding the necessity of getting rid of comforting irrational
>> thoughts.
> This talk of "getting away with" and avoiding getting rid of
> "conforting" irrational thoughts is an invention out of whole cloth.
> It begs the question of whether some of the definitions of "rational"
> used or implied are in fact rational and whether some things some
> speakers claim are irrational or at least arational are not essential
> aspects of thinking/learning/knowing.

Okay. Maybe the question ought to be asked, and answered? Do you think
the definition I'm using of "rational" is irrational, and if so, why? Do
you think that something I've called "arational" is an essential aspect of
thinking, and if so, why? (Note that if it is, then the BPT is underneath
it somewhere, and I was wrong to call it "arational"... If you have a
nonaccidental success that you think is *not* governed by the BPT, drag it
out and let my BPT-detectors give it a sniff.)

> It may be your opinion but you have not justified it nor justified that
> persons arguing these claims are doing it because they find them
> "comforting". I find this line distasteful as it seems to border on
> ad-hominem or some kind of more rational than thou thrust.

The social reaction to all this is straightforward, and predictable... but
all people are *not* equally rational. There's nothing implausible about
the thought that some people might be more rational than other people.
Choosing to speak those thoughts openly is a choice that I've made under
my own goals, of course, but part of those goals is learning to treat
thoughts as representative of truth, and not just means for manipulating
other people... so if thoughts are just representative of truth, why would
truths about which people are more rational than other people be special?
  It's obvious that they have a special status in social interactions.
But since I know about that, to care about it would require a special
decision. Maybe it makes more sense to just pursue the truth as far as it

Incidentally, if Ben succeeds in building an AI and I do not, it would not
be the least bit unusual to suggest that Ben was enormously more rational
than I, as an enabling factor in his being able to do that. The converse
is also true. Among those who dare attempt to meddle in the Singularity
it is neither unlikely nor unexpected that enormous differentials in
rationality might exist. I think that a sufficiently rational mind, even
a human mind, should be able to reason about this as an ordinary fact.
Perhaps I am mistaken.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT