RE: Metarationality (was: JOIN: Alden Streeter)

From: Ben Goertzel (
Date: Fri Aug 23 2002 - 19:49:08 MDT

Gordon wrote:
> > I think that most scientists and engineers like to delude themselves
> > that
> > they are more rational than they really are. People in this category
> > probably *are* generally more rational than most people, but this
> > doesn't
> > mean they're as rational as they think they are...
> They know more about logic and, therefore, are able to manage to see
> through their own irrationality more often. Usually, though, they just
> get lucky and have good heuristics for limiting bias.

I don't think that's all there is to it. I think that science gets you in
the habit of having your emotionally dear ideas refuted by fact. So it
prepares you emotionally for *any* of your dear ideas -- in any domain --
potentially being refuted.

> This is not rationality!
> What you are doing, Ben, is trying to moderate your irrationality. This
> is no different than what those `rational' scientists do. All this
> allows you to do is stumble onto rational conclusions more often than
> other people.

Actually, I think that moderating your own irrationality can be a very
rational act!!!

I don't see *eliminating* my own irrationality as a viable prospect, except
thru radical neuromodification! It can be *reduced*, and then what's left
can be *moderated*.

> Consider my earlier example with the Randians. If I were thinking like
> you, I'd have a heuristic that would tell me to avoid getting into
> arguments with them. Consequently, I wouldn't ever discuss my thoughts
> on Randianism with them. This is just as irrational as arguing with
> every last one of them! What is rational is to determine *why* you want
> to argue them and if it is a valid reason or just a rationalized reason
> generated by your brain. This is what is essential to rationality: you
> don't have a heuristic to make you more rational in a given situation,
> you think the situation through rationally (using such tools as logic
> and Bayesian statistics) and arrive at the most rational conclusion.

You are proposing conscious rational inference above heuristics as a guide
for action.

I think that human consciousness has limited bandwidth compared to human
unconscious. In situations requiring the integration of a huge body of weak
pieces of evidence, conscious reasoning can't grapple, and you have to make
do with unconscious reasoning (aka intuition). And ridding your unconscious
intuition of emotional biases is the hard part.

It sounds like you're advocating conscious logical reasoning as the solution
to all problems. But conscious logical reasoning only works when there are
a relatively few items of relevant information; otherwise the well-known
limitations on STM size come into play.

Of course, the power of conscious logical reasoning is extended by tools
such as writing, diagramming, programming, and mathematics. However, even
with these tools, it is not as good at the human unconsciousdoing inference
based on a huge number of weak pieces of evidence.

> I sometimes question whether you have made the fundamental shift to
> rationality and if you are just a very logical thinker. At any rate,
> even if you have, you have a *long* way to go to reach a reasonable
> degree of rationality. Or, a better way of putting it is that you
> haven't come very far. I and Eliezer have come a long way, but there is
> still a long way to go, some of it very difficult because it's attacking
> very fundamental ways of human thought. I hope that my book, when done,
> will serve to bring people along to greater degrees of rationality.

Based on my interactions with both of you (and I've interacted with Eli
significantly more than with you), I do not believe that either of you is
significantly more rational than I am.

I've seen you each make many judgments that seemed to me purely emotional...

I *have* met a few people who I thought were more consistently rational than
I am. The most rational person I know is Cassio Pennachin, my chief
collaborator on Novamente. That guy has astoundingly, consistently good
judgment. Jeff Pressing, a former collaborator who unfortunately died
recently, was also a tremendously consistently rational individual.

Interestingly, though, I have not yet met anyone who

a) seemed to me to be more rational than myself
b) also seemed to me to be roughly as creative as myself -- or even *almost*
as creative as myself

I believe that high levels of creativity often go along with willful or
automatic suspensions of rationality. (Of course, this may be rational on a
meta level: one may find that it is rational to sometimes let yourself be
irrational!) But when a mind spends a lot of "creative time" pursuing
unlikely, irrational trains of thought, it often has difficulty shifting
back into a less creative but more rational mode. I have seen this in very
many others, as well as myself.

-- Ben G

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT