Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]

From: Eliezer S. Yudkowsky (
Date: Sat Aug 26 2006 - 20:37:31 MDT

Richard Loosemore wrote:
> Worst of all: there is no discussion of the actual topic! All of what
> Yudkowsky has to say is what he thinks I have or have not read 8-|. He
> implies, over and over again, that Loosemore has not read this or that
> .... but it seems like Loosemore's argument goes straight over
> Yudkowsky's head, because he doesn't mention it and doesn't refute it.
> Asonishingly, the argument simply never comes up at all.

Correct. I am not particularly interested in arguing with you on this
topic until you actually read something about the field, just as I am
not interested in arguing with the inventor of a perpetual motion
machine. Do you understand? You are ignorant of an established
experimental field. I did you the courtesy of informing you that you
were ignorant, because human knowledge is vast, and even competent
people cannot always be expected to know when they stumble into
territory that has already been thoroughly explored.

Of course, when someone ignores this advice and insists that they
already know all about it, meanwhile thoroughly demonstrating complete
ignorance, and dropping the names of personal-acquaintance scientists
who turn out not to work in the field, then the offense stops being
innocent. Either it is a deliberate bluff followed by the attempt to
cover it up when caught, or it is the arrogant smart guy's trap of
thinking that their infinitesimal fraction of all human knowledge
constitutes the whole. Either way it is not particularly forgiveable.

It is, in fact, a serious enough offense against scientific ethics,
whether committed intentionally or unintentionally, that I am wondering
whether to ban you from SL4 for it. There are many people on this
mailing list who are not experts in some particular field such as
Bayesian probability or parallel programming, and who need to rely on
the notion that sane-sounding people on SL4 who claim expertise in a
field are probably actually experts. Now here you are, someone who
manages to sound very stuffy and formal like you actually know
something, but who nonetheless claims expertise where no expertise
exists. That is a misleading cue that could deceive even competent
readers, and it threatens the integrity of this mailing list.

> Huhh!!? 8-| These people (Chater and Oaksford, at least) know
> everything there is to know about the entire field of human reasoning,

An absurd claim. No one knows everything there is to know about the
entire field of human reasoning; the existing science (as of 2006) is
too much for a human to learn in a hundred human lifetimes.

> including the subfield that Yudkowsky refers to as "heuristics and
> biasses". The above paragraph is entirely upside down and backwards:
> only someone who had a chronically deficient understanding of cognitive
> psychology would not know the connection between Mental Models and all
> the various human reasoning studies.

The people who write research papers about mental models, are not in
general the same people who write research papers about heuristics and
biases, although they do talk to each other and read the other field's
classic papers. Still, someone who doesn't work professionally in the
field of mental models, yet does know something about the theory of
mental models, may not know anything about the theory of heuristics and
biases. And I am informing you, flatly, that you don't.

Any audience members can either trust me on this, or google "Mental
models" followed by "Heuristics and biases". There's obvious relevance
of one field to the other, but they are very different fields in
practice - even as to the kind of daily work carried out by the
practitioners - and cursory reading in both fields will confirm this.

> The reader may want to contrast Yudkowsky's bizarre remarks, above,
> about Johnson-Laird and Oaksford, with the following excerpts from an
> elementary textbook of Cognitive Psychology:
> In the chapter on "Reasoning and Deduction" there is an entire section
> devoted to Mental Models, which starts with the words:
> "One of the most influential approaches to deductive reasoning is the
> mental model theory of Johnson-Laird (e.g. 1983, 1999)." (p. 516)
> And the following two recommendations in the Further Reading section at
> the end of the same chapter, on page 532, where the book's authors only
> choose seven publications, two of which are by Chater and Oaksford:
> Chater, N., & Oaksford, M. (2001). Human Rationality and the psychology
> of reasoning: Where do we go from here? British Journal of Psychology,
> 92, 193-216. "The authors give a very useful overview of theory and
> research on reasoning."
> Oaksford, M., & Chater, N. (2001). The probabilistic approach to human
> reasoning. Trends in Cognitive Sciences, 5, 349-357. "This article
> provides a succinct account of the main assumptions of the authors'
> probabilistic approach."
> (Excerpts from Eysenck and Keane "Cognitive Psychology: A Student
> Handbook" 5th Edition. 2005).

Anyone familiar with the field of heuristics and biases will recognize
these papers as, while being of course legitimate cognitive science, not
within the particular cognitive science known as heuristics and biases.

Browsing Wikipedia's entry on "cognitive biases" and checking some of
the paper titles should be sufficient to make it clear that the above
papers don't belong in the same category; one of these things is unlike
the other, one of these things doesn't belong.

For benefit of the audience, I remark that this is sufficient to
thoroughly demonstrate Loosemore's ignorance of heuristics and biases to
anyone actually familiar with heuristics and biases.

> Nick Chater won the Spearman Medal, Britain's highest honor to a
> cognitive psychologist, so Yudkowsky's implication that these guys
> probably don't have much insight into the particular part of the field
> that he labels "heuristics and biasses" is laughable.
> Oaksford and Chater could eat Yudkowsky for breakfast and still have
> enough room to wash him down with a few cups of VSOP Darjeeling.

I wouldn't be surprised if Nick Chater is quite familiar with heuristics
and biases, but you, Richard Loosemore, are quite ignorant of it. Nor
may you claim Chater's accomplishments in support of your own expertise.

At this point you're matching one of the classic crank patterns, of
someone who becomes acquainted with a tiny fraction of scientific
knowledge, and who then decides that this science is so amazingly
wonderful that it must be relevant to everything. I recall a fellow who
emailed me *insisting* that I was failing to take sufficiently into
account The Periodic Table of Elements, which he felt was highly
relevant to AI theory.

So I give you three choices, the next time anyone on SL4 informs you
that you are not an expert in an existing science:

1) Read the actual literature of that field.

2) Stop claiming expertise. I don't mean that you have to admit you
were wrong and apologize or anything like that. Just be aware that, by
the law of the list laid down by me, you are unjustly and tyrannically
prohibited from again claiming expertise in that field, either
explicitly or by implication.

3) Leave the SL4 mailing list.

Eliezer Yudkowsky,
SL4 List Owner.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT