Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]

From: Eliezer S. Yudkowsky (
Date: Fri May 12 2006 - 09:31:44 MDT

Joshua Fox wrote:
> Eliezer S. Yudkowsky wrote:
>> .... the way that some people apply their theories of human
>> irrationality to the larger processes of cognition (in this case,
>> judgments of future scenarios), and come to conclusions as if teh
>> Conjunction Fallacy, and the general lack of logical reasoning skills,
>> were the main determinants in those analyses.
> Because flaws in rationality such as the Conjunction Fallacy are not
> explicitly adaptive to the human evolutionary environment, might it be
> that these are necessary side-effects of practical General Intelligence?
> Every design requires compromise in some parameters, and perhaps this is
> an /essential /compromise for the first generation of AGI.

I didn't write that, Loosemore did.

I think some forms of the Conjunction Fallacy will survive in
better-designed minds, others not.

A mind evenhandedly penalizing a surface description by the
improbability of each of its conjuncts does not sound very hard to do.
An AI should not think that Linda is more probably a feminist bank
teller than a bank teller.

Where a possible cause is mathematically simple but very expensive to
find - that is, there's a very simple explanation that retrodicts all
the data, but coming up with that simple explanation is a cognitively
hard task - then a boundedly rational mind may:

1) Be asked "What's the probability of B?", and reply with a low
probability for "B".
2) Then be asked "What about A&B?" (where A is the simple cause of B),
and reply with a higher probability for "A&B?" than was answered for
"B?", but also updating the probability for B alone so that the new
probability given to B is higher than A&B.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT