**From:** Eliezer S. Yudkowsky (*sentience@pobox.com*)

**Date:** Fri May 12 2006 - 09:55:39 MDT

**Next message:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Previous message:**Eliezer S. Yudkowsky: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**In reply to:**Eliezer S. Yudkowsky: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Next in thread:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Reply:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Eliezer S. Yudkowsky wrote:

*>
*

*> I think some forms of the Conjunction Fallacy will survive in
*

*> better-designed minds, others not.
*

*>
*

*> A mind evenhandedly penalizing a surface description by the
*

*> improbability of each of its conjuncts does not sound very hard to do.
*

*> An AI should not think that Linda is more probably a feminist bank
*

*> teller than a bank teller.
*

*>
*

*> Where a possible cause is mathematically simple but very expensive to
*

*> find - that is, there's a very simple explanation that retrodicts all
*

*> the data, but coming up with that simple explanation is a cognitively
*

*> hard task - then a boundedly rational mind may:
*

*>
*

*> 1) Be asked "What's the probability of B?", and reply with a low
*

*> probability for "B".
*

*> 2) Then be asked "What about A&B?" (where A is the simple cause of B),
*

*> and reply with a higher probability for "A&B?" than was answered for
*

*> "B?", but also updating the probability for B alone so that the new
*

*> probability given to B is higher than A&B.
*

Actually, there's a third case, where you know A, you're asked about C,

and you reply with a low probability. Then you're asked about A&B&C,

where B obviously follows with high probability from A, and C obviously

follows with high probability from B, but B itself is cognitively

expensive to find. Then you may reply with a higher probability for the

entire chain A&B&C while also updating your probability of C.

This, in fact, is how mathematical proof works. You go up to a

mathematician and ask, "Are all left-handed hamsters isomorphic to

Fred?" and the mathematician says "I'm not sure." Then you ask, "Well,

what about the probability that 10 is an even number, from which it

follows that 100 is even, and therefore all left-handed hamsters are

isomorphic to Fred?" and the mathematician says, "Oh, of course, that's

obvious." But the mathematician also updates the probability of the

final conclusion; this does not involve falling prey to an explicit

conjunction fallacy at a single moment in time, like a feminist bank teller.

-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence

**Next message:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Previous message:**Eliezer S. Yudkowsky: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**In reply to:**Eliezer S. Yudkowsky: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Next in thread:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Reply:**Philip Goetz: "Re: The Conjunction Fallacy Fallacy [WAS Re: Anti-singularity spam.]"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:00:56 MDT
*