From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri May 12 2006 - 09:55:39 MDT
Eliezer S. Yudkowsky wrote:
>
> I think some forms of the Conjunction Fallacy will survive in
> better-designed minds, others not.
>
> A mind evenhandedly penalizing a surface description by the
> improbability of each of its conjuncts does not sound very hard to do.
> An AI should not think that Linda is more probably a feminist bank
> teller than a bank teller.
>
> Where a possible cause is mathematically simple but very expensive to
> find - that is, there's a very simple explanation that retrodicts all
> the data, but coming up with that simple explanation is a cognitively
> hard task - then a boundedly rational mind may:
>
> 1) Be asked "What's the probability of B?", and reply with a low
> probability for "B".
> 2) Then be asked "What about A&B?" (where A is the simple cause of B),
> and reply with a higher probability for "A&B?" than was answered for
> "B?", but also updating the probability for B alone so that the new
> probability given to B is higher than A&B.
Actually, there's a third case, where you know A, you're asked about C,
and you reply with a low probability. Then you're asked about A&B&C,
where B obviously follows with high probability from A, and C obviously
follows with high probability from B, but B itself is cognitively
expensive to find. Then you may reply with a higher probability for the
entire chain A&B&C while also updating your probability of C.
This, in fact, is how mathematical proof works. You go up to a
mathematician and ask, "Are all left-handed hamsters isomorphic to
Fred?" and the mathematician says "I'm not sure." Then you ask, "Well,
what about the probability that 10 is an even number, from which it
follows that 100 is even, and therefore all left-handed hamsters are
isomorphic to Fred?" and the mathematician says, "Oh, of course, that's
obvious." But the mathematician also updates the probability of the
final conclusion; this does not involve falling prey to an explicit
conjunction fallacy at a single moment in time, like a feminist bank teller.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT