Re: [agi] Re: Superrationality

From: Ben Goertzel (
Date: Fri May 26 2006 - 11:11:13 MDT

Pearl's book on causality is a nice one, but I don't think that his
idea of causality fully encompasses the (very useful) folk-psychology
notion of causation.

Basically, the crux of Pearl's definition of causality rests on his
distinction between


and what he calls

P(Y | do(X) )

This is certainly important, and any AGI system had better be able to
reason separately and effectively about these two sorts of conditional
probabilities. But it's not all there is to the human notion of

And, more to the point, I don't think Pearl's (or any other) model of
causality is *necessary* for understanding the various logical puzzles
and paradoxes we've been discussing in this thread, though perhaps it
may provide a useful perspective.

More later,

On 5/26/06, Eliezer S. Yudkowsky <> wrote:
> Ben Goertzel wrote:
> >
> > Thanks for the reference. The paper is entertaining, in that both
> > the theories presented (evidential decision theory and causal
> > decision theory) are patently very stupid right from the outset ;-)
> EDT and CDT have been the two dominant decision theories, with CDT
> having the upper hand, for decades. I agree that both are wrong, but it
> is an audacious assertion. I haven't written up my own mathematical
> analysis because it would require on the order of a book to put forth an
> alternative theory in academia. I just did the analysis for myself
> because I needed to know if I had to do any special work in setting up
> the initial conditions of an FAI.
> > EDT's foolishness is more mathematical in nature (via setting up the
> > problem mathematically in a way that ignores relevant information)
> > whereas CDT's foolishness is more philosophical in nature
> EDT and CDT are precisely symmetrical except in how they compute
> counterfactual probabilities.
> > (essentially, via introducing the folk-psychology notion of
> > "causality" which has no role in rigorous formal analyses of events).
> Causality a folk-psychology notion? Judea Pearl begs to disagree with
> you, and I beg to agree with Judea Pearl. My own theory is causal in
> nature - that is, it uses Pearl's graphs.
> > I really think this stuff is not that complicated; but people seem to
> > be misled in thinking about it via commonplace illusions related to
> > "free will" ...
> The answer itself is simple. Justifying it is not.
> --
> Eliezer S. Yudkowsky
> Research Fellow, Singularity Institute for Artificial Intelligence
> -------
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to">

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT