Re: [agi] Re: Superrationality

From: Ben Goertzel (
Date: Thu May 25 2006 - 05:43:37 MDT

Hi Eliezer,

> I worked out an analysis based on correlated computational processes -
> you treat your own decision system as a special case of computation and
> decide as if your decision determines the output of all computations
> that are similar to the decision. Or to put it another way, you don't
> choose as if you believe that multiple instantiations of an identical
> abstract computation can have different outputs. This can be formalized
> by extending Judea Pearl's causal graphs to include uncertainty about
> abstract computations, and reworking Pearl's surgical formalism for
> "acts" accordingly, which in turn is justified by considerations that
> these margins are too small to include.
> I haven't published this, but I believe I mentioned it on AGI during a
> discussion of AIXI.

Yes, that must be the long-ago discussion I remember...

Your approach sounds as if it may be conceptually equivalent to the
analysis I've come up with ... but the approach you describe is
definitely mathematically different in the details as I don't use
causal graphs or Pearl's surgical formalism, etc.

Relatedly, I just asked Douglas Hofstadter if he knew of any formal
work on superrationality, and he pointed out the book

"Paradoxes of Rationality and Cooperation:
Prisoner's Dilemma and Newcomb's Problem", edited by Richmond
Campbell and Lanning Sowden (University of British Columbia Press,

but said he wasn't sure if there was anything fitting the bill in
there or not.... I'll check it out and see.

-- Ben

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT