**From:** Wei Dai (*weidai@weidai.com*)

**Date:** Mon Jul 14 2008 - 16:14:25 MDT

**Next message:**Krekoski Ross: "Re: [sl4] prove your source code"**Previous message:**Anna Salamon: "[sl4] Volunteers needed for SIAI summer research project"**Next in thread:**Krekoski Ross: "Re: [sl4] prove your source code"**Reply:**Krekoski Ross: "Re: [sl4] prove your source code"**Reply:**Wei Dai: "Re: [sl4] prove your source code"**Reply:**Rolf Nelson: "Re: [sl4] prove your source code"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

A couple of months ago Lee Corbin and Eliezer Yudkowsky had a conversation

about superrationality on the extropy-chat mailing list [1]. Eliezer took

the position that two dissimilar SIs may be capable of superrationality when

they know each other's source code. He wrote:

*> Yes, but in this case a *motive* exists to *deliberately* correlate
*

*> your behavior to that of your opponent, if the opponent is one who
*

*> will cooperate if your behaviors are highly correlated and defect
*

*> otherwise. You might prefer to have the opponent think that your
*

*> behaviors are correlated, and then defect yourself; but if your
*

*> opponent knows enough about you to know you are thinking that, the
*

*> opponent knows whether your behaviors are really correlated or not.
*

*>
*

*> I'm thinking here about two dissimilar superintelligences that happen
*

*> to know each other's source code.
*

Putting aside the issue of superrationality for now, I wonder if anyone else

finds it plausible that two dissimilar SIs can know each other's source

code. If we assume that they start out without such knowledge, but each

wants the other gain it, what can they do? Can one SI prove to another what

its source code is? Or, is there some other argument for why SIs might know

each other's source code (beyond "we don't know what SIs might be capable

of, so we can't rule it out")?

[1] http://lists.extropy.org/pipermail/extropy-chat/2008-May/043362.html

**Next message:**Krekoski Ross: "Re: [sl4] prove your source code"**Previous message:**Anna Salamon: "[sl4] Volunteers needed for SIAI summer research project"**Next in thread:**Krekoski Ross: "Re: [sl4] prove your source code"**Reply:**Krekoski Ross: "Re: [sl4] prove your source code"**Reply:**Wei Dai: "Re: [sl4] prove your source code"**Reply:**Rolf Nelson: "Re: [sl4] prove your source code"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:01:03 MDT
*