From: Wei Dai (weidai@weidai.com)
Date: Mon Jul 14 2008 - 16:14:25 MDT
A couple of months ago Lee Corbin and Eliezer Yudkowsky had a conversation
about superrationality on the extropy-chat mailing list [1]. Eliezer took
the position that two dissimilar SIs may be capable of superrationality when
they know each other's source code. He wrote:
> Yes, but in this case a *motive* exists to *deliberately* correlate
> your behavior to that of your opponent, if the opponent is one who
> will cooperate if your behaviors are highly correlated and defect
> otherwise. You might prefer to have the opponent think that your
> behaviors are correlated, and then defect yourself; but if your
> opponent knows enough about you to know you are thinking that, the
> opponent knows whether your behaviors are really correlated or not.
>
> I'm thinking here about two dissimilar superintelligences that happen
> to know each other's source code.
Putting aside the issue of superrationality for now, I wonder if anyone else
finds it plausible that two dissimilar SIs can know each other's source
code. If we assume that they start out without such knowledge, but each
wants the other gain it, what can they do? Can one SI prove to another what
its source code is? Or, is there some other argument for why SIs might know
each other's source code (beyond "we don't know what SIs might be capable
of, so we can't rule it out")?
[1] http://lists.extropy.org/pipermail/extropy-chat/2008-May/043362.html
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT