From: Gwern Branwen (gwern0@gmail.com)
Date: Wed Dec 31 2008 - 22:58:50 MST
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
On Wed, Dec 31, 2008 at 11:55 PM, Norman Noman wrote:
> (rolf's gambit is a method for cooperation between powers in different
> worldlines via mutual simulation, gone over in detail in this earlier sl4
> thread: http://www.sl4.org/archive/0708/16600.html)
>
> (if you haven't read that thread, you really should, it's probably the most
> interesting thing to ever come out of this list)
>
> I was thinking today about a puzzle. Let's say you're a friendly AI, and
> you're going to enact rolf's gambit. But before you do that, you take over a
> few solar systems, and you discover obvious proof that your world is a
> simulation. For the sake of argument, let's say it's an indestructible tuba
> on the third moon of saturn.
>
> The question is this: assuming you continue with rolf's gambit, do you
> include the tuba in your subsimulations? Why or why not?
An interesting question. I think I would reason as follows.
If we find a tuba, then we know we're in a simulation. All of a
sudden, our attitude towards the gambit changes. Before, we acted
believing that all kinds of universes were possible; now we're
restricted to that small subset of universes in which entities would
simulate entities like ourselves, and Rolf's gambiteers* make up a
distressingly large fraction of that subset - much larger than they
were before. Thus, we/the AI now has that much greater an incentive to
act nicely than before.
How does that affect the sub-simulations? I am not sure. Intuitively,
I want to say, 'No - You dummy, if you give the RAI any usable way to
discern between reality and simulation, then deception maximizes its
gains - it'll act nice when it finds itself in a universe with tubas,
and it will act rogue in non-tuba-containing universes!'
But the problem is this view is for a FAI standing in reality; I'm not
sure how to deal with the higher levels. Why did *they* choose to add
a tuba? Presumably they didn't have to, presumably it was optional.
The FAI knows the gambit is not directed at it, so perhaps it can
assume that the upper levels are allied with it; which would suggest,
analogous to 'Bayesians cannot disagree', the FAI should just trust
the upper levels and imitate them and add tubas to its simulations.
* can I coin this nickname along the lines of "Pierson's puppeteers"?
- --
gwern
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEAREKAAYFAklcW5cACgkQvpDo5Pfl1oK9NACaAi5s8gLKyzcv+XzcJja7CnuY
QhMAoIFhpyT/t9dtE/Sdr1sS9RfqfpKR
=D/Sy
-----END PGP SIGNATURE-----
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT