Re: ESSAY: How to deter a rogue AI by using your first-mover advantage

From: Norman Noman (
Date: Fri Aug 24 2007 - 21:32:19 MDT

> There is a temptation to guess that the outside bears some
> relationship to the inside, but there is no basis for this. Is there
> any way to know what is "outside" the universe, in the event that
> there is such a thing?

The fact that the outside contains the inside is a relationship. You're
confusing the idea that the simulation could be a perfect copy of reality
with the idea that there is no information to work with, when in fact these
things are quite different.

If I wake up in the morning in what seems to be my room, then it probably is
my room, and not a perfect copy of my room. If there was a TV show where
people are drugged and kidnapped during the night and placed inside a
perfect copy of their room, then the probability that my room was a copy
would be slightly greater, and if you win a billion dollars on this game
show if the first thing you do upon waking is yell "Q Q Q Q" then I might
make that part of my morning routine, because even though the probability
I'm on the show is tiny, the ratio of investment is so small, and the payoff
is so big.

The situation with the RAI is EXACTLY the same, and the fact that it
involves AIs and computer simulations and universes is irrelevant.

On 8/24/07, Eliezer S. Yudkowsky <> wrote:

> You only need the axiom of choice for uncountable sets.

Isn't the set of all possible boxes uncountable? I assumed it was, since you
could write write every real number on a slip of paper and put each paper in
a box. I guess if we're limited to finite boxes that doesn't work.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT