From: Norman Noman (overturnedchair@gmail.com)
Date: Sun Aug 26 2007 - 09:55:34 MDT
On 8/26/07, Stathis Papaioannou <stathisp@gmail.com> wrote:
>
> On 26/08/07, Norman Noman <overturnedchair@gmail.com> wrote:
> > > They might announce it as soon as they hear of Rolf Nelson's idea.
> >
> > I very much doubt that they WILL announce it, as soon as they hear of
> rolf's
> > idea or at any point afterward. If you believe in god anyway, the idea
> of
> > replacing divine judgment with a machine is blasphemous and accomplishes
> > nothing. In any case, I imagine most people would get to about "ESSAY:
> How
> > to deter a rogue AI by" before they stopped reading.
>
> I guess it's also blasphemous for an organisation to rakes in billions
> from the gullible faithful despite what it says in the Bible about
> camels, wealth and the eye of a needle...
Are you honestly telling me we're going to see a televangelist saying "Give
me your money, and your soul will go to heaven! Simulated heaven, inside a
computer. Here in the real world, heaven and hell don't exist. Hallelujah!"
> > In fact, every special interest group could make a similar proposal,
> >
> > But they won't, and no one would take them seriously if they did.
>
> Why not? Arguably there is more motivation to implement a plan that
> gains you something today than one that might guarantee your
> descendants' future. That's why it's so difficult to deter people from
> burning up fossil fuels and polluting the environment, even if they do
> believe that it is ultimately a bad thing.
Right, because people who don't believe in global warming are sure to
believe in THIS.
> > having only to commit to run the simulation when computing resources
> > > become cheap enough, which eventually they will.
> >
> > Hopefully by that point the computing resources will be regulated such
> that
> > this sort of tomfoolery is not allowed. By the friendly post-singularity
> > whatsit.
> >
I notice you didn't respond to this part at all, so perhaps I should
elaborate on it. There's no reason to expect the church, or PETA, or the ice
cream council, to have the capacity to simulate universes. Now, or at any
point in the future.
I suppose big tobacco could be secretly making an AI in some shady back
room, in order to run a simulation where smokers go to heaven and everyone
else dies of lung cancer, but if you keep such a thing a secret it's useless
and completely insane, like the doomsday device in dr. strangelove.
And the thing is, it's not going to work either way. If they reveal the
plan, and say SMOKE OR DIE! it's only going to make everyone hate them even
more. Actually, it's only going to make everyone laugh at them and think
they're nuts, but assuming they were taken seriously for some reason...
We might not still be fighting the battle, because we might be in a
> simulation run by the God-schmucks (or whoever). You can't tell it's a
> simulation, that's the point.
We're still fighting in the real world piece of the probability pie, which
is inseparable from the fake one, and whether or not we win determines which
piece the final pie is made of. The RAI doesn't have this leverage, it's a
simulation of one possible future being run by another possible future, not
a simulation of the past leading to the future that's simulating it.
> B. Two-way interaction. In rolf's plan, the simulated and nonsimulated
> RAIs
> > are in separate branches and thus the real RAI has no way to stop itself
> > from being simulated.* In your thing, we can go to the house of whoever
> says
> > JOIN ME OR DIE and punch them in the face.
>
> You can do that, but if it turns out that you're in the simulation
> you'll be sent to hell for your trouble.
If you make sure they never build their simulation, then you won't be in it.
> C. Goal conflict. Under your system, by cooperating we help doom a
> > significant fraction of humanity to hell, which for many people would be
> > regarded as Bad Thing. Under rolf's plan, all the AI loses by
> cooperating is
> > the raw material humanity is composed of, which it barely cares about
> one
> > way or the other.
>
> By cooperating and trying to spread the good word you maximise the
> chances that you and everyone else will go to heaven, at minimal cost
> - sort of like Pascal's wager.
Not quite. Under pascal's wager there's nothing we can do to change the
existence of god. He's either there or he's not. Here we can work toward one
option or the other. By cooperating, you may save yourself, but you fund
god's existence and in so doing, help to doom everyone who doesn't
cooperate. This is not minimal cost.
And even the "minimal cost" of praying or lending a futuristic mafia ten
bucks is still much more significant than the cost to the RAI in rolf's
deterrent plan.
Of course, the problem with Pascal's
> wager is that there are so many actual and possible religions that you
> may as well ignore all of them, unless you find good evidence that a
> particular one is true - which translates to evidence that you are in
> fact in a simulation.
If I am an RAI, the evidence that I may be in a simulation is as follows:
"The people who created me were trying to create a FAI. With more luck, they
would have succeeded. The FAI they created would run karmic simulations of
likely RAIs that could have been created instead.
Ergo, the probability I am one of them is XY/(1-X)Z, where X is the
likelihood they'd create a friendly AI rather than a rouge one, Y is the
likelihood a friendly AI would simulate me specifically, and Z is the
likelihood I would be created as a result of real human error."
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT