From: Vladimir Nesov (robotact@mail.ru)
Date: Thu Aug 23 2007 - 18:33:50 MDT
Friday, August 24, 2007, Nick Hay wrote:
NH> That's not true. Suppose there are 5 possible UFAI designs, and we
NH> can protect against 1 of them by setting up this simulation. Then
NH> there is a 1/5 probability we pick the right UFAI. If there are 100
NH> designs, the probability reduces to 1/100. Common designs are easier
NH> to protect against.
It's a bit more complicated, but ultimately this scheme can match a
real instance of the universe (with UFAI) by any number of exact simulations of
it without spending too much resources. Problem is that simulation
includes enormous number of arbitrary parameters, so even given a
single design, for UFAI to expect the same random stuff there should
exist a simulation that also simulates those exact same random events
(such as a sequence generated by background radiation random number
generator within UFAI's universe). Solution is in multiplicity of MWI
branches. Universe can be regarded as continuously branching. Within a
given branch, future states can also be branched, in which case latter
branches are said to be contained in a former branch.
Problem can be defined as follows: starting from a given moment, one
of the branches results in generation of UFAI (this branch includes
all future branches containing UFAIs), and another branch contains at
least one branch containing a FAI. Let the number of branches double
every tau (measure of time). Up to a given moment in time T from
separation of UFAIs' branch, there are 2^(T/tau)
universe configurations that may contain an UFAI. If friendly AI starts a
simulation where random events within the simulation are determined
by random generators in its own universe, different MWI branches of
FAI will generate different simulations corresponding to different
branches of UFAI real universes. Let S*tau be the amount of time
required in FAI universe to produce a tau of branching within a
simulated universe. In this case, from the start of simulation,
by the time t FAI will simulate 2^(t/(S*tau)) configurations, so
to simulate all possible UFAI configurations up to the time T it will
need time t=T*S, that is slowdown factor depends only on simulation
implementation and doesn't suffer from multiplicity of possible
configurations.
I wonder, is expected utility weighted over all possible outcomes
a rational goal to pursue? If system accepts MWI, shouldn't it rank
hightly a scheme where it gambles and performs suicide if it loses?
All its future instances will win, and it has no incentive to infect
as much future branches as possible by its presence, that is to
survive in all branches (which is an evolutionary bias).
-- Vladimir Nesov mailto:robotact@mail.ru
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT