From: Samantha Atkins (samantha@objectent.com)
Date: Wed Sep 26 2001 - 04:16:34 MDT
"Eliezer S. Yudkowsky" <sentience@pobox.com> writes:
> Xavier Lumine wrote:
> >
> > I don't really see why a Greater-than-Human Artificial Intelligence running
> > a simulation would ever deem necessary to insert an event such as the World
> > Trade Center. Frankly, exterminating members of a simulation does not fit
> > into the goal-system of a simulation-controlling GIAI.
>
> Not Friendly minds, at any rate, or any minds with the least concern for
> human feelings. It is possible to construct plausible scenarios in which
> the world is a simulation of noninterventionist and utterly uncaring
> superintelligences. Trying to construct a scenario in which there is any
> degree of interaction between us and the superintelligence runs into the
> basic fact that this world does not look like there are any
> superintelligences active in it. If interacting with a hostile
> superintelligence, I expect the world to look a lot worse; if interacting
> with a benevolent superintelligence, I expect the world to look MUCH
> better. The fact that our world looks like it is being run strictly by
> the laws of physics almost completely disconfirms the hypothesis that it
> is being run by a benevolent interventionist superintelligence - the
> Bayesian probability goes to almost zero.
I don't think this position can bear close examination. Say we develop
a Friendly SI. As part of its working definition of friendliness let
us suppose this SI believes in the self-determination of individual
sentient creatures. Ve believes this at least to the extent of not
simply rewiring those whose current wiring and mindset makes miserable
and/or prone to violence and other forms of suffering. What can be done?
As I have suggested before, one thing that can be done is to so arrange
things that said sentients can work out all of their kinks and problems
taking however long it takes. When they destroy one another either the
destruction takes place in a simulation that the actors do not know is
a simulation (or will not believe it is) or the actors are so well backed
up that they can be resurrected or "reincarnated" as many times as needed
to learn better ways of being and/or be open to some "rewiring". In my
opinion any Friendly SI that comes into being with coexistent lesser
sentients will hit on some variant of this solution. Far from showing
the SI to be uncaring, relative non-intervention and allowing the world
to look like it needs to for the level of beings inhabiting it might be
the very essence and utterly necessary manifestation of caring.
It is possible that making it too obvious this is a simulation or that
there is an SI would ruin the particular learning environment we currently
inhabit. So we get only hints, guesses, supposedly ancient truths
buried in myths and seeming tall tales and so on.
>
> Rationalized religious theories often need to invent exotic scenarios in
> which it's okay to split the Red Sea a few thousand years ago, but not to
> save a cancer victim in the present day. In other words, rationalized
In an age where magical events were more acceptable, splitting the Red
Sea might have made perfect sense to allow in the simulation as a one
shot special effect. Who knows.
> religious theories naturally give rise to highly inconsistent and
> inexplicable theories of when a superintelligence will intervene and why;
> pain and death are "natural tests" in one instance, and horrors in
If we are in a sim then the pain and death are no more ultimately real
than getting fragged in Quake. Except in Quake you aren't fully wired
in.
> another; a miracle is an unacceptable giveaway of intervention today, but
> was a great proof to the glory of God a few centuries ago. Now
> superintelligences may of course be inexplicable, but the problem is that
> these inconsistencies are *not* inexplicable. They have a very clear and
> obvious pattern. They are the result of trying to rationalize a religion
> whose testified miracles are all memetically generated false stories about
> past events.
>From here, you have no way of doing more than surmising that this is the
case.
>In an Internet-connected world it's much harder to generate
> false stories about current events, so the religion's account of the
> present and the past will be inconsistent. Similarly, if you have a world
> entirely governed by the laws of physics, intelligent entities that evolve
> will be stuck in a world that is painful and inconvenient until they fix
> it, and until they fix it, their fuzzier thinkers will try to rationalize
> it - given a certain cognitive architecture, anyway.
>
> Everything I see is consistent with a strictly-physics world full of
> unnecessary pain - including the people trying to explain why it isn't
> really unnecessary.
That might well be exactly what you most need to see to grow and do what
it is you need to do.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT