From: Lee Corbin (lcorbin@tsoft.com)
Date: Wed Apr 30 2003 - 20:18:58 MDT
Rafal writes
> Perry Metzger wrote:
>
> > I'll leave the argument about whether anyone has the desire to stop
> > people from performing certain computations that simulate the
> > torturing of sentients to others, and stick to the question of the
> > ability to stop them.
> >
> > It is likely (given the way the world is structured) going to
> > be rather difficult for anyone to stop such things, whether
> > they have the desire or not.
I don't agree. By using enough force, by exponentially analyzing
every aspect of a completely subordinate computation, total submission
can be achieved, even if it means simply stopping the subordinate
computation, or deferring it arbitrarily long.
> > It is trivial to determine that your neighbor has not killed his
> > children (observe the children playing in the yard, say), but it
> > is rather difficult to prove he hasn't performed a particular
> > computation. As I recall, Rice's Theorem would rather get in the way
> > even if you could produce some sort of reasonable rigorous definition
> > of the computations that [are forbidden].
Thanks for that. I had often supposed that determining just what
your client is doing might be intractable, and your Rice's Theorem
http://kilby.stanford.edu/~rvg/154/handouts/Rice.html
seems to confirm that.
> > As a practical matter, even doing a very bad job might require devoting a
> > considerable fraction of the mass of the universe to watching what the
> > rest of the mass was being used for at all times. Not a particularly
> > likely scenario, IMHO.
Well, yes, but it would still be easy to be proactive about it,
supposing one were ruthless enough.
Rafal comments
> ### Since the suffering of simulations would in most cases happen only
> if caused by actions of sentients willing to cause it (=malicious), or
> unwilling to take simple precautions (=neglectful), supervision of the
> volitional aspects of sentient computations should be sufficient to
> prevent most of it. All you need is to look into the equivalent of the
> fronto-orbital cortex of the sentient, check if this part of the system
> contains the requisite rules (e.g. Rawlsian reciprocity), is stably in
> control of the whole personality, and voila, all the computational substrate
> under the personality's control is safe for simulated sentients, even if
> there is space for multitudes of Lee Corbins in simulated dungeons.
Your conditions seem to amount to
(1) determine what is the sentient's equivalent of the fronto-orbital
cortext
(2) determine the set of (hopefully simple) rules or general laws
controlling it
(3) verify that all computation under this entity's control is also
under the control of these predispositions
I might suppose that any one of these three might be intractable, or worse.
(Intuitively, we seem to be running up against the problem of "central
control", the most awful unconscious premise of 20th century thinking.)
Moreover, the Extropian principle of continual self-transformation, if
allowed in the dominated subjects, would require continual re-evaluation.
Perhaps Eliezer will succeed in the much more modest project of enabling
(or evolving) the FAI to keep *itself* from going off the rails, but as
for keeping everyone else from doing X---to the degree that it makes sense
to suppose computation exists outside the SI itself---that's much harder.
Lee
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT