Re: AI Boxing:

From: Stathis Papaioannou (
Date: Wed Jun 04 2008 - 01:39:41 MDT

2008/6/4 Randall Randall <>:
> On Jun 3, 2008, at 8:26 AM, Stathis Papaioannou wrote:
>> 2008/6/3 Vladimir Nesov <>:
>>> This is not a universal law, in the sense that it doesn't apply to all
>>> situations. People can be manipulated to do some things, but not all
>>> things, and not all people, and not equally reliably. There just isn't
>>> any 2-hour-long essay that will make me shoot myself in the head
>> Yes: there must be some possible combination of words that would make
>> you do a particular thing in order for a superintelligent being to
>> discover what that combination of words is.
> The assertion that there is no such combination of words is equivalent
> to the assertion that the human brain is perfectly secure. Given that
> more complex systems have more vulnerabilities (all else equal) and
> that brains were evolved rather than designed, it seems to me to be
> wildly implausible that there are no possible exploits for the brain.

A cardboard box is not perfectly secure, but that doesn't mean that if
I were intelligent enough I would be able to figure out how to open it
just by saying the magic words. And even if it were true that a
particular sequence of words would make a person perform any given
act, it might be necessary to have certain minimal information about
the person to determine what that sequence of words is. A
superintelligent being won't necessarily be able to deduce from your
shoe size what the name of your pet cat is, or what words would make
you commit murder.

Stathis Papaioannou

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT