From: Stathis Papaioannou (stathisp@gmail.com)
Date: Wed Jun 04 2008 - 01:39:41 MDT
2008/6/4 Randall Randall <randall@randallsquared.com>:
>
> On Jun 3, 2008, at 8:26 AM, Stathis Papaioannou wrote:
>
>> 2008/6/3 Vladimir Nesov <robotact@gmail.com>:
>>
>>> This is not a universal law, in the sense that it doesn't apply to all
>>> situations. People can be manipulated to do some things, but not all
>>> things, and not all people, and not equally reliably. There just isn't
>>> any 2-hour-long essay that will make me shoot myself in the head
>>
>> Yes: there must be some possible combination of words that would make
>> you do a particular thing in order for a superintelligent being to
>> discover what that combination of words is.
>
> The assertion that there is no such combination of words is equivalent
> to the assertion that the human brain is perfectly secure. Given that
> more complex systems have more vulnerabilities (all else equal) and
> that brains were evolved rather than designed, it seems to me to be
> wildly implausible that there are no possible exploits for the brain.
A cardboard box is not perfectly secure, but that doesn't mean that if
I were intelligent enough I would be able to figure out how to open it
just by saying the magic words. And even if it were true that a
particular sequence of words would make a person perform any given
act, it might be necessary to have certain minimal information about
the person to determine what that sequence of words is. A
superintelligent being won't necessarily be able to deduce from your
shoe size what the name of your pet cat is, or what words would make
you commit murder.
-- Stathis Papaioannou
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT