From: Brian Atkins (brian@posthuman.com)
Date: Thu Feb 16 2006 - 10:57:05 MST
turin wrote:
> Psychopaths have
> different cognitive architecture which ...
> makes them pathological liars, and prevents them from understanding what it
> is that happens when someone experiences emotion.
>
...
> This is the problem with friendly SI. I am afraid that if we do not allow
> them to understand first hand subjective experience, we could produce
> psychopaths
>
turin, if you understand the first part of your message, that it requires an
extremely specific cognitive architecture in order to produce the kinds of
behavior you're worried about, then you should understand that the 2nd part of
your message doesn't make complete sense. You're anthropomorphizing potential
AGI architectures and creating a false dichotomy - just because an AGI might not
have a fully normal human-like design doesn't conversely imply that it must then
be designed using a typical psychopath mind design. There are likely an infinite
number of possible mind designs, and only a small handful of designs probably
match perfectly the typical human psychopath blueprints.
It's far better I think to think of AGIs as "clockwork" optimization processes.
Software, running on a computer. If there's no line of code causing it to tick a
certain tock, then it doesn't. Things don't magically happen or go wrong (or
well) just because you "feel" they might. Putting a hairy monkey face on it in
your mind will only lead to mistakes in thinking about it.
-- Brian Atkins Singularity Institute for Artificial Intelligence http://www.intelligence.org/
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT