From: Michael Vassar (michaelvassar@hotmail.com)
Date: Fri Dec 09 2005 - 09:31:47 MST
Look, it's very simply. Telling an AI not to take actions is an ethical
injunction. It's a more powerful version of the 3 laws, but has all the
same problems. Ultimately "take actions" is not well defined, especially as
distinct from "think" either for a human or for an AI but especially for the
latter. Processing inputs necessarily involves effecting the outside world.
Something without a goal system is not an AI. The story about what such an
"AI" would do implies that it has a goal system and that the creators easily
load in other subgoals.
>Lets say Mr. A want ice cream. Some part of his brain “says”: “I want ice
>cream.” Some other part of his brain has the definition of ice cream. Some
>other part can infer things. I.e.: it can infer that he remains seated his
>odds of getting ice cream are lower than if he goes to his fridge. Various
>other parts do various things. The important thing is that only the
>“wanting” part can initiate action
This is a hideously crude analogy from folk psychology to AI behavior.
>Now, we remove the "wanting" part(s) without damaging any other parts.
In a Fantasy story, but not in a sf story informed by modern popular
science, not to mention not in reality.
>“Readout” displays thoughts crossing his mind. “Send” is the thoughts we
>send through the interface.
This reminds me of the "Black box" idea from early SIAI speculation
>Readout: <empty> Send: What is ice cream? Readout: <definition of ice
>cream>
[Playing along], and why would the AI want to think about what ice cream was
when asked?
>Send: How can you increase your odds of getting ice cream?
Are you giving it the goal system "maximize user's knowledge of how to
increase the odds of getting ice cream?" or "name an action that increases
odds of getting ice cream" or something different
>Readout: Maximum “ice cream getting” odds will occur if I go to the fridge.
That's not what the above goals do.
The first goal kills/remakes the user and probably destroys the world as a
side effect.
The second goal does nothing, because you didn't tell the "AI" when to do
it. Hey, how did the AI learn to understand natural language anyway? To
model the world? Did it do all this without any independently directed
action?
>Send: Do you want ice cream? Readout: No
Why, without a goal system, is it even 'motivated' to answer?
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT