From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sat Mar 31 2001 - 23:28:51 MST
Gordon Worley wrote:
>
> Not only this, it would be in the interest of Friendliness to be
> emotionally stable. Therefore, all AIs that are Friendly must be
> emotionally stable. It would only take a time or two of self re
> programming to get this set.
It wouldn't require any programming at all, self-re or otherwise. I just
can't see the effect Alicia seems to be visualizing - I can't see an
obvious way to implement it on purpose, much less how it would happen by
accident.
How do you implement "fragile feelings", anyway? An AI's evaluation of a
strong Bayesian binding between others' images of verself and the AI's
actual self? Peculiar pictures held by others resulting in sharp
self-reevaluations in ways that have strong qualitative effects on
behavior? I just don't see that happening, and certainly not in ways that
are understandable or predictable using human emotional intuitions. The
beliefs that have large effects on behavior are not *always* the ones that
are most confirmed by immediate experience (and hence too confirmed to be
flipped around by others' opinions) - but as a general rule, yes. And
those *non*-experientially-confirmed beliefs that affect immediate
behavior were almost certainly put there by programmer affirmation,
meaning that it would take sensory information at least as strong as a
deliberate programmer affirmation to trash them.
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT