Why is Friendliness sacrosanct?

From: Alden Streeter (astreeter@msn.com)
Date: Fri Aug 23 2002 - 13:22:37 MDT

As humans, our interests have been shaped by nothing more than evolution.
>From the strictly scientific viewpoint, our only reason for our existence
(and that of all life) is to continue our existence by fulfilling our
biological imperatives. Our intellectual persuits are not technically goals
in and of themselves, but only adaptations of survival methods given to us
by evolution, which still have the ultimate purpose to simply promote our
continued existence.

The entire concept of a "Friendly" AI to me seems irrationally
anthropocentric. Why should our human goals of survival take precident over
any of the AI's goals? Our human goals, including the ultimate goal of
survival, as well as our subgoals (is it valid to apply such AI terminology
to humans as well? I don't see why not) of happiness, pleasure, desire for
knowledge, etc., were determined by our primitive evolution, and are
ultimately determined by our physiology; so when we have the ultimate power
to control our evolution in the future, instead of enhancing our ability to
achieve those goals (apotheosis), why couldn't we instead just change the
goals? But then the conundrum is that our existing goals should determine
what future goals we should want to have instead, but if we change them,
then we might not have wanted those new goals in the first place. So does
that mean that we must be stuck with the primitive goals we have evolved?

So then the same question can be asked of a Sysop-level AI - instead of
working to help humans to acheive their petty, primitive, evolutionarily
determined goals, why not just use its power to change the humans so they
have different goals? Shouldn't it, with its vastly superior intelligence,
be able to think up better goals for the humans to have than the humans have
thought of for themselves? And why should humans not want the AI to have
this type of power? - if the AI changed their goals for them, they would of
course immediately realize that their new goals were the right goals all

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT