From: Ben Goertzel (ben@goertzel.org)
Date: Fri May 03 2002 - 17:14:35 MDT
Hi,
Here is my take on these things, which is not the same as Eliezer's of
course.
As anyone who has worked with complex nonlinear dynamical systems knows,
predicting the stability of a complex system from first principles is a very
hard problem. There is a very high probability that our intuitions about
these things are going to be wrong. All we are giving are heuristic
arguments. Heuristic arguments about weather systems and their stability
were quite popular, until in the 70's Lorenz's older work on chaotic
dynamics in weather systems became well-known. Now that we know there is
some chaos there, we still can't make detailed predictions of the stability
or other dynamical properties of given weather systems. I suspect that the
real study of stability and other dynamical properties of goal systems will
not begin until we have AI's of, say, chimp intelligence to experiment with.
> 1. Are there supergoals other than Friendliness that can be stable for
> SI's?
It seems to me that pure selfishness would probably be a pretty stable goal.
In fact my intuition says this one would be less susceptible to drift than
Friendliness, because of its greater simplicity. Estimating the
Friendliness of an action is harder than estimating the selfish benefit of
an action, so there's more room for drift in Friendliness IMO.
But of course, this is just an intuition...
> 2. If the answer to question 1 is yes, will the first SI created
> by humans
> will have the supergoal of Friendliness? Given that for most people
> selfishness is a stronger motivation than altruism, how will Eliezer get
> sufficient funding before someone more selfish manages to create an SI?
The selfishness of the individual creating the AI is not necessarily
correlated with the selfishness of the AI. A selfish person might well
create an AI that valued humans over itself -- while putting in a special
clause to make it value its creator most of all!
ben g
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:38 MDT