From: Ben Goertzel (ben@goertzel.org)
Date: Wed Feb 15 2006 - 13:11:42 MST
Eli,
> You can't "add Friendliness". It adds requirements like determinism and
> verifiability to the entire architecture. It rules out entire classes
> of popular AI techniques, like evolutionary programming,
I have never seen you give a convincing argument in favor of this
point, though I have heard you make it before.
I don't see why it must be true that an AI system which is Friendly in
the sense that one can prove its probability of Friendliness is
greater than 99.99% (to pull a number from a certain well-known
orifice), must necessarily be "deterministic" in all its operations.
Firstly, I am not really sure what you mean by "deterministic", since
of course even techniques that are typically considered stochastic
such as evolutionary programming are actually deterministic when
implemented on digital computers (assuming one is not using a quantum
random number generator or any other weird tricks). Perhaps you mean
something like: "a process whose outcome is relatively easy to
approximately predict given an approximation of its initial state"?
In the rest of this email I will use the terminology "hard-to-predict"
rather than "nondeterministic" to avoid confusion when discussing
algorithms like evolutionary programming.
It seems to me plausible that a FAI system could utilize evolutionary
programming or other hard-to-predict techniques in a subsidiary role,
e.g. for "speculative concept creation", and then utilize a more
easily-predictable subsystem to decide how and whether to use the
output of the hard-to-predict subsystem.
After all, a Friendly AI is going to have to deal with hard-to-predict
sensations and reactions from the external world. If it can maintain
Friendliness in the face of external difficulty-of-prediction, I don't
see why it would be unable to maintain Friendliness in the face of an
internal hard-to-predict subsystem.
I think it is more plausible to conjecture that an FAI would have to
have a relatively easy-to-predict subsystem in a "controlling role"
(although I realize that specifying the notion of "controlling role"
is a very tricky matter, I hope the intended conceptual meaning of the
statement is clear).
This point is not just a minor technical point, I think it also
pertains indirectly to your claim that Friendliness cannot plausibly
be added onto an AGI architecture. I do not believe you have made a
convincing argument in favor of this assertion. Potentially, one
could have a multi-component AGI system without guaranteed
Friendliness --- but one could then figure out how to insert a
Friendliness-guaranteeing subsystem into the AGI system in a
"controlling role", thus making the overall architecture Friendly.
Of course, this latter point is not just theoretical for me because I
think it might be possible within the Novamente architecture.
Plausibly (but quite speculatively), within some future Novamente
version, it might be possible to transition from a NGF (non-guaranteed
Friendly) Novamente version to a GF (guaranteed Friendly) version via
replacing the inference control" subsystem with a
Friendliness-compliant inference control subsystem, while leaving the
other parts of the system untouched or close to untouched.
For example, the evolutionary learning component of Novamente might
make up some whacky possibilities for possible actions --- but the
inference control system would be in charge of running the inferences
that decide whether these actions should be taken, and this is where
the judgment of Friendliness-compliance would be implemented.
[Unfortunately I do not currently know how to implement
Friendliness-compliance within a probabilistic-inference-control
subsystem in a computationally tractable way, though.]
All in all, I find that your statements about the relation between FAI
and AGI and the needed capabilities and psychologies of researchers in
these areas are predicated on your particular hypotheses about FAI and
AGI, for which which you have not argued all that convincingly. I
respect your perspective and find it interesting but after many years
of listening to you carefully and reflecting upon your writings and
statements I still am not convinced!!
-- Ben
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:23:30 MST