RE: Dispersing AIs throughout the universe

From: Ben Goertzel (
Date: Sat Jan 10 2004 - 06:48:36 MST

I wrote:

> Well, my views have changed a bit. I am now reasonably intuitively
> confident that the revised Novamente design will lead to a Friendly AI.

I should have elaborated on this a bit. The Novamente AI design has been
revised in a way that, among other consequences, appears to make it
compatible with a goal system similar to the one you've described in CFAI.

The concerns I have repeatedly expressed before, regarding the stability of
a Friendly goal structure under repeated self-modification, still exist.
Intuitively, under the modified Novamente design, it seems that these
concerns are not likely to cause a problem. However, one would like to have
more than an intuition on such a matter, of course!

I actually think I see the direction one must go in, theoretically, to make
a rigorous analysis of self-modifiable-goal- structure stability. However,
the math will be very hard, and I am not going to be focusing time on this
in the near future. It may be that semi-AGI-based theorem-proving software
(which I think I know how to make using Novamente, though it's a large
project unto itself) will be useful for exploring this sort of theory.

-- Ben G

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:44 MDT