RE: SIAI's flawed friendliness analysis

From: Bill Hibbard (
Date: Fri May 30 2003 - 10:48:46 MDT

On Fri, 30 May 2003, Ben Goertzel wrote:

> I note you have stuck "freedom" in there -- which is related to Eliezer's
> "volition" -- and is a mighty can of worms, which a seed AI is gonna have to
> learn by example since there is no viable theory of it right now...

Yes, learning by example is what intelligence is all about.
We humans learn these things, and machines much more intelligent
than humans will learn these things better than we do.

> This is where a pure reinforcement learning approach *may* be more
> dangerous than a mixed-cognitive-methods approach like Novamente...

I think consciousness and cognitive functions are part of a
robust solution to reinforcement learning. They help minds
create the simulation model of our very general world, in
order to solve the credit assignment problem. Of course, when
you are designing a system like Novamente, you may place them
in different modules. But that doesn't change the fact that
cognitive methods are necessary for effective reinforcement
learning in a complex world.

Bill Hibbard, SSEC, 1225 W. Dayton St., Madison, WI 53706 608-263-4427 fax: 608-263-6738

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT