Re: About "safe" AGI architecture

From: Harvey Newstrom (
Date: Sun Jun 13 2004 - 11:06:20 MDT

On Sunday, June 13, 2004, at 12:07 pm, Ben Goertzel wrote:

>> The things
>> Java removed
>> from C++ do not afaik make it any more powerful, certainly not more
>> succinct, and arguably no easier to prove a program is correct in.
> This is not true. Java lacks pointers and explicit heap memory
> management, and this is what makes it orders of magnitude more feasible
> to analyze Java programs mathematically, than C++ programs.

As I pointed out in my other note, this is not true. The Java syntax
does not give human programmers control over pointers or memory
management. But the binaries produced still have pointers and memory
management. It is just that they were programmed into the Java runtime
system programmers rather than the Java source code programmers. This
just moved the problem, it technically didn't solve anything.

> There is a LOT of knowledge in the area of formal program verification
> and transformation, and plenty of cool academic-ish technology, even
> though very little of this has found its way into the commercial
> software community.

This is an important point! I hope that all AI developers are aware of
and are using all these formal program verification techniques.
Friendly AI cannot be as unreliable as today's typical commercial


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT