From: Charles D Hixson (charleshixsn@earthlink.net)
Date: Sat Feb 25 2006 - 22:48:12 MST
On Saturday 25 February 2006 09:26 am, Philip Goetz wrote:
> On 2/25/06, Ben Goertzel <ben@goertzel.org> wrote:
> > At some point my colleagues and I may need to try hard to solve that
> > decision problem in a more rigid sense -- if I have a powerful AGI on
> > hand and I have to decide whether to set a switch that will let it
> > start self-modifying and hence potentially move toward hard takeoff.
> > I am not at that point now....
>
> This is a key problem with Friendly AI, though... You have to test
> your programs to learn anything, to progress towards AI. You will
> have to have programs that learn, for you to progress towards AI. We
> may very well reach the point where we need to build self-modifying
> programs, in order to progress further towards AI, long before those
> programs are actually smart enough to be dangerous.
>...
>
> - Phil
This *ISN'T* a sharp distinction.
We have actually ALREADY reached the point where many computer programs
self-modify. I can practically guarantee that they aren't friendly, as they
don't even have any model at all of the world external to the computer. It's
hard to be friendly if you don't know that anyone's there. (A chain of
increasingly less relevant examples of self modifying code follows.)
P.S.: Check out: Evolutionary algorithms, Genetic Programming, Self
Optimizing Systems, and Self Organizing systems. You might also look into
Core War and Tierra. These are all (currently) quite primitive, but they are
actual self-modifying code.
P.P.S.: For another example, out of my own experience, back around 1975 I had
the occasion to study a sort library being used at LBL (we were renting time
on their computer). A piece of it was coded in assembler, and when I
investigated it, it turned out that one particular instruction within it
would be modified when it was called to do different comparisons depending on
input values. I.e., the assembler code re-wrote itself. This was the first
time that I encountered it, but it was considered a very "elegant" thing to
do, to save a few words of run-time memory by using self-modifying code. It
was hard to do it in a High Level Language, like Fortran, however, so the
routines that would do it were usually in assembler.
Additionally: Pretty much every LISP program I've ever encountered used
self-modifying code. It's one of the basic techniques. When the program is
a list, and your language CALLS itself a list processor, it just begs to be
used that way. Think of this WRT every language that has an EVAL command,
but it can even be done in C if you work at it a bit. (At most you need to
execute a shell script that compiles and links code that your program
modifies while it's running.)
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT