Re: Friendliness, Vagueness, self-modifying AGI, etc

From: J. Andrew Rogers (
Date: Wed Apr 07 2004 - 12:26:47 MDT

> The other issues are that Eliezer doesn't have a lot of time and has
> relatively little actual coding or architecture experience. You may
> need to write a million words and throw them away to be a good
> writer, but you certainly need to write a million lines of code and
> throw them away to be an excellent programmer.

In general, I think the importance of this is often underestimated. A
great many things that sound simple in computer science in theory are
very difficult to implement well in practice on real systems with real
limits and real asymmetries. Elegant theoretical models often run into
the barrier of not having a useful elegant implementation in practice
for anything but toys.

One can easily spend more than a decade steeped in algorithms and code
and still be discovering new deep patterns in that allow tractable and
elegant software implementation. And there is little substitute for
jumping in and getting one's hands dirty. There are a great many things
I can design today that I could not have designed even five years ago,
despite having a ton of real-world experience at the time. You never
know just how many important things you don't know until you actually
have to do them. Software system design is very hard to do right and do
well, and the real trick is knowing the patterns of how theory
translates into implementation, which is typically non-obvious.

I would go as far as to say that even if someone came up with a complete
theoretical and mathematical model for AGI, they could not design an
implementation unless they had a strong background in software systems
implementation. In theory this shouldn't be the case, but in practice
there is because the theory maps poorly to real world machines in a
great many respects.

j. andrew rogers

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT