From: Russell Wallace (russell.wallace@gmail.com)
Date: Fri Dec 31 2004 - 02:50:10 MST
On Thu, 30 Dec 2004 19:15:36 -0800 (PST), Thomas Buckner
<tcbevolver@yahoo.com> wrote:
> Evolutionary trial-and-error tournaments between
> subsystems of itself, with winning strategies
> globally adopted, but still under control of the
> singleton.
Well yes, that's what I believe in too. Evolution as a tool under
control of something that is not itself evolved.
Or put another way: A "self-modifying" entity must, to produce useful
results in the long run, consist of a static part that modifies the
dynamic part, but is not itself modified in the process.
(Biological evolution is not a counterexample (the static part being
the laws of physics + the terrestrial environment) nor is human
culture (the static part being the laws of physics + the terrestrial
environment + the human genome).)
However, that won't "fold the graph in on itself" to make a magic FOOM
as Eliezer appears to believe.
As I understand him to mean it, "recursive self-improvement" means
modifying the whole stack. That's the part I don't believe in; more to
the point, that's the part that would have to work in order for a
"hard takeoff" scenario to be realistic.
I don't think that can happen, but if it could, it would make a
difference to rational planning at the present time, which is why I'm
asking whether there's a reason to believe it could.
- Russell
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT