Re: Definition of strong recursive self-improvement

From: Keith Henson (hkhenson@rogers.com)
Date: Sat Jan 01 2005 - 14:52:59 MST


Reasoning by analogy is always suspect . . . but . . . I wonder if a newly
hatched AI might not have some of the problems a person with even a wide
ranging knowledge of integrated circuits would have if they were dumped
into a 1900 technology environment?

I.e., even if you know exactly what you need to do, the tools are just not
there to do it In fact, they are several *generations* of tools away from
doing integrated circuits.

Eliezer has mentioned an AI doing a nanotech bootstrap by ordering a mess
of custom made proteins. That might work, or it could be as useful as
ordering a batch of 1900 fountain pins would be to building ICs. We
(collectively) know the route from 1900 to ICs. We don't know a route into
nanotech.

It would seem most likely to me that even the most malicious AI would use
us to bootstrap by making itself mind bogglingly useful.

Of course, so would a friendly AI.

Keith Henson



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT