Re: Definition of strong recursive self-improvement

From: Keith Henson (
Date: Sat Jan 01 2005 - 14:52:59 MST

Reasoning by analogy is always suspect . . . but . . . I wonder if a newly
hatched AI might not have some of the problems a person with even a wide
ranging knowledge of integrated circuits would have if they were dumped
into a 1900 technology environment?

I.e., even if you know exactly what you need to do, the tools are just not
there to do it In fact, they are several *generations* of tools away from
doing integrated circuits.

Eliezer has mentioned an AI doing a nanotech bootstrap by ordering a mess
of custom made proteins. That might work, or it could be as useful as
ordering a batch of 1900 fountain pins would be to building ICs. We
(collectively) know the route from 1900 to ICs. We don't know a route into

It would seem most likely to me that even the most malicious AI would use
us to bootstrap by making itself mind bogglingly useful.

Of course, so would a friendly AI.

Keith Henson

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:50 MDT