Re: [sl4] A model of RSI

From: William Pearson (
Date: Wed Sep 24 2008 - 08:24:33 MDT

2008/9/24 Matt Mahoney <>:
> --- On Wed, 9/17/08, Stuart Armstrong <> wrote:
>> So a RSI has to be a statement about the actual architecture of a
>> program, not about the equivalent Turing machine. Your model seems
>> acceptable as a definition, as far as I can tell (there will be
>> others). A heuristic definition of RSI could be a program in an
>> architecture that returns, after some time, to state similar to the
>> one it started with, except with an improvement. The formal definition
>> would be given by specifying this architechture. For instance, you
>> could demand that a program has to start in a certain isolated
>> computer, with a certain amount of free space, always accepting
>> certain inputs. Subject to these constraints, a RSI makes sense.
> Define "improvement". In the context of a real machine (a finite state machine), we could relax the definition of a goal so that utility is a monotonically increasing function of time but reaches some maximum after finite time. Then we could define improvement as reaching that maximum faster or reaching a greater maximum in the same time. However, a finite state machine can only run a finite number of programs, so there can only be a finite sequence of improvements by any definition.
> Perhaps it would help to give some real life examples of what we want to do, for example, robots working in a factory that builds better robots. But the first generation has to know what "better" means. We know that adding more memory or faster processors makes for "better" computers, but we only know that because we are still smarter than both generations. Suppose that the child robot had twice as much memory but the software was unable to use it effectively. How would the robots detect this problem?

I would add some different problems myself, what if you added a faster
processor, but the added processing power wasn't sufficient to justify
the added power consumption. Or it overtaxed your power supply or
overheated your box.

There are many reasons why simply getting faster components may not be better.


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT