Re: Changing the value system of FAI

From: Ben Goertzel (
Date: Tue May 09 2006 - 23:15:15 MDT


You wrote:
> The problem I haven't been able to solve is *rigorously* describing how
> a classical Bayesian expected utility maximizer would make changes to
> its own source code, including the source code that changes the code;
> that is, there would be some set of code that modified itself.
> Classical decision theory barfs on this due to an infinite recursion.

I'd like to get a clearer idea in my mind of the precise question
you're thinking about...

Is it like this?

Suppose we have a computer C with a finite memory (containing a
program of size N). Suppose that the computer has the power to
write down a program of size N in its hard drive, and then push a
button causing this new program to be implemented in its main memory,
overwriting the current contents. (I.e., the computer has complete
self-modification power on the software level.)

Suppose that at time 0, the computer has a certain state, which
embodies a program P doing Bayesian expected utility maximization/

Specifically, suppose the program P solves the problem: given a
computer C1 with memory size N1, and a utility function U (with
algorithmic information <= M1 < N1), figure out the optimal state for
the computer C1's memory to be written into, in terms of optimizing
the expected utility over time.

There is then the question whether P can be applied to C ... or
rather, in what generality can P be applied to C ... ?

Is this close to the question you are thinking about?

-- Ben

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT