**From:** Ben Goertzel (*ben@goertzel.org*)

**Date:** Tue May 09 2006 - 23:15:15 MDT

**Next message:**Ben Goertzel: "Re: Changing the value system of FAI"**Previous message:**Ben Goertzel: "Re: Changing the value system of FAI"**In reply to:**Eliezer S. Yudkowsky: "Re: Changing the value system of FAI"**Next in thread:**Ben Goertzel: "Re: Changing the value system of FAI"**Reply:**Ben Goertzel: "Re: Changing the value system of FAI"**Reply:**Eliezer S. Yudkowsky: "Re: Changing the value system of FAI"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Eliezer,

You wrote:

*> The problem I haven't been able to solve is *rigorously* describing how
*

*> a classical Bayesian expected utility maximizer would make changes to
*

*> its own source code, including the source code that changes the code;
*

*> that is, there would be some set of code that modified itself.
*

*> Classical decision theory barfs on this due to an infinite recursion.
*

I'd like to get a clearer idea in my mind of the precise question

you're thinking about...

Is it like this?

Suppose we have a computer C with a finite memory (containing a

program of size N). Suppose that the computer has the power to

write down a program of size N in its hard drive, and then push a

button causing this new program to be implemented in its main memory,

overwriting the current contents. (I.e., the computer has complete

self-modification power on the software level.)

Suppose that at time 0, the computer has a certain state, which

embodies a program P doing Bayesian expected utility maximization/

Specifically, suppose the program P solves the problem: given a

computer C1 with memory size N1, and a utility function U (with

algorithmic information <= M1 < N1), figure out the optimal state for

the computer C1's memory to be written into, in terms of optimizing

the expected utility over time.

There is then the question whether P can be applied to C ... or

rather, in what generality can P be applied to C ... ?

Is this close to the question you are thinking about?

-- Ben

**Next message:**Ben Goertzel: "Re: Changing the value system of FAI"**Previous message:**Ben Goertzel: "Re: Changing the value system of FAI"**In reply to:**Eliezer S. Yudkowsky: "Re: Changing the value system of FAI"**Next in thread:**Ben Goertzel: "Re: Changing the value system of FAI"**Reply:**Ben Goertzel: "Re: Changing the value system of FAI"**Reply:**Eliezer S. Yudkowsky: "Re: Changing the value system of FAI"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:00:56 MDT
*