Re: Changing the value system of FAI

From: Phillip Huggan (
Date: Sun May 07 2006 - 11:59:06 MDT

I guess the program would need to weigh its own available computer resources and divide by the potential aggregate # of human-years of suffering or happiness that hangs in the balance of the decision, and then pick the highest (tractable) pathway solution.

Ben Goertzel <> wrote:
  Eliezer wrote:
> The problem I haven't been able to solve is *rigorously* describing how
> a classical Bayesian expected utility maximizer would make changes to
> its own source code, including the source code that changes the code;
> that is, there would be some set of code that modified itself.
> Classical decision theory barfs on this due to an infinite recursion.

Hmm.... If you feel like taking the time to give more detail on this,
it might be interesting.

Maybe someone on the list will present a different angle on the
problem that will direct your thinking in a different (and useful)
direction... (hey, anything's possible ;-)

Love cheap thrills? Enjoy PC-to-Phone calls to 30+ countries for just 2/min with Yahoo! Messenger with Voice.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT