From: Eliezer Yudkowsky (sentience@pobox.com)
Date: Thu Oct 21 2004 - 04:26:13 MDT
Christian Szegedy wrote:
> Eliezer Yudkowsky wrote:
>
>>
>> For example, suppose that we start with a Godel Machine that,
>> meta-level aside, starts with an object-level program that repeatedly
>> presses a button delivering the equivalent of a nasty electrical
>> shock. If the meta-level speeds up the object level so that we get
>> nasty electrical shocks twice as fast, this is not an expected utility
>> improvement.
>
> It depends on your utility function. If the utility function is simply the
> number of received electrical shocks, then it would be a good idea to
> press the buttons as fast as possible.
Yes, I'm aware of that. I was giving an example of a situation where the
starting object-level program fails to fulfill the starting utility
function, so what you want to do is create a new program rather than
optimize the existing one.
> Typically, people working on algorithm theory want to create programs that
> produce solutions to well defined problems as quickly as possible, where
> the objective function is typically easy to compute. This and nothing else
> is the goal of the Goedel machine Find solutions to hard problems as fast
> as possible. The utility function may be a bit different, but it is
> always about performance (speed, memory consumption, quality of solution, etc...)
Presumably it also requires that the solution be correct. This is a
problem because no consistent proof system can prove that the system only
produces correct theorems, so no proof system can prove the usefulness of a
faster, rewritten proof system. Unless you specifically state that you're
trying to prove admissibility in the old system, rather than correctness,
usefulness, or other expected utility criteria on the object level. Hence
the need to patch the Godel Machine.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:47 MST