From: Marc Geddes (marc_geddes@yahoo.co.nz)
Date: Fri Jan 21 2005 - 23:12:12 MST
--- Eliezer Yudkowsky <sentience@pobox.com> wrote:
>
> This breaks down FAI into four problems:
>
> 1) Devise a technical specification of a class of
> invariants, such that it
> is possible to prove a recursively self-improving
> optimizer stays within
> that invariant.
> 2) Given an invariant, devise an RSIO such that it
> proves itself to follow
> the invariant. (The RSIO's proof may be, e.g., a
> proof that the RSIO is
> stable if Peano Arithmetic is consistent.)
is/does. being/becoming. What perhaps you have
failed to realize is that the *process* of proof may
in the end be no different to the proof itself.
Is there, ultimately, any distinction between the
mathematical, the physical and the perceptual worlds?
I think the proof that the RSIO follows the invariant
is uncomputable. But it can still be approximated to
any desired probability of success. And what do I
think the 'proving process' is? In the end I don't
think it's any different to the 'invariant' itself.
> 3) Devise a framework and a formal verification
> protocol for translating a
> human intention (e.g. "implement the collective
> volition of humankind")
> into an invariant. This requirement interacts
> strongly with (1) because
> the permitted class of invariants has to be able to
> represent the output of
> the protocol.
> 4) Intend something good.
Here is my (non-technical) specification of what I
think the invariant is/what it does/is good. is/does.
being/becoming. Again, are you sure that there is
any real distinction between these terms? Are you
sure that (1), (2) (3) and (4) are really seperate
from each other? Think about it!
The Good thing I intend:
http://www.sl4.org/wiki/TheWay
=====
Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:51 MST