From: Marc Geddes (marc_geddes@yahoo.co.nz)
Date: Wed Aug 17 2005 - 23:09:00 MDT
--- Ben Goertzel <ben@goertzel.org> wrote:
> Godelian arguments can show, at most, that for any
> finite computational
> system there is some limit (of theorem-proving
> power) beyond which
> self-improvement can't lead the system.
>
> However, this limit can be vastly larger than
> anything comprehensible by
> humans, and certainly more than enough to cause our
> destruction in a manner
> similar to us stepping on an ant colony...
>
> So, Godelian arguments are obviously no use for
> FAI...
>
> -- Ben G
>
>
>
I disagree Ben. I think Godel is the key to this
whole thing! As you know, I'm desperate to try to
prove Universal Morality and I've been lunging and
stabbing wildly at anything that might save the day
for altruism. Godel is my last straw! ;)
For how can an AI self-improve unless it can
*understand it's own system* ? You yourself have been
thinking about this problem (recall your ITSSIM). A
self-improving AI, it seems to me, has to maintain
temporal continuity with past versions of itself - how
can it be sure that the future version of itself is an
improvement on its current version? It seems to me
that the only way it can do this is by somehow
*understanding it's own system*.
An AI which did not understand it's own system could
not be sure that any modification it made to itself
would be an improvement. Lack of understanding in
recursive self-modification would probably quickly
destroy the system altogther, since it would make a
fatal blunder.
My idea for 'Godel to save the day for friendliness',
is simply that the unfriendly ai's can't understand
their own systems properly and hence they can't
self-improve.
Now, I'm sure a paperclip maximizer could have
*complete awareness* of its own system. BUT COMPLETE
AWARENESS IS *NOT* THE SAME THING AS COMPLETE
UNDERSTANDING! It is not just complete AWARENESS of
one's own system that is needed for self-improvement,
it is complete UNDERSTANDING.
Perhaps only a friendly AI could have complete
understanding of its own system?
--- Please vist my website: http://www.riemannai.org Science, Sci-Fi and Philosophy --- THE BRAIN is wider than the sky, For, put them side by side, The one the other will include With ease, and you beside. -Emily Dickinson 'The brain is wider than the sky' http://www.bartleby.com/113/1126.html Send instant messages to your online friends http://au.messenger.yahoo.com
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:23:01 MST