From: John K Clark (johnkclark@fastmail.fm)
Date: Mon Oct 12 2009 - 22:09:19 MDT
On Mon, 12 Oct 2009 "Robin Lee Powell" <rlpowell@digitalkingdom.org>
said:
> For most (but not all) small programs,
> it's trivial to prove whether they halt or not.
You can show that some programs halt and you can show that other
programs will never halt, but what Turing proved is that in general the
only way to know what a random program will do is to watch it for
eternity and see. The trouble is that eternity is a long time.
> It's also trivial to fix: have a second, very simple thread, proven
> to be infinite-loop-free[1], that just does "Hey, havn't heard from
> the master thread in a while; push the reset switch!".
That is of course arbitrary, it may just be a hard problem and not be in
a infinite loop at all; and yes even a non fixed goal Jupiter Brain
wouldn't know for certain if he was following an infinite loop or not,
but he could certainly make a better guess if he was wasting his time
than your silly egg timer. But no matter, the point is that every time
that egg timer of yours chimes it is giving the AI permission to ignore
its top goal. Even a human would find that security hole to be easy to
exploit, to a Jupiter Brain it would be child's play.
John K Clark
-- John K Clark johnkclark@fastmail.fm -- http://www.fastmail.fm - Email service worth paying for. Try it for free
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT