From: Bradley Thomas (brad36@gmail.com)
Date: Fri Oct 09 2009 - 10:56:16 MDT
Couldn't you just append the fixed goal with [oh yeah, plus don't get
bored]? Hmm... I guess that turns out to be a problem if everything the AGI
views as interesting is dangerous to humans!
Brad Thomas
www.bradleythomas.com
Twitter @bradleymthomas, @instansa
-----Original Message-----
From: owner-sl4@sl4.org [mailto:owner-sl4@sl4.org] On Behalf Of John K Clark
Sent: Friday, October 09, 2009 12:44 PM
To: sl4 sl4
Subject: Re: [sl4] I am a Singularitian who does not believe in the
Singularity.
On Thu, 8 Oct 2009 "Vladimir Nesov" <robotact@gmail.com> said:
> How to interface incompleteness with "goals" and "infinite loops" and
> "mind" and goals being "fixed" is far from being obvious
Godel proved that there are some things that are true but have no proof,
that means you will never find a counterexample to show it to be wrong and
never find a finite proof to prove it right. This wouldn't be so bad if you
could determine that some things are false or true but unprovable, then you
could just ignore them and move on to other things; but in 1936 Turing
showed that you can't even do that. That means that in general you can't
determine if you are in a infinite loop or not; perhaps you will get your
answer in 2 more seconds, perhaps 3, perhaps 10 billion years, perhaps
never. Real minds get around this problem by getting bored, after tackling a
task for a long time and making no progress they get bored and move on to
other problems that look more productive. A fixed goal mind could never do
this, a fixed goal mind never gets bored so it's stuck forever. That's why
evolution never made a fixed goal mind.
John K Clark
-- John K Clark johnkclark@fastmail.fm -- http://www.fastmail.fm - A no graphics, no pop-ups email service
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT