From: Durant Schoon (durant@ilm.com)
Date: Wed Dec 13 2000 - 13:31:49 MST
> From: "Ben Goertzel" <ben@intelligenesis.net>
> To: <sl4@sysopmind.com>
> Subject: RE: When Subgoals Attack
> Date: Wed, 13 Dec 2000 15:03:16 -0500
>
> Of course, what you describe happens all the time in the human mind.
>
> Your goal is to have fun. So you start playing a video game. You get
> hooked, but are doing badly,
> and you get pissed off. The subgoal of playing the game has overthrown the
> supergoal of having
> fun. Having spawned the subgoal, you then become
> thoroughly involved in it.... Now a transhuman AI should be able to avoid a
> lot of this: being more flexible
> it can just execute an internal program telling it "if this ever stops being
> fun, you should probably
> stop doing it, unless some goal higher up than 'having fun' says otherwise."
So let's pretend you're a transhuman by adding a simple, although bizarre,
twist to your example:
Imagine that the video game is hooked up to your brain and can influence
your thoughts (hey that's not so bizarre, SL3?). All of a sudden it becomes
fun to mess with your fun seeking center....to the point where seeking fun
is suddenly not worth pursuing...
So you'd better not play that video game, and know "not to" before ever
playing it.
So you'd better be able to detect which video games are potentially harmful,
a priori.
So you have to simuluate playing every game you might every play in a safe
"sandbox" environment,
So...
> I don't think a transhuman AI will be able to completely overcome the
> problem of "subgoal alienation", which is
> endemic in human psychology, but I suppose that the more memory & processing
> power you have, the more you can
> avoid the problem...
Which leads me to the conclusion that when we get an AI, ve is going to
want more memory & processing power...always. But will ve try to convince
us we should give more to ver? ... the subgoal of the nascent AI vs. the
supergoals of humans ...
Or suppose your working and reading and writing posts to a hypothetical
mailing list (arbitrarily call is "sl4" for "some list for") is really,
really fun! and the "work" supergoal is overthrown and...hey wait a
second...
--
Durant "Computer games don't affect kids... I mean if Pac Man affected us
x2789 as kids, we'd all run around in a darkened room munching pills and
listening to repetitive music." - anonymous (< o o o o o
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT