Re: AI Goals [WAS Re: The Singularity vs. the Wall]

From: Ricardo Barreira (
Date: Tue Apr 25 2006 - 09:39:08 MDT

What about uncounscious (fulfilled automatically) goals which you can
articulate if you know about them. For example, breathing. You also
need a category for that. But I bet that there are already tons of
detailed studies about this.

On 4/25/06, Richard Loosemore <> wrote:
> Ben Goertzel wrote:
> >> I think that the question of an AI's "goals" is the most important issue
> >> lurking beneath many of the discussions that take place on this list.
> >>
> >> The problem is, most people plunge into this question without stopping
> >> to consider what it is they are actually talking about.
> >
> > Richard, this is a good point.
> >
> > "Goal", like "free will" or "consciousness" or "memory", is
> >
> > 1)
> > a crude abstraction that we use to describe certain aspects of a
> > complex cognitive system (the human mind/brain), but that does not
> > really fully describe these aspects correctly
> >
> > 2)
> > An abstraction that some complex cognitive systems use as an
> > **ideal**, i.e. they try to modify themselves so as to be more
> > explicitly goal-oriented, never fully succeeding (at least in the
> > human case)
> >
> > When I say that an AGI has certain goals, I don't necessarily mean
> > that the AGI is orienting all its actions toward these goals, just
> > that
> >
> > * the statement that the AGI is "pursuing" these goals is a fair
> > approximation of its behavior
> >
> > * the AGI is explicitly involved in an ongoing process of making
> > itself more goal-oriented so that it can better achieve these goals
> > which it explicitly conceptualizes
> So let's begin to construct a better definition of what's happening here.
> What about the following distinctions:
> 1) There are explicit, self-conscious "goals" that the system sets for
> itself and that it can talk about easily when you ask it "What are you
> doing this morning?" [Answer: "Well, my goal for this morning is to
> finish writing a paper."]
> 2) There is the actual thing it is doing, which we might call a "task
> goal" that is currently occupying the system's attention, which comes up
> if we ask "What are you actually doing right now?" [Answer: "Okay, so
> I got distracted and so what I am actually doing right now is writing a
> message to reply to one of Ben's posts."]
> 3) The above two points highlight the fact that the system can have many
> of these goals hanging around, functioning on different timescales. I
> have a work list that has a couple of hundred house-maintenance, house
> renovation and other tasks on it, but it impinges on my behavior in
> various ways. Even if these are goals that operate on a timescale of
> years, what makes them similar to other ones is that I can explicitly
> articulate them.
> 4) Might there also be other goals that I cannot articulate because they
> work underneath the surface? That I cannot articulate (even if my
> therapist might be able to)? This is difficult: let's make a category
> for this type of goal, even though I think in the end there may not be
> any that belong in this category rather than in the next one..........
> 5) Motivations. I would distinguish between these and goals. Right now
> I am swayed in my behavior by some nebulous "dispositions" that I want
> to satisfy. One is gustatory pleasure (I come to a coffee shop to work,
> where I can get a pot of tea to sip while I am writing). One is human
> company (the coffee shop has people in who I don't often interact with,
> but who have something I need, which is perhaps the sound of voices and
> activity). One is the need to achieve something intellectual, or to
> make a mark upon the world, or to be noticed by peers .... hence I am
> driven to write a paper rather than just go dig the garden.
> I could list many other background motivations or drivers that I think
> contribute to my decision to be here doing this, rather than other
> things. I think these are very different from explicit goals because
> they are (a) hard to articulate, and (b) primal, or out of my control.
> I do not decide to want gustatory pleasure, I just want it. The fact
> that they are hard to articulate means, in my theory of mind, that they
> are not directly a part of the main area where all the explicit thinking
> gets done. They impinge on this area, but are distinct from it.
> 6) Motivations are not the end of the process, because there is a system
> that organizes the motivations. I don't just let gustatory pleasure
> drive me all day long, I wax and wane between the various motivations.
> There are saturation and novelty seeking mechanisms at work here: too
> much of one motivation starts to make me want to do something different.
> The interaction between the motivational system and the place where all
> the contents-of-consciousness stuff goes on (the "foreground," in my
> terminology) is of greatest interest to me. It is not simply a matter
> of some motivation marching and putting an explicit goal on top of the
> goal stack: the motivational system is dumb, so it cannot know exactly
> how to do this. There is a coupling between the two, but it is quite loose.
> How, for example, does a nine-month old child have an episode of the
> "rebellious independence" motivation when it does not yet have the
> concepts to put a "rebel!" goal on its goal stack and then try to work
> out a plan of action that satisfies that goal? I watched this happen in
> my son (and then watched and saw the character streak develope from that
> point on, so it was real, that first time), and I find it fascinating
> that the motivation was there before the concepts that he could have
> used to discuss it. This is a very abstract example, I know, but it
> cuts right to the heart of the problem.
> Sorry for the ramble. I'll turn it into a paper soon. I especially
> want to disect your own comments about how you see the significance of
> the term "goals", which seems to be sort of operationalist or behaviorist.
> [Sigh: another paper for the goal stack]
> Richard Loosemore.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT