RE: Diaspora, the future of AI & value systems, etc.

From: Ben Goertzel (
Date: Mon Dec 11 2000 - 14:26:48 MST

It seems that this conversation is doomed to get strangled by wording

No, I didn't mean to literally say that conscience is an "inner voice."
Having in fact "heard
voices" on a few occasions, I know the difference between that and
conscience. It's just a
very common metaphorical expression. Similarly, in case you were wondering,
 when people refer to the "the voice of the people," it's not referring to
some really bizarre collective
acoustic wave formation, it's just another metaphor.

In order to avoid this kind of confusion, we could agree to communicate only
in formal mathematics,
but that does get kind of awkward after a while (and most mathematics isn't
as formal as people like
to think; see for some truly formalized, unambiguous

Anyway, I don't feel like spending time arguing that conscience is a
widespread psychological
phenomenon among human beings.... This is sufficiently amply documented
that it's silly to argue about it.

However, I have to take issue with the obvious silliness of the statement
that conscience may be just another
way of reasoning about possible consequences.

Don't you see that reasoning, like all mathematics, is just a way of
transforming statements into
equivalent statements (with a greater or lesser degree of certitude, in the
case of probabilistic
reasoning).... Reasoning doesn't give a system goals and motivations, it's
a method for figuring out
how to fulfill its goals and feelings, etc. It can split goals into
subgoals, or merge them into
supergoals, but it must have something to begin with.

Physical survival of the individual is the source of some of our human
goals. Conscience is another --
we have the goal not to do what is "culturally perceived as bad." Maybe you
don't have this goal, or
maybe you do and just don't want to admit it ... but anyway, my point was
about humans in general, not
about super-evolved ubermenschen such as populate this list ;>


> -----Original Message-----
> From: []On Behalf
> Of Gordon Worley
> Sent: Monday, December 11, 2000 4:09 PM
> To:
> Subject: RE: Diaspora, the future of AI & value systems, etc.
> At 2:42 PM -0500 12/11/2000, Ben Goertzel wrote:
> >Is it only because they're afraid of what will happen when
> they're caught?
> Yes.
> >No, we have consciences, which are the "internal voices of society." The
> >inner urge or will to
> >kill the other person is counteracted by another inner
> voice/feeling telling
> >you that it's
> >bad to kill someone.
> If I ever start hearing voices, I'm on my way to a doctor. I think
> things through, consider the trade offs, and then make a decision. I
> have never felt the urge to kill anyone, and doubt that I would
> unless there were a good reason to do it. Even then I would only act
> on it after thinking about it. It is not a matter of getting caught,
> but all consequences. Maybe voices are just another way of looking
> at the reasoning process, whereby one considers the situation from
> different angles before arriving at a decision.
> >The conscience is often viewed as an "internal voice of one's
> parents." An
> >AI won't have any parents
> >in the strict sense.
> I'm not buying this conscience thing. It is true that parents
> propagate memes to their children, and that children then use those
> memes when making decisions. I don't, though, hear the voices of my
> parents in my head. Just because an AI won't be born of anyone,
> doesn't mean that ve won't adopt memes of its creators. The only
> difference between a natural child and an AI is that the AI doesn't
> have parents, but other means of getting memes.
> >We are evolutionarily programmed to pass along values and morals
> or whatever
> >you like to call them,
> >to our children. I.e., our genes cause us to propagate certain memes.
> I would rather think that it is a meme that makes us propagate memes,
> since otherwise we might try to propagate memes, even if we have none.
> --
> Gordon Worley
> PGP: C462 FA84 B811 3501 9010 20D2 6EF3 77F7 BBD3 B003

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT