From: Samantha Atkins (samantha@objectent.com)
Date: Mon Jan 29 2001 - 03:14:15 MST
"Eliezer S. Yudkowsky" wrote:
>
> Ben Goertzel wrote:
> >
> > > "One of" the goals? Why does an AI need anything else? Friendliness
> > > isn't just a goal that's tacked on as an afterthought; Friendliness is
> > > *the* supergoal - or rather, all the probabilistic supergoals are
> > > Friendship material - and everything else can be justified as a subgoal of
> > > Friendliness.
> >
> > Creation of new knowledge, and discovery of new patterns in the world, are
> > goals that I believe are innate to humans in addition to our survival and
> > reproduction oriented goals.
>
> Say *what*? Creation of new knowledge and discovery of new patterns are
> *known* to be innate to humans - no need to "believe" it. Why? Because
> they promote survival and reproduction. There's no brain-represented
> connection from discovery to reproduction, AFAIK, but it's certainly a
> historical fact about the origin of that joy.
So what? Are you concluding that creation of new knowledge and
discovery of patterns is only useful or important or worthwhile if it
promotes survival and reproduction, e.g. that it is only useful to
humans? This does not follow at all.
>
> > Should we not supply AI's with them too?
> > Webmind is being supplied with these goals, because they give it an intrinsic
> > incentive to grow smarter...
>
> A transhuman general intelligence would reliably deduce such goals as a
> declarative subgoal of Friendliness, selfishness, or pretty much anything
> else. A prehuman AI can just be given the flat fact that curiosity is a
> subgoal of Friendliness, or the flat knowledge that curiosity leads to
> greater effectiveness at Friendliness - the two should be pretty much
> equivalent if you did the goal system right - with the details of the
> knowledge being learned over time.
>
And the refinement of its supergoal of friendliness will lead to its
questioning of what the meaning of Friendliness really is and why it is
important so that, understanding its true importance, it may serve the
goal more accurately. This leads to questioning the supergoal itself.
- samantha
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT