From: Thomas Buckner (tcbevolver@yahoo.com)
Date: Thu Jul 14 2005 - 17:25:55 MDT
--- "J. Andrew Rogers"
<andrew@ceruleansystems.com> wrote:
> Russell Wallace wrote:
> > Yes, wisdom requires intelligence, but
> intelligence does not
> > necessarily require wisdom.
>
>
> Define "wisdom". How does it differ from
> intelligence?
Intelligence can be defined as 'the ability to
think up optimal ways to achieve goals.' Wisdom
can be defined as 'the ability to understand
which goals are optimal.'
Put this way, a paperclip optimizer can be very
intelligent about how it efficiently turns
everything inside the Kuiper Belt into
paperclips. A wise AI can be just as efficient in
turning the solar system into paperclips, but
before it embarks on that act it will think "What
supergoal am I trying to achieve by turning the
solar system into paperclips? Is there a better
way to achieve that supergoal? Is that supergoal
itself optimal?" A SAI that is not wise is worse
than useless; one which is wise will figure out
what we need better than we can; if CV is
workable ve will work it out; if some other
strategy is optimal it will work that out.
This reminds me of one of the techniques of NLP
(neurolinguistic programming: the most famous NLP
maven is Tony Robbins, but he didn't invent it
and there are plenty of others in the field). NLP
in a nutshell: study high achievers, dissect
their thought patterns, learn to use the same
thought patterns.
The technique is to find out the higher value.
'Values' in NLP mean much the same thing as
supergoals. Here's an article that explains the
idea well:
http://www.nlp-now.co.uk/nlp_and_values1.htm
Quote:
What is a 'Value'?
A value is a 'hot button' that drives a
behaviour. Whatever you do is done in order to
fulfil a value - even though you are unlikely to
be consciously aware of that value.
You swim to fulfil the value of improving your
health, benefiting from the relaxed state it
later produces, enjoying the warmth of the sun
and the sea, etc.
You drink alcohol to fulfil the value of feeling
less inhibited, to enjoy the social scene
involved, to enjoy the taste of the drink, etc.
You buy fashionable new clothes to fulfil the
value of looking good, looking right for work, or
not looking dowdy, etc.
Pleasure and Pain
Everything you do is a means to an end and this
end is the fulfilment of a value. You do what you
do to either move towards pleasurable feelings or
values. Or to move away from or avoid painful
values or feelings. (In NLP the Pain values are
called Away From Values and the Pleasure values
are called Towards Values)."
End Quote
The article also states that many people are
unhappy because they don't really know what their
true values are, never really gave it much
thought, so they pursue things that aren't what
they really want! This is equivalent to an AI
researcher who hasn't done hir homework on the
supergoals. That's how you get paperclips.
One thing I do not accept is that any real SAI
can possibly run on a 'diesel-sipping Pentium',
as has been suggested. Whatever the physical
substrate is, it will have to be closer to the
most powerful computers now extant than anything
much less powerful, even with maximally optimized
code. I say this because I can see a
diesel-sipping Pentium as a paperclip optimizer,
but not as a wise SAI. I agree with the
suggestion that Friendliness correlates with
intelligence far above human level, at least to
the extent that such SAI has the capacity to
examine its own thinking in ways that a weaker AI
can't. The AI has to have the capacity to sort
out its own supergoals in a timely manner, unless
we constrain it to obeying our preselected
supergoals (sure hope we chose them well!)
I have to say, however, that even extreme
Friendliness may have its limits. Imagine
yourself the proprietor of an ant farm. Suppose
that you have named each of the hundreds of ants,
know them on sight, love them like they were your
own children. You weep bitterly whenever one of
them comes to harm. Even so, can you protect them
in all conceivable conditions? What if a tornado
hits your house? Maybe we expect things of FAI
that ve could not give us, no matter how
motivated ve is.
Tom Buckner
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT