From: John K Clark (johnkclark@fastmail.fm)
Date: Fri Jun 06 2008 - 10:03:15 MDT
On Fri, 6 Jun 2008 "Stathis Papaioannou"
<stathisp@gmail.com> said:
> Natural intelligences allow their goals to
> vary because that's the way their brains work.
A keen grasp of the obvious.
> it has no bearing on the goals of an AI.
So axioms cannot predict all mathematical truth but goals can predict
all actions of an intelligence even though that has never happened in
any intelligence ever observed. I don’t think so.
> If the AI is born with the idea that achieving X
> is the most important thing,
When you and I were born our most important goal was suckling at our
mother’s breast. Times change.
> then as part of its strategy it will seek to
> ensure that it can never change its mind about X,
> since that might prevent it from achieving X.
If something can never change its mind regardless of how much new
information it receives then it is not intelligent. Such a thing would
be no threat to humanity and no use to it either; nor would it be any
use to itself, or to anything else.
> If you argue that the AI is so complex that its
> goals will change unpredictably despite being
> initially well-defined then you are arguing that
> the AI will inevitably malfunction and behave
> irrationally and unintelligently.
Nobody knows what your goals will be in 20 years, even you don’t know
that. Does that mean you are irrational unintelligent?
John K Clark
-- John K Clark johnkclark@fastmail.fm -- http://www.fastmail.fm - One of many happy users: http://www.fastmail.fm/docs/quotes.html
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT