From: John K Clark (johnkclark@fastmail.fm)
Date: Wed Jun 04 2008 - 11:26:24 MDT
On Wed, 4 Jun 2008 "Stathis Papaioannou"
<stathisp@gmail.com> said:
> The motivations of an AI could be completely alien or
> at odds with our own, yet completely understandable.
If it’s completely alien I’ll be damned if I can see how it’s completely
understandable. Alien means strange and strange isn’t understandable,
that’s what makes it strange.
> There is also no necessary contradiction in the idea
> of superintelligent AI which remains predictable to
> us, since its goals may make it predictable.
Goals be damned. The axioms of mathematics do not make mathematics
predictable and axioms are one hell of a lot more fundamental and
inviolate than goals which change at the drop of a hat. How many
sincerely made new years resolutions are broken by January 2? Most.
> For example, if the AI is born with the aim of
> adding two given numbers together and then turning
> itself off, it will do just that, quickly and efficiently.
That does not require intelligence, much less super intelligence, but
let me suggest something only slightly more complicated. Suppose the
machine was born with the aim of finding the smallest even number
greater than 2 that was not the sum of two primes greater than 2, and
then turn itself off. What will the machine do? Will it turn itself off?
Nobody knows, all you can do is watch it and see. The machine has free
will, at least it does if the term has any meaning at all.
John K Clark
-- John K Clark johnkclark@fastmail.fm -- http://www.fastmail.fm - The way an email service should be
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT