From: Peter C. McCluskey (pcm@rahul.net)
Date: Tue May 25 2004 - 17:03:31 MDT
ag24@gen.cam.ac.uk (Aubrey de Grey) writes:
>I don't really see what you mean. Surely the only people who need to
>be educated/enticed in this way are those with the capacity to have a
>go at building full-blown AI? Or to build it by accident, I guess --
>but even then I can't see those people being hard to educate on this.
I see them as being hard to identify. That means we need to broadcast
warnings in order to educate people, and broadcast channels tend to have
a surplus of memes competing for attention.
I also expect that many possible AI developers would be focussing their
attention on the goal of intelligence to the point of ignoring most
arguments about how to save the world.
And I expect that some of the people who understand the risks will be
eager enough to show off their skills that they will fool themselves
into thinking that the software they are creating won't become a full-blown
AI.
For example, look at the attempt in Engines of Creation to predict the
development of a global hypertext system. There was one obvious project
(Xanadu) that had a bunch of smart people who cared about the social
effects of their ideas, and were probably devoting more effort than any
other project towards perfecting the design. Then Tim Berners-Lee got the
urge to write something quick to solve an interesting subset of what
Xanadu was attempting. And ignored many concerns of Xanadu such as how
to guarantee scalability, how to prevent broken links, and how to do
create backlinks so people could find responses to bad ideas.
It's hard to see how someone trying to influence the design of global
hypertext systems could have discovered Berners-Lee while he was designing
the web, in part because that took only a tiny fraction of the time over
which people were worried about how to do it.
Unfortunately, this is also evidence relevant to whether we should expect
Eliezer to create the first AI (although Eliezer seems quicker to admit his
mistakes than the Xanadu team was, so the analogy isn't perfect). Which is
a reason not to give up on trying to educate some of the smarter people
who work on an independent AI.
--
------------------------------------------------------------------------------
Peter McCluskey | I see no greater impediment to scientific progress
www.bayesianinvestor.com | than the prevailing practice of focusing all of
| our mathematical resources on probabilistic and
| statistical inferences while leaving causal con-
| siderations to the mercy of intuition - J. Pearl
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT