From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Fri Jun 21 2002 - 02:22:45 MDT
Samantha Atkins wrote:
>
> I see him building awareness and momentum now and stepping back, for now,
> from taking any technological lead. Perhaps he believes more can be
> accomplished by him at this point in that capacity and/or that technology
> is proceeding apace in any case. I am a little worried to see you or
> anyone casting moral aspersions on another if there actions are not what
> you would have done in their place. I don't think that is very fruitful.
>
Samantha, what I'm worried about is that Kurzweil seems to be spreading a
mutant nonactivist form of the meme. Vernor Vinge's original stories of the
Singularity were based around the blazing disruptive power of
smarter-than-human intelligence, not the inexorable grinding of the wheels
of industry churning out transhumanity as a safe, ordinary luxury.
Therefore Vinge's stories often show, realistically, individuals or projects
that play a critical role in the final Singularity (regardless of how much
background technological progress contributes to it). Vernor Vinge's 1993
NASA lecture emphasizes both the speed of the changes once created and the
size of the stakes - and while we're at it, also emphasizes the need to
enter the Singularity with our best foot forward, the possible differences
between technologies, and the fact that the intrinsic rate of some
technologies may be faster than others. At this moment Kurzweil's version
of the Singularity meme doesn't include *any* of Vinge's strategic, activist
thinking, except in the very diluted form of deciding which inevitability to
predict.
Vinge and I have very different approaches toward how our lives can matter
in humanity's sweep toward the Singularity, but structurally our thinking is
similar enough that I have no worries about the memes he spreads. If Vinge
should make what I regard as a strategic error, I don't expect it will be
any problem to point it out from within Vinge's framework.
Kurzweil gave a presentation at Extro 5. During the Q&A session, I got up
and said (if I recall correctly, that is; the video is on Kurzweil's site):
"Well, you've told us to expect a Singularity, but you haven't told us
whether it's good or bad, or what we should be doing about it. What are our
marching orders?" Kurzweil stood stock still for around 7 seconds until
some still-unforgiven voice from the audience gave him an out by asking
"Well, what do you mean by the Singularity?", and Kurzweil took it. Ben
Goertzel, after Kurzweil's presentation, said: "I don't think we should get
so caught up in predicting that nanotechnological immune systems will exist
that we forget to actually develop the immune systems," and got a round of
applause from the audience - because there was no hint of that in Kurzweil's
presentation. At the Foresight Gathering, I asked "Have you considered
whether we could accelerate the Singularity?" and Kurzweil said, in an
incredulous tone, "How could anyone possibly accelerate the Singularity?"
I don't read this as Kurzweil having considered the possibility of activism
and rejected it because he can't think of anything good. I read it as
"activism" structurally conflicting with Kurzweil's thinking about the
Singularity.
This causes me to worry about Kurzweil being the Singularity spokesperson in
a way that I would not worry about for Vinge. I am sure that Kurzweil has
done enormously more in the way of concrete technological development than
Vinge. But Vinge has an activist outlook on the Singularity. Kurzweil is
speaking from within a view which does not structurally permit activism. I
am worried about this view being transmitted to Kurzweil's audience.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT