From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Feb 06 2000 - 21:56:38 MST
From: "Eliezer S. Yudkowsky" <sentience@pobox.com>
DaleJohnstone@email.com wrote:
>
> A few years ago I came up with a design for an AI that could possibly end up triggering a singularity-type affair mentioned on Eliezers' web site.
> It's been a frustrating time trying to find the time to build it and still make a living too.
Join the club.
> I also have some serious ethical concerns regarding a possible military seizure of the project. I think a 'grey goo'-like situation is possible with AI, call it a 'chrome army' scenario if you like. Truely smart weapons would be truely terrifying, and probably unstoppable (with the exception of having a nanotech defence).
My book on the situation has been that AI that's dumb enough to be a
willing tool will, indeed, have military applications. Virtually
everything does. I don't think those military applications are so
extreme as to grant an unbeatable advantage in war, especially if both
sides have the AI. And there's no intrinsic imbalance between offense
and defense. In information warfare, for example, an ounce of defensive
AI will probably turn out to be worth a pound of offensive AI.
Genuine AI might seriously exacerbate the nanotechnology problem,
however. As it is, we're likely to see giant, unwieldy vats
synthesizing diamondoid fighter jets years before we have to deal with
actual goo. AI could seriously compress that time period.
I don't really see a feasible alternative, however.
> So as you can imagine I'm a little concerned at the idea of open source AI. This possible scenario is not mentioned anywhere on the website. As the #1 rule is 'Don't fry the planet' I think it deserves some attention.
I'll join in a sigh to the general sentiment. Ultraproductivity kicking
hell out of the economy, military applications of everything...
accelerating the future is like tap-dancing through a minefield. Around
the most you can do is hope to accelerate the defensive, social, and
cushioning aspects of a technology ahead of the offensive,
destabilizing, and sudden-shock aspects. A tradition going back to
Drexler and hypertext.
--
sentience@pobox.com Eliezer S. Yudkowsky
http://pobox.com/~sentience/beyond.html
Typing in Dvorak Programming with Patterns Writing in Gender-neutral
Voting for Libertarians Heading for Singularity There Is A Better Way
--------------------------- ONElist Sponsor ----------------------------
Valentine's Day Shopping Made Simple.
<a href=" http://clickme.onelist.com/ad/SparksValentine7 ">Click Here</a>
------------------------------------------------------------------------
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:06 MDT