sl3 policing of sl0-sl4 risks (was Re: Think of it as AGI suiciding, not boxing)

From: Phillip Huggan (
Date: Thu Feb 23 2006 - 00:54:34 MST

Building AGI to save the world from MNT and building MNT to save the world from AGI is an ironic symmetry. Depending on safeguards already in place, with MNT sensor bots a bioweapon might be the first threat to mitigate. Then chemical weapons and/or their missile delivery systems. Then MNT. Then AGI. Then salted nukes. MNT and AGI would be the ones that require a level of sensor surveillence stringent enough to scare anyone. Hopefully we can find a good way of conducting it.
  There are three main interrelated reasons why it doesn't make much sense now to project who will control diamond von Neumann assemblers and what will happen.
  1. Don't know time frame. The world 20 years from now will be very different than the world 60 years from now will be. Diamond machines in 2065 might be treated like the Atari video game console is today.
  2. Don't know the specific mechanism that will floor the cost of diamonds. MNT is different from CVD is different from recreating the conditions under which natural diamonds form. Each pathway has a different maximum industrial versatility of product functions.
  3. Don't know the specific administrative type of the possessors. National leader(s)? Military? Government department? Corporate Board? Corporate Research division? University Research team? Who knows.

Charles D Hixson <> wrote:
  On Tuesday 21 February 2006 11:16 am, Phillip Huggan wrote:
> It is feasible to police as computation increases towards ubiquity if
> policing implements also increase at the same rate or higher. It isn't
> happening now, but there is a class of technologies on the horizon that do
> encompass the mass production of sensor devices. The technical side of
> things seems feasible (one day, maybe not before AGI is built or something
> else earth-shattering happens), it is the political and administrative
> risks that scare me. That is why I'm only advancing a *let's never allow
> an AGI to get out* vision of the future as a suggestion to chew on.
Odd. The thought of such technologies being under a central agency that
WASN'T friendly is one of the reasons I feel an AGI is *necessary*!!! Of
course, it also slightly raises the stakes on getting it correct in the first
place, but it's a bit hard to raise practical infinity in any meaningful way.

Relax. Yahoo! Mail virus scanning helps detect nasty viruses!

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT