RE: One or more AIs??

From: Mitch Howe (
Date: Sun May 30 2004 - 09:53:37 MDT

Philip Sutton and Mark Waser wrote:

(that multiple AGI is better than one)
You both believe that checks and balances are important.  You both seem to
understand that an AGI will be immensely powerful.
And you are both correct.
However, by suggesting that creating multiple, different new minds would
make us safer you are fighting the correct battle in the wrong country.
*Any* successfully launched AGI launched in isolation will have the keys to
the Solar System.  Being vastly more intelligent than any other entity
around does that to you.  But you speak as if *different* AGIs could be sent
ahead together, like three explorers in a dingy rowing out from the mother
ship to a new shore -- all with something to gain from cooperation, and all
free to throttle the others at will if either threatens to horde El Dorado
for himself.  
Different AGIs would launch into new domains of intelligence at different
speeds and along different trajectories.  At any given moment, one would
almost certainly be far more powerful than any other.  Instead of three
explorers in a dingy, you would have Tiny Tim in an inner-tube, Paul Bunyan
on a hovercraft, and a Borg Hive on a Death Star.  No heated discussions.
No family council.  Not even a clash of the titans. No contest at all.
So yes, there is a very important time for checks and balances, and that
time is before any AGI program goes live.  Yes, programmers are flawed.
Success must be ensured in spite of this.  This is what Friendliness theory
is all about, and one of the reasons it is so damned tricky. And, not
coincidentally, it is the reason why anyone taking it seriously right now
doesn't have any code to show for it.  Solution first, implementation later.
--Mitch Howe 

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT