Re: How to make a slave (was: Building a friendly AI)

From: Emlyn (
Date: Mon Dec 03 2007 - 22:57:20 MST

On 20/11/2007, Robert Picone <> wrote:
> On Nov 19, 2007 3:00 PM, Thomas McCabe <> wrote:
> > (sigh) The point of FAI theory isn't to figure out what the AGI
> > *should* do. It's to get the AGI to do anything at all besides random
> > destruction, and to do it predictably under recursive
> > self-improvement. If we can program an AGI to reliably enhance the
> > jumping abilities of lizards, and to continue following this goal even
> > when given superintelligence, the most difficult part of the problem
> > has already been solved.
> >
> > - Tom
> >
> And the idea of a superintelligence that is unable to change its own goals
> when appropriate isn't at all disquieting to you?...
> To use your example, wouldn't the resulting species of genetically modified,
> technology-using monitors get a bit annoying when they start jumping over
> fences and through windows to get to whatever tasty prey they can get?
> Do you believe that there is any goal or set of goals out there that when
> obsessed over exclusively by a smart enough being never has the potential to
> do harm?

Several hours after the intelligence augmented lizards wipe out the
last of humanity, they use their newfound god given abilities to jump
into hyperspace...


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:01 MDT