AI testing and containment Re: [SL4] Programmed morality

From: Brian Atkins (
Date: Fri Jul 07 2000 - 10:33:56 MDT

petervoss1 wrote:
> > Probably the best solution is to let them learn the very real value of
> cooperation naturally. Put a group of them in a virtual environment with
> limited resources and they will have to learn how to share (and to lie &
> cheat). These are valuable lessons we all learn as children and should form
> part of a growing AI's education also. Only when they've demonstrated
> maturity do we give them any real freedom or power. They should then share
> many of our own values and want to build their own successors with similar
> (or superior) diligence. Anything less in unacceptably reckless.
> I agree that 'anything less' is risky, but I don't see that we have a choice
> (other than not doing AI):
> 1) General machine intelligence will invariably be connected to the Web
> during development & learning.
> 2) It seems that the only effective way to get true AI going is the seed
> route. In this scenario, there may not be a community of (roughly equal)
> AIs. Only one will bootstrap to superior intelligence.
> I am very concerned about the risks of run-away AI (unlike Eli I *do* care
> what happens to me). I'm desperately searching for ways of trying to predict
> (with whatever limited certainty) what goal system an AI might choose. Any
> ideas?

I would like to say two things, first off I don't agree with #1. Storage
will be so cheap by the time we are ready to try running the first seed
that we can store a huge chunk of the WWW (if not all of it- could this
be a new business opportunity in the future, selling copies of the WWW?)
locally and run the seed completely offline.

Secondly I agree with #2... I prefer some way of testing out slightly
different seeds. However to really test what an AI will do once it is
"loose" you would have to provide it with a quite awesome simulation
of the real world (Matrix-like) and then see what it does to the humans.
I don't think we will be able to do that even if we had the hardware.
So I would be interested to know of other possible ways to test what
the AI would do.

Remember four years of good friends, bad clothes, explosive chemistry

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:35 MDT