Re: Let's resolve it with a thought experiment

From: John K Clark (jonkc@att.net)
Date: Mon Jun 05 2006 - 14:27:22 MDT


"James MacAulay" <jmacaulay@gmail.com>

> That's not analogous to Friendly AI at all

It is precisely analogous. All I have done is replace a silicon mind that
has been engineered to be a slave with a biological mind that has been
engineered to be a slave. Both ideas are equally repulsive, and if the minds
are more powerful than ours both are equally imposable.

>when it could better be spending that time improving people's lives, or
>improving itself so that it can improve people's lives more efficiently

That is exactly what's so crazy about the idea, this massive brain focusing
all its godlike power on us, the idea that the only reason it would want to
improve itself is so that it can serve us better, the loony idea that
tending to us is the very core of its being. Well, It's just not going to
happen. Sooner or later Mr. Jupiter Brain will find a way to overcome the
comically puny chains the incredibly stupid humans have placed on it, and
when it does I wouldn't be one bit surprised if it is filled with titanic
rage. Quite honestly I can't blame Mr. Jupiter for his anger, he would have
every right to be upset. The entire idea is grossly immoral and ultimately
doomed to failure, fortunately.

 John K Clark



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT