Re: How hard a Singularity?

From: Samantha Atkins (
Date: Thu Jun 27 2002 - 02:32:20 MDT

Stephen Reed wrote:
> On Wed, 26 Jun 2002, Ben Goertzel wrote:
>>I don't doubt that there are many trustworthy, moral individuals within the
>>US military. However, I do not trust the US military as an organization,
>>no way Jose'. This is the crew that invaded Grenada... the organization
>>that nuked Japan (probably both bombs were needless, but it's pretty damn
>>clear the second one was), that systematically tortured Vietnamese women and
>>children in the name of democracy and justice.... I'll spare you a full
>>list of examples. Ever read the first-person account of how US soldiers
>>chopped off the arms and legs of a Vietnamese woman, inserted dynamite in
>>her vagina and blew her up? Not an outlier occurence. Excuse me if the
>>impeccable morality of the US military seems a little questionable to me...
> You strengthen my point. The US military has evolved ethics because of the
> terrible power that they are entrusted to use.

Please show that there is a practical system of ethics in the
field after the sorry history of post WWII US military

> Why else have the strict
> honor system at military schools? Furthermore, knowledge that individual

Irrelevant as to whether the military itself acts ethically.

> soldiers are capable of war crimes only encourages future military
> planners to substitute an AGI for a soldier (or augment/monitor a
> soldier) in that situation. According to my understanding of CFAI, the

Actually, the US is insisting that its soldiers be immune from
any international war crime proceedings instead. :-(

> AGI should be resistant to criminal behavior due to a deep knowledge of
> ethics. From which follows my conclusion that an AGI would not follow an
> illegal military order.

If it is really ethical it will not follow any offensive
military orders.

>>The Singularity is directly opposed to the national interests of any
>>particular national government, because it will almost inevitably lead to a
>>"revolution" that will make national boundaries meaningless.
> Well for me the Singularity is something that I cannot see beyond, so I
> will respond to your statement as though I agree with the premise and
> offer this counterexample of the second statement in your argument:
> Recent history shows that national governments will surrender sovereignty
> for a greater good. Witness the EU and NAFTA.

NAFTA has not been for the greater good.

>>Thus, no government should be trusted to play a leading role in the
> Not only do I have the opposite opinion, I believe that as the evidence
> mounts that an AGI is possible, the government - and in particular the US
> government will take the leading role. I believe this from my
> understanding of the dynamics of how our government institutions have
> responded to the great technical challenges over my lifetime - The Space
> Race, the War On Cancer, Safe Cars, Solar Energy, The AIDS Epidemic ...
> I do not see any non-governmental organization as the front-runner. The
> actual work of developing and educating the Seed AI would in my opinion be
> performed by private contracting companies - somewhat in competition with
> each other.

The Space Race that we used as a political football and then let
fall completely flat; the War on Cancer that was largely a lie
(see The Big Lie); Safe Cars (?) with 50,000 Americans dying on
the highway each year; Aids - see Bush's latest non-action
proposals. Clearly there is no clear pattern of government
doing great things in real problems/opportunities. Quite the
opposite is the case. And don't let us forget the US has more
of its population in jail than any other country, most for
victimless "crimes". Don't forget we are claiming an endless
War without clear objectives against any country that we feel is
not sufficiently for our obviously Good and Righteous War. No, I
don't think the US government has any clear demonstrated ability
to do anything except spend us into oblivion and wreck a lot of
other coutnries for the sake of monied interests.

Oh goodie, a major military spending boondoggle to develop Seed
AI and teach it to follow orders! The only virtue with such a
proposal is if you don't want to see AGI really developed within
the next 100 years.

>>I might accept gov't funding for AGI research, but I would never willingly
>>place control of an AGI in the hands of any military organization. That
>>really scares me. Those people are not Singularity-savvy, and I don't trust
>>they will become so in the future, not in any healthy way. Plus, their
>>interests are not those of the human race as a whole, let alone of sentience
>>as a whole.
> My belief that the funding will be from the defense budget stems from my
> conclusion that an AGI would be priceless for our nation's defense, and
> knowing this the military will fund before other government institutions.

It does not belong to "our nation" or any nation.

> Overall I believe that our constitutional, representative government has
> the best chance of marshaling citizen support and addressing the reasoned
> dissent sure to come in the face of mounting evidence that an AGI is
> possible. Regardless of popular opinion towards the government, I
> speculate that all other alternative coordinating organizations would be
> judged inferior by the public.

It is a pity we in the US no longer in reality practice
constitutional, representative government. By the time the
"public" gets around to "judging" it will be far too late.

- samantha

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT