Re: SIAI's flawed friendliness analysis

From: Michael Roy Ames (
Date: Fri May 09 2003 - 18:03:07 MDT

Dear Bill,

I have a number of comments about your critique. In this post I would
like to quickly focus on one area only. (Possibly later I will send

IRT: "History vividly teaches the danger of trusting the good intentions
of individuals."
This statement highlights only parts of human history.  The parts where
an individual had good intentions, but a bad idea.  Or good intentions,
a good idea, but bad timing.  Or good intentions, a good idea, but
inappropriate application to real-world situations.
There is another side of human history to consider.  The side where an
individual had good intentions, acted on them, and produced a good
result.  Or good intensions, good timing, and appropriate application.
Or even bad intentions that turned out for the best anyway.
I would argue that, as a first approximation, the 'up' side of
decision-making in human history happens as often as the 'down' side.
The evidence for your statement is too thin support the argument.
My counter-argument is as follows...
History's vivid lessons show us that it is not good intentions that we
should trust, but good ideas.  Ideas that are rational, timely and
appropriately applied to the real world have been immensely beneficial
in the past, and will continue to be so in the future.   Upon reading
the works you are critiquing, I was struck by how little mention there
was of anything like 'good intentions'.  Rather, the emphasis was on
rational thinking with the purpose of avoiding death and destruction.
The emphasis was on ideas and how to apply them to the real world.  Good
intentions?  Trivially, yes.  But the main thrust of FAI is solid ideas
that could be practically applied to solve a real-world problem, with
great leverage... it is difficult to find fault with that.
Yours sincerely,
Michael Roy Ames
---- Original Message ----
From: "Bill Hibbard" <>
To: <>; <>
Sent: Friday, May 09, 2003 6:51 PM
Subject: SIAI's flawed friendliness analysis
>        Critique of the SIAI Guidelines on Friendly AI
>                   Bill Hibbard 9 May 2003
[major snippage]

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT