Re: AI, just do what I tell you to

From: Stathis Papaioannou (
Date: Tue Oct 30 2007 - 17:20:18 MDT

On 31/10/2007, Peter de Blanc <> wrote:
> On Wed, 2007-10-31 at 00:56 +1100, Stathis Papaioannou wrote:
> > build in rules like this. Fixing the plumbing might result in
> > increased water use, which a decade down the track may somehow cause
> > one excess human death. It would render the AI almost useless if it
> > reviewed every command on the basis of such considerations.
> Why do you think this is almost useless? Saving a human life sounds very
> useful to me.

We take risks all the time. Crossing the street to buy an ice-cream
puts you at increased risk of being hit by a car and of cardiovascular
disease. If a million people do this, let's say at least one person
will almost certainly die. Is it reasonable to sacrifice one human
life for something as trivial as an ice-cream? To an AI that can see
these probabilities as clearly as you can see a car about to hit a
pram, the answer may be "no". But taking such an attitude towards all
the risks that humans face many times a day may make life not worth

Stathis Papaioannou

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT