Re: Ethical experimentation on AIs

From: Stathis Papaioannou (
Date: Sun Oct 21 2007 - 05:24:32 MDT

On 21/10/2007, Lee Corbin <> wrote:

> > It would pay for a fly to suffer terrible, unremitting pain until the
> > noxious stimulus was removed.
> Instead, I would say that *we* are capable of experiencing terrible
> unremitting pain precisely because we have so many higher-level
> means of overriding it. Often, as humans, we really do have to
> experience a little or a lot of physical discomfort to achieve a more
> subtle purpose. That's why the pain is terrible---it's evolution's way of
> saying "Are you really sure about this?", if you were to try to override it,
> or to attempt to deter you from making plans that would damage your
> body.
> But a fly has no abilities to override its basic instincts. Hence, there
> is no purpose to be served by the its experiencing severe pain, and
> so nature does not provide it the capabililty.

There must be a gradient of pain levels in all organisms of any
sophistication, since even the simplest behaviour might involve
choosing the lesser of two evils. It seems to me that the best top
pain level for a fly would be the highest pain level that is
physically possible, since otherwise there would be pain
"compression", with a lesser ability to discriminate between stimuli
on the basis of how pleasant or unpleasant they are.

You might speculate that it is not physically possible to cram a lot
of pain into a fly nervous system, but you can't be sure that either
the absolute or the relative number of pain neurons is proportional to
the maximal pain intensity. Similarly with a computer program or AI,
it is not obvious that the maximal pain intensity it can experience is
proportional to its overall size or complexity.

The other aspect of it is the unremitting nature of pain. Humans can
effectively reduce the severity of pain, or suffering in general, by
being able to reason about it. The experience of chest pain is worse
if you think you're having a heart attack than if you can be reassured
that it's just indigestion. With simpler organisms, there is no room
for such cognitively mediated modulation. A bad pain is just a bad
pain, and the only way to stop it would be to remove its source.

Stathis Papaioannou

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:58 MDT