Forbidden Planet

From: Keith Henson (
Date: Sun May 09 2004 - 23:35:32 MDT

Been writing about a really dismal subject that has application to one way
some people think is a route to AI.

I have been looking at conditionally switched on human psychological traits
and was working on the traits that are the origin of war. But recently I
was distracted into writing about the counterpart to capture bonding in
response to the news about the Iraq prison abuse.

Years ago, Zimbardo had to shut down his famous Stanford prison experiment
early because those chosen as "guards" were getting too vicious to those
being "prisoners."

The experiment showed that people switch into abuse mode when they have
captives present. This doesn't make sense in the present world, but it
very likely did in the world in which we evolved (and which shaped our
psychological traits including the conditional ones).

We know that humans respond to capture and abuse by socially reorienting
and bonding--the Stockholm Syndrome. The evolutionary argument as to where
this trait came from is obvious. (If you reoriented and bonded to the
tribe that had captured you, you had a good chance of being an ancestor,
otherwise, you were most likely killed.)

It also makes sense that the trait to abuse captives would have also been
selected. The argument isn't as obvious as the survival link with
capture-bonding. But it figures that in a world where 10% of an average
tribe's females were captured, those who had the genes for an "instinct"
for the brutal behavior needed to capture and turn on the capture-bonding
trait in the captives left more descendents than those without it.

And, like the capture-bonding trait, over a long enough time the trait to
induce capture-bonding (TTICB) would become nearly universal. I.e., it
would be triggered in response to the conditions needed to turn it
on. That's probably the evolutionary origin of the trait expressed by
Zimbardo's "guards" The trait to be brutal gets automatically switched on
by the mere presence of captives.

These traits had overall positive effects (for genes) when our remote
ancestors all lived as hunter-gatherers. The abuse caused the captives to
socially reorient to the captors and they were incorporated into the tribe
that had captured them. The closest modern behavior is frat hazing,
limited duration abuse followed by incorporation into the "tribe."

But prisons (which of course didn't exist in the environment in which we
evolved) stall the process of taking a captive into a tribe so the abuse
phase goes on without letup.

A dire and dismal business. Unfortunately it has application to the
friendly AI problem.

One path to AI that has been proposed, "if we can't do it any other way" is

If you upload human minds into very capable hardware, you want to be
absolutely sure that you don't turn mechanisms such as this one or the ones
that lead to wars. And how many of these conditionally switched on
dangerous psychological mechanisms do humans have left over from the
Pleistocene? And what environmental conditions turn them on?

Darned if I know.

John Tooby figured out capture-bonding about 1980 (but didn't publish for
some reason). I figured it out in 1996. Took 8 years for it to dawn on me
that the corresponding evolved trait to treat captives brutally (to turn on
capture-bonding) accounted for Zimbardo's prison experiments.

In retrospect these mechanisms are so obvious that it makes you wonder if
we are not subject to a psychological blindness to conditionally switched
on psychological mechanisms. Thinking about them may make us feel we are
less in control than we would like, but most people understand that there
are environmental circumstances (such as a loved one being attacked) will
make killers out of anyone. It perhaps more disturbing to realize that
there are simple environmental circumstances will make us vicious toward
helpless captives.

In any case, uploading looks far more dangerous than it did a week ago.

Keith Henson

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT