RE: Volitional Morality and Action Judgement

From: Thomas Buckner (
Date: Sat May 29 2004 - 16:54:21 MDT

--- Michael Wilson <> wrote:

> > With a complete Novamente system that is enabled
> to self-modify its
> > cognitive schemata, there will be a greater than
> zero risk, and more
> > careful risk analysis will be needed.
> Wherever there is the possibility of evolutionary
> dynamics as you know
> them, there is takeoff risk.
> * Michael Wilson

In relation to this, and to an earlier post on the
difference between forecasting and control (as in
weather) I am reminded that intelligence without
control is possible. The largest brains belong to
whales (sperm whales have 20 pound brains) and very
little of that is needed to control the body; some
species of cetaceans might have human-equivalent
intelligence, used differently of course; their songs
are complex and passed on culturally. But fate has
placed them in a world where they have almost no
control over their environment. They have neither
manipulative hands nor tools. You can't build a fire
down there, smelt metal, make circuit boards. But they
haven't been able to ruin their environment either. It
might be said that the whales struck an evolutionary
bargain: an oral culture might have existed among
whales since before the first australopithecines, but
the whales were denied both the fruits and perils of
technology. Whatever intelligence whales may possess,
it is a quarantined intelligence. If an AI could be
quarantined as efficiently as this, takeoff risk would
be minimal.


Do you Yahoo!?
Friends. Fun. Try the all-new Yahoo! Messenger.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT