RE: supergoal stability

From: Ben Goertzel (ben@goertzel.org)
Date: Fri May 03 2002 - 20:21:26 MDT


My own view is that, even if the Singularity is consciously pushed along by
a self-modifying intelligence in a hard-takeoff scenario, it will *still*
have "many aspects that are not consciously
chosen and that are suboptimal from the perspective of the intelligences
swept along by the system."

You seem to be assuming that the seed AI is effectively omniscient during
the hard takeoff process. I suspect it won't be; I suspect it will be very
very smart compared to us but not omniscient. And perhaps the complex
dynamics of society and technology as the Singularity unfolds will be beyond
even *its* capability to comprehend fully.

To my way of thinking , in the hard takeoff scenario, the seed AI is alive
and is orchestrating but not fully controlling nor comprehending the
Singularity. The Singularity is a series of events taking place over time,
rather than a specific system that one can say is alive or not. Perhaps the
total mass of minds on earth, considered as a whole, could be considered a
"living system", in which case the Singularity is a particular phase in the
life of this living system... still the Singularity is not "alive" in
itself. It seems to me that this way of thinking is more agreeable with
YOUR OWN definition of "alive" than is your statement that "the Singularity
is alive."

-- Ben G

> In some scenarios, the Singularity takes its pattern from nonintentional
> processes - the versions of the Singularity where it's the inevitable
> emergent output of a whole species and so on. But in the hard takeoff
> model, which is the one I think most likely, the Singularity is
> not a global
> process like an economy, which has many aspects that are not consciously
> chosen and that are suboptimal from the perspective of the intelligences
> swept along by the system. The Singularity is created by the first
> self-improving mind to undergo a hard takeoff; this doesn't imply that
> others are left out, but it does mean that the qualities of the
> Singularity
> are there because a mind chose them. If the Singularity takes its shape
> from intelligence and is implemented by intelligence, I would argue that
> "the Singularity is ALIVE" is a better metaphor than "the Singularity is a
> PROCESS".
>
> -- -- -- -- --
> Eliezer S. Yudkowsky http://intelligence.org/
> Research Fellow, Singularity Institute for Artificial Intelligence
>



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:38 MDT