From: Paul Hughes (psiphius@yahoo.com)
Date: Mon Mar 15 2004 - 02:01:26 MST
Hi Michael,
You wrote:
>>My first problem with the analogy is that it subtly
implies a developmentally predetermined positive
outcome to the Singularity, when this needn't be the
case. The first recursively self-improving
intelligence could easily be selfish, or obsessively
focused on a goal whose accomplishment entails the
destruction of humanity.<<<
Yes, this is definitely true, and Mark Pesce and I
agree with you. Mark has said that the bios was not
prepared for the emergence of the logos. We could
look at current species extinction as analagous
evidence of what you are saying.
>>>In this scenario, the arrival of the Singularity
needn't represent the "next stage of intelligence and
complexity in the universe" in the positive,
uplifting sense at all. A superintelligence devoted
solely to manufacturing as many paper clips as
possible could easily delegate all of its complexity
and intelligence towards plans for the complete
material conversion of the universe into
self-sustaining paperclip manufacturing facilities,
for example.<<<
I agree this is a possibility, however I think the
reality will be vastly more complex than that. The
question then is will this new emergent complexity be
friendly in any sense we can fathom, or will we be
consumed or destroyed by it? I don’t think anyone
knows what the answer to that is, and as Mark Pesce
says it is the greatest challenge we face.<<<
>>>The universe has steadily been increasing in
complexity, yes. From the anthropic point of view,
this makes perfect sense; a threshold level of
complexity is clearly required for a universe to
generate agents capable of observing it to begin with.
But once the agents have come into being,
there are not necessarily any promises for continued
survival. The forces responsible for building stable
forms in this universe across the eons do not give a
damn about us, and would continue chugging along
should we become extinct one day.<<<
Again, agreed. Nothing in Mark Pesce’s thinking would
disagree with that. This is precisely the problem
that worrying him the most.
>>My second problem with the analogy is that it
actually seems to understate the potential magnitude
of a successful Singularity.<<<
I don’t see where he does that. All Mark says is that
the next change is beyond anything our language can
fathom, and that is at least as big as the jump from
bios to logos. His 10-million fold figure was only
pertaining to the speed of logos over bios, not techne
over logos.
>>>My point here is that the Singularity radically
outclasses any historical event, even when we make
incredibly conservative assumptions. To me, making an
analogy between the rise of general intelligence and
the Singularity sounds like trying to make an analogy
between a firecracker and the Big Bang. Any analogy
will be appealing, but as far as I can tell, the
Singularity seems to be an event which is *genuinely*
new. Consider a hypothetical world where a large
transition that occurs can objectively be said to lack
any analogies. If humans lived in such a
world, don't you think they would grasp for whatever
analogies they could, just to achieve the sensation of
cognitive closure? Obviously the ancestral environment
didn't have anything remotely like the
Singularity in it, so we're clearly adapted poorly to
modeling changes of this size. Therefore I think it
makes sense to be extremely careful in which analogies
we choose to use, if any.<<<
Exactly. That was Mark Pesce’s whole point! He said
the coming changes are so great the completely
outstrip ANY linguisitic model we could possibly hope
to attach to them.
Quoting him again here,
"
And that search for a language to describe the world
we’re entering is, I think, the grand project of the
present civilization. We know that something new is
approaching."
"So we have three waves, biological, linguistic, and
technological, which are rapidly moving to
concrescence, and on their way, as they interact,
produce such a tsunami of novelty as has never before
been experienced in the history of this planet."
Paul Hughes
__________________________________
Do you Yahoo!?
Yahoo! Mail - More reliable, more storage, less spam
http://mail.yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT