From: Michael Anissimov (michael@acceleratingfuture.com)
Date: Sun Mar 14 2004 - 21:54:18 MST
Heya Paul!
While a useful tool for understanding the Singularity, some aspects of
the analogy between the rise of Homo sapiens and the possible rise of
superintelligence worry me. John Smart (www.singularitywatch.com) has
been using this analogy heavily since he appeared on the Singularity
analysis scene, so my concerns with Pesce's analogy definitely apply to
him as well.
My first problem with the analogy is that it subtly implies a
developmentally predetermined positive outcome to the Singularity, when
this needn't be the case. The first recursively self-improving
intelligence could easily be selfish, or obsessively focused on a goal
whose accomplishment entails the destruction of humanity. In this
scenario, the arrival of the Singularity needn't represent the "next
stage of intelligence and complexity in the universe" in the positive,
uplifting sense at all. A superintelligence devoted solely to
manufacturing as many paper clips as possible could easily delegate all
of its complexity and intelligence towards plans for the complete
material conversion of the universe into self-sustaining paperclip
manufacturing facilities, for example.
Approaching the Singularity with the belief that "the history of our
universe has been a continuous ingression of intelligence into matter"
(as I originally did!) can give us a false sense of relief, a feeling
that the overarching hand of meta-evolution is guiding us towards a
desirable outcome. As it so happens, I do disagree that the universe has
been unfolding in this way. The word "intelligence", when unpacked,
subtly signifies a substantial set of traits; problem-solving ability,
carefulness, better choice of goals, conscientiousness, synergy with
surroundings... even sociability, altruism, and fairness. Yet the vast
majority of these traits are possessions of humanity alone. Passing
these precise traits on to superintelligence will not be easy, but
hopefully it is at least possible.
The universe has steadily been increasing in complexity, yes. From the
anthropic point of view, this makes perfect sense; a threshold level of
complexity is clearly required for a universe to generate agents capable
of observing it to begin with. But once the agents have come into being,
there are not necessarily any promises for continued survival. The
forces responsible for building stable forms in this universe across the
eons do not give a damn about us, and would continue chugging along
should we become extinct one day.
My second problem with the analogy is that it actually seems to
understate the potential magnitude of a successful Singularity. John
Smart argues that the discrepancy in information-processing between
major phase transitions (say, genetic and epigenetic information
patterns) works out to around 10 million, but this difference actually
seems minor in comparison to what a Singularity might bring. If human
general intelligence is represented by our brains, and our brains weigh,
on average, 1.3 kilograms, each processing roughly 10^17 ops/sec of
information (whether all these operations are functionally relevant to
symbol manipulation or general intelligence is heavily questionable),
then the 100 billion or so humans that have existed throughout history
add up to around 10^28 total ops/sec running on roughly 1.3 x 10^11
kilograms of meat.
Say that the first transhuman intelligence acquires a modest level of
nanotechnology, like the rod-logics that Drexler described in
"Nanosystems", which process roughly 10^18 ops/sec with a single
kilogram. Say that this transhuman intelligence quickly went off into
the Solar System and grabbed a quantity of mass equal to about one
thousandth the mass of the Earth. That works out to about 6 x 10^21
kilograms. Say it then released self-replicating nanobots onto that
mass, converting it into computing components to expand itself, upload
people, run more intelligences...whatever. Already you have roughly
10^39 ops/sec of computing power there. And with that amount of
computation, surely someone would figure out how to build better
computing elements, and gather more matter, and increase the overall
intelligence of the general collective...
My point here is that the Singularity radically outclasses any
historical event, even when we make incredibly conservative assumptions.
To me, making an analogy between the rise of general intelligence and
the Singularity sounds like trying to make an analogy between a
firecracker and the Big Bang. Any analogy will be appealing, but as far
as I can tell, the Singularity seems to be an event which is *genuinely*
new. Consider a hypothetical world where a large transition that occurs
can objectively be said to lack any analogies. If humans lived in such a
world, don't you think they would grasp for whatever analogies they
could, just to achieve the sensation of cognitive closure? Obviously the
ancestral environment didn't have anything remotely like the Singularity
in it, so we're clearly adapted poorly to modeling changes of this size.
Therefore I think it makes sense to be extremely careful in which
analogies we choose to use, if any.
Thanks for sharing this article with the list, in any case! I thought it
was particularly interesting because it brings up the analogy of casting
spells on the external world to achieve effects, an analogy I've been
using carefully since I first discovered the concept of the Singularity.
It's also nice to see a new prominent figure taking interest in the
Singularity!
Michael Anissimov
Paul Hughes wrote:
>Based on some readings I've done recently I've come
>acress a new context to understand the singulairty.
>
>I think it can be stated without much argument that
>the history of our universe has been a continuous
>ingression of intelligence into matter. From lesser
>complexity to greater complexity. If this is not
>apparent, then I suggest reading either Stephen
>Wolfram’s or Stuart Kaufmann’s work.
>
>Even though this acceleration of intelligence seems
>continuous there is at least a few punctuated periods
>of more rapid acceleration. I think the singularity
>represents a punctuated rupture with at least one
>historical precedent - the birth of language, the jump
>from genetic to epi-genetic information transfer.
>
>Until Modern Humans emerged on the scene, all
>evolution was happening via DNA information transfer –
>a very slow glacial process of evolution. At some
>point in Homo sapiens evolution we developed symbolic
>manipulation ability.
>
>Here is some snippits about the Singularity (a bit
>long but interesting) from Mark Pesce, inventor of
>VRML.
>
>~~~
>
>The thing that separates us from the Neanderthal isn’t
>brain size, or brute strength, but a symbolic
>manipulation capability.
>
>In order to have symbols, you need to have a
>consciousness capable of symbolic manipulation, that
>is to say a linguistic consciousness. While
>paleoanthropologists believe that the Neanderthal had
>some very basic linguistic capabilities, it is
>believed that these abilities were very limited –
>perhaps similar in nature to those of a year-old
>child, capable of identifying objects or actions, but
>little more.
>
>What we see with homo sapiens is that this linguistic
>ability overflowed into the entirety of consciousness.
>The first benefit of this was the emergence of what we
>understand as language: nearly every human being has
>an innate capability to take a few symbols and
>manipulate them infinitely.
>
>For example, although few of us ever use more than
>about 2000 English words, we can describe just about
>anything with those words, because we can
>instantaneously recombine them in any sensible order
>to create new forms of expression.
>
>For 4 billion years, DNA was the recording mechanism
>of history, the memory of biology. As soon as we
>developed language, we no longer needed the slower
>form of DNA for memory; we could use the much faster
>form of language, which produced with it a deep sense
>of memory within the individual – since the linguistic
>symbols could be contained within the human mind.
>
>Since we became a symbol-manipulating species, our
>forward evolution, in DNA terms, has come to a dead
>stop. (This has recently been proposed by reputable
>scientists.) However, our linguistic capabilities
>allow us to perform acts of memory much faster than
>DNA, probably at least 10 million times faster!
>So, suddenly, homo sapiens is not just a biological
>entity working within the matrix of DNA and its slow
>historical recording, but now bursts through and
>starts processing its interactions within the
>environment 10 million times faster than ever before.
>
>For all of evolutionary time, information had to
>travel the slow route through biology – through the
>bios - before it would be coded into our DNA. Now we
>had this additional process – which we call the logos,
>the Word – which was a completely new thing, and not
>something that the bios had any time prepare for.
>
>>From its first recognizable moment, humanity
>demonstrates an entirely new relationship between bios
>and logos. Information, freed from its need to be
>embedded in the slow, dense vehicle of our DNA, speeds
>up 10-million-fold.
>
>However – and this is the second most important point
>I want to make today – the logos has its own
>teleology, its own entelechy, its own drive to some
>final dwell-state.
>
>We assume that we are masters of language, of word and
>world.
>
>I disagree.
>
>The situation is exactly reversed. We are not in
>control of words, they control us.
>
>Evolutionary biologist Richard Dawkins got it entirely
>right when he invented the concept of “memes,” which
>can be thought of as the linguistic equivalent of
>genes. Rather than being part of the bios, memes are
>the carriers of the logos.
>
>OK, so we’ve covered the emergence of the bios, some 4
>billion years ago, and the emergence of the logos,
>perhaps as much as 150 thousand years ago.
>We, as a species, have been driven by memes for the
>last hundred thousand years, and this has forced us
>further and further away from any direct connection
>with the natural world.
>
>We’re being hollowed-out by our memes. That is to say
>that our interiority, which is an artifact of the
>slow, quiet progression of the bios, is rapidly
>vanishing.
>
>The modern conception of interiority is really a
>creation of the Enlightenment in Western Europe, and
>was only noted by philosophers as it was beginning to
>vanish utterly.
>
>So here’s the central point of what I wanted say: the
>singularity is absolutely inevitable, and absolutely
>meaningless. The closest analogy we could make would
>be the whine of feedback you get when you place a
>microphone too close to an amplifier. The screech
>drowns everything else out, just as what we are – as
>individuals and culture – are being replaced by a
>rising form of activity dedicated to a single goal:
>making a clear path for the transmission of the logos.
>We’re improving the fidelity of meme swapping until it
>asymptotically approaches its theoretical limits.
>
>The transmission of facts and ideas is already
>becoming instantaneous, and the speed of the
>development of novelty followed. When ideas move
>faster, there’s a greater capacity for them to
>interact, to produce concrescence.
>
>The history of the 20th century could accurately be
>described as a series of advancements in
>communication, beginning with radio and ending with
>the Internet, each technology successively colonizing
>the world, and each more rapidly than the technology
>before.
>
>Technological artifacts are concretized language; that
>is, any technology is a bit of language that has been
>turned into a physical object.
>
>The first technology that was turned into a physical
>object was the linguistic technology itself. Writing
>is the first real technology of importance, because it
>freed linguistics from their oral substrate, and made
>the carrier medium much more durable. We have an idea
>of history from 3500 BCE forward because of the
>invention of writing, which has created a continuity
>in humanity.
>
>All other technologies, are, each in their way, the
>descendants of writing. Writing was the
>exteriorization of our drive to communicate.
>We’ve seen the linguistic acceleration of DNA as
>codes, and the linguistic acceleration of
>communication as telecommunication, but we’re only now
>on the threshold of the acceleration of technology.
>
>Things may look as though they’re going fast now, but
>this is nothing – literally, absolutely nothing – next
>to what’s about to happen, because (and now we have
>precedent for it) we’re about to see a technological
>acceleration on a similar order to the acceleration we
>saw when the logos separated from the bios. In this
>case, techne, our ability, is about to be freed from
>logos, our ability to describe it.
>What do I mean when I say this?
>
>There’s an emerging science, known as nanotechnology,
>which will, before the next few years have passed by,
>give us a very fine-grained control over the material
>world.
>
>With nanotechnology we should be able to precisely
>design molecules to order, for whatever purpose we
>might desire.
>
>This is the coming linguistic revolution in
>technology, because, at this point, the entire fabric
>of the material world becomes linguistically pliable.
>Anything you see, anywhere, animate, or inanimate,
>will have within it the capacity to be entirely
>transformed by a rearrangement of its atoms into
>another form, a form which obeys the dictates of
>linguistic intent.
>
>It’s very hard for us to conceptualize such a world,
>and I have continuously been forced to draw on the
>metaphors of world of magic for any near analogies.
>It will be as if we have acquired the ability to cast
>spells upon the material world to achieve particular
>effects.
>
>This isn’t to say that we’re about to acquire the
>omnipotence we normally ascribe to God, but that our
>abilities will be so far beyond anything we’re
>familiar with today that we have no language to
>conceptualize them. No language at all.
>
>And that search for a language to describe the world
>we’re entering is, I think, the grand project of the
>present civilization. We know that something new is
>approaching.
>
>So we have three waves, biological, linguistic, and
>technological, which are rapidly moving to
>concrescence, and on their way, as they interact,
>produce such a tsunami of novelty as has never before
>been experienced in the history of this planet.
>
>
>
>__________________________________
>Do you Yahoo!?
>Yahoo! Mail - More reliable, more storage, less spam
>http://mail.yahoo.com
>
>
>
>
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:46 MDT