From: Jef Allbright (jef@jefallbright.net)
Date: Sun May 23 2004 - 21:39:36 MDT
Eliezer Yudkowsky wrote:
> Eliezer Yudkowsky wrote:
>
>>
>> I am not saying that you will end up being stuck at your current
>> level forever. I am saying that if you tried self-improvement
>> without having an FAI around to veto your eager plans, you'd go
>> splat. You shall write down your wishlist and lo the FAI shall say:
>> "No, no, no, no, no, no, yes, no, no, no, no, no, no, no, no, no,
>> yes, no, no, no, no, no." And yea you shall say: "Why?" And the
>> FAI shall say: "Because."
>>
>> Someday you will be grown enough to take direct control of your own
>> source code, when you are ready to dance with Nature pressing her
>> knife directly against your throat. Today I don't think that most
>> transhumanists even realize the knife is there. "Of course there'll
>> be dangers," they say, "but no one will actually get hurt or
>> anything; I wanna be a catgirl."
>
>
> Just in case it is not clear, I do not think I am grown enough to mess
> with my own source code. At best I am grown enough to be scared away
> in my own right, not just because an FAI tells me it would be a bad idea.
>
Those who attend to transhumanist discussion lists are conversant with
ideas of radical morphological change, super-intelligences, and the
whole spectrum of possible worlds explored almost exclusively within the
Sci-Fi literature and discussion groups such as this one. We tend to be
young, smart, and therefore arrogant to an extent that is appreciated
only with increasing life experience. Although we can't know what we
don't know, we are smart enough to recognize this heuristic and use it
to temper some of our speculation in the domain of the near-term here
and now.
What I think is under-represented in these groups is an effective level
of awareness and appreciation of human nature on the broader scale.
Humans are diverse, creative, and driven to improve their conditions.
Humans, within their cultural matrix, comprise the most intelligent
system we know of. This intelligent system even has built-in human
values at its core!
Unfortunately, this intelligent system today suffers from a scaling
problem; the processes that worked well enough at the level of the tribe
and village are not effective at the world global scale and humans are
at risk of failing due to a lack of global scale "management"
(perception, control and feedback) of their vital processes.
Fortunately, the necessary tools are becoming available at the time when
they are most needed. Global communications and knowledge sharing are
increasing rapidly while the economy is evolving to become completely
global -- the system is evolving beyond the human scale and humans are
taking on the role of functional components. And yes, along with this
ongoing process of growth and adaptation comes some measure of pain,
visible in terms of human suffering.
The point here is that we humans are truly and effectively part of a
larger system, and it is to our benefit to see this clearly and use this
knowledge to guide our choices. Working with the larger system means
taking actions that make the system function more effectively. What is
qualitatively different about this stage of evolutionary development is
that the cells of the organism can now predict and plan, not just
respond, to the needs of the larger organism of which they are a part
and on which they depend for their survival and well-being.
A systems design engineering approach can be effective here and requires
knowledge of the materials and operating environment (currently humans
and their ecosystem and culture) and also requires implementation of
effective sensing, control, and feedback processes as mentioned earlier.
At this stage, to minimize pain and suffering as humanity moves to the
next plateau, we don't need (yet) radical intelligence optimization of
humans or AIs. What we need is the systems-level improvements that
will allow us to get to the next level.
*Further globalization of the economy is key to creating an
interdependent planetary civilization where large-scale destruction of
"the Other" will become meaningless.
*Global communication, knowledge sharing, and adoption of new concepts
of privacy and transparency will match and help us handle the threat of
terrorist acts, and lay the groundwork for the next generation of
humanity to grow up with a global perspective.
*Global thinking will progress to the kind of larger-scope rationality
that makes the prisoner's dilemma and most zero-sum game problems trivial.
*Intelligence augmentation will progress beyond our currently primitive
mind-amplification tools and provide us with the necessary capability
for information management and collaborative decision making during the
next stage, and help us create the future.
Recently on this list, human examples of enlightened altruism were
presented. Sidhartha Gautama, Mahatma Gandhi, Martin Luther King Jr.,
(perhaps the Dalai Lama should be also included here -- all are
unenhanced humans, biologically similar to the rest of humanity, serving
as models of what is already within our reach, given the necessary and
conducive conditions.
SIAI and similar efforts play a valuable role in promoting discussion
and raising awareness of our accelerating times. AI will play a very
large role in our future, but many on this list may be surprised what
will be done without requiring machine sentience and recursive self
modification. I believe these too will come, but we have to survive the
global maturing of humanity before then.
- Jef
http://www.jefallbright.net
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:36 MST