Re: Pre-Singularity human enlightenment

From: Eliezer S. Yudkowsky (
Date: Fri Sep 20 2002 - 12:21:01 MDT

Samantha Atkins wrote:
> Considering that we haven't really bad even written records for more
> than 10,000 years, that is about 40,000 years too much to consider.
> Also, until recently, as I have attempted to point out, we were not in a
> technological and economic position to escape certain harsh realities.
> Those potentially open new possibilities for how we consciously perceive
> what "life is all about".

Okay. I have no problem with restricting it to the last 10,000 years if
it's something that requires writing but hasn't been tried over that time,
or the last 500 years if it's something that requires printing but has
never been tried, and is significantly different and significantly more
powerful than anything that has previously been tried.

Again, not trying to be difficult; this is a test met by seed AI, as a
strategy in the battle against human suffering.

> Now, don't get me wrong. The above is a good question. However, I am
> not yet prepare to offer an answer I believe goes deep enough to satisfy.

Okay. I'm glad you're not jumping on an answer. Just remember that there
*is* at least one answer which *does* satisfy the above the requirements.
    And that some of us are moving on it already. Speed counts,
sometimes. If you want us to do something differently, such as spend X%
of our resources on something else, we're going to want concrete answers
to certain questions.

>> Remember, us AIfolk have to deal with the shadow of fifty years of
>> failure; everyone wants to know what we're going to do that hasn't been
>> tried before. You have to dig yourself out from under the heap of other
>> people's failures; show that your thinking is new enough not to belong in
>> the trash heap with the things that have been tried before.
> I think this is a poor way of thinking. Each effort stands more on its
> own merits without being bogged down by everything before. Yes we learn
> from past failures but we do not have to personally redeem all that
> baggage.

Why not? It seems like a good test to me.

>> Why is your thinking that new? Why are your plans that original?
> New and original are not necessarily primary criteria. The situation is
> new. The communication bandwidth is new. Can something effective be
> done with that? Perhaps. It is to be designed.

Then just do it. It's the very first test you're facing. It is very far
from the last. It's what you have to do right at the beginning in order
to get started. *Meet* the challenge, don't just acknowledge it. Face it
down, accomplish it, get it over with, and go on to the next step on the
path. You are running out of time. Admiring a problem is a poor
substitute for solving it, especially if it's just a first step.

>> Understand, I am not asking this in order to be difficult. I can
>> think of
>> at least three things you could try, in terms of promoting greater human
>> enlightenment before the Singularity, that have never been tried
>> before. What I want to know is what *you're* thinking of.
> Please share those things. They may make a real difference. My thinking
> is early and tentative. I am not fully ready to lay it out.

1: See whether humans can use feedback from realtime fMRI to help
identify internal rationalization, hatred, tribal-based thinking, etc.
You may not get all the bugs, but if you can get just some, it may be
enough to tip the internal mental balance. Evolution has no experience
puppeteering humans with access to that information.

2: Collaborative filtering and/or the Earthweb; a means whereby one good
idea can very rapidly spread to billions and be built upon by thousands.
Accelerated memetic evolution.

3: See whether humans who talk to infrahuman but clear-thinking Friendly
AIs learn anything from that about how to think rationally. In other
words, AI-trained humans may be doable pre-Singularity.

What I'm trying to say is that you have not accomplished anything until
you pick one of those strategies or something like it, and get started on
implementing it directly. The Singularity is not a religion, it is a
GANTT chart. You cannot worship it, admire it, believe in it, or accept
it into your heart; you have to find one of the open boxes and turn it green.

>> It doesn't do much good to issue calls to action without a strategy that
>> promises to work. Not a strategy that *sounds* good. Not a strategy
>> that blazes like a banner and uplifts your heart. A strategy which,
>> unlike the stuff that's been previously tried over the last 50,000 years,
>> will actually work.
> Uplifting the heart and firing the imagination is of course still a
> crucial part of reaching enough people. What do you believe will
> actaully work?

Seed AI.

Or computer-mediated telepathy between 64-node clustered humans whose
prefrontal cortices have learned to talk to one another over meganeuron
broadband connections.

Some form of transhumanity which can be achieved as rapidly as possible
using our existing technological base; that's a big enough hammer. I
don't believe there's much else that *really would* work, however nice it

Uplifting the heart and firing the imagination is a crucial part of
reaching enough people, but having a workable strategy comes first.
Otherwise the cheering doesn't *accomplish* anything. Humans like
cheering and will cheer whether or not anything gets done as a result. I
am deeply suspicious of cheering, and I speak as a cheerleader.

>> Again: What are you going to do, and why is it going to work when
>> everything previous has failed? To quote Doonesbury, "Let's kick
>> butt!" is not a plan unless you know which butts to kick, how far, and
>> in which direction.
> That notion of kicking butt is actually almost diametrically opposed to
> what I believe needs to happen.

I know. Still a good quote.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:40 MDT