Re: Superhuman AGI awareness

From: Gordon Worley (
Date: Fri May 31 2002 - 19:53:21 MDT

On Friday, May 24, 2002, at 09:33 AM, Mike & Donna Deering wrote:

> Christian writes:
> "Again, how do you prepare for the unknown (or even unknowable)? I
> think most
> people would get very scared when faced with the unknown. The most
> likely
> reactions is IMO either apathy or panic."
> Back in some of Eliezer's writings he recommends if you can't
> contribute money or programming to learn to program a computer.  In my
> opinion the best way for the regular joe prepare for the Singularity is
> to read about science and emerging technology such as physics, QM,
> relativity, cosmology, string theory, quantum gravity, M theory, super
> symmetry, chemistry, biology, anatomy and physiology, cell biology,
> genetics, molecular biology, genetic engineering, molecular
> engineering, nanotechnology, and the Singularity.  You will have a
> better chance of understanding what is going to happen if you have some
> understanding of what is going on now.

This makes for a good segue into something I wanted to discuss with the

As some of you are aware, for a few months we've had a Rational Thinkers
list (for a while it was called Singularity Ethics, but that went
nowhere). Not a whole lot has happened there, but something I'm working
on starting is an owner's manual to the human mind which will include in
some fashion, after explaining how human's think (to the best of our
knowledge, anyway), information and instructions on how to become a
rational thinker (if `rational' is a blurry term for you, I roughly mean
eliminating anthropomorphic biases/thinking and learning to think

The first step is to develop a reading list of books that have been
influential in developing rational thought. In order to expand the
list, I'm asking you to reply with any additions that you might have.
Here's what we have so far (putting my initial list and Eliezer's
additions together):

Beyond anthropomorphism by Eliezer Yudkowsky (section 2 of CFAI)
The Origins of Virtue by Matt Ridley
Man: The Moral Animal by Robert Wright
The Mind's Past by Michael S. Gazzaniga
Surely You're Joking, Mr. Feynman by Richard Feynman
Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig

The next step will be to set up a collaborative Website(?) where people
can work together on this book (the project is not only more than I
could personally take on right now, but being an amateur cognitive
scientist I don't really have the credentials to write such a book on my
own). With this in mind, does anyone know of any good Web applications
that make this possible. Gratis tools are preferred, since the current
budget amounts to nothing more than some of my time. I'm sure this kind
of thing is out there, but I haven't used any of it before. In
particular, I want features like version control, user logins, and the
ability to have a variety of levels in the book (e.g. subsubsubsections).

If anyone is wondering how the heck this relates to SL4, I see
encouraging rational thought as a way to reduce public resistance to the
Singularity since, as we have seen many times on this list and other
places, most people reject the Singularity because they think too much
like an ape and not enough like a rational mind. And, even if something
like this doesn't work out to help the Singularity, it's still neat to
compile something that we can print out and hand to people when they
don't understand why they have some thought or feeling.

Gordon Worley                     `When I use a word,' Humpty Dumpty            said, `it means just what I choose                it to mean--neither more nor less.'
PGP:  0xBBD3B003                                  --Lewis Carroll

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:39 MDT