Re: Donaldson, Tegmark and AGI

From: Russell Wallace (
Date: Sat Aug 12 2006 - 16:51:42 MDT

On 8/12/06, Brian Atkins <> wrote:
> I worry that subconscious factors involved with faith-based
> reality-view belief systems could affect important decision making if one
> were
> not superhumanly vigilant in remaining unbiased.

To expand a little on this: you remind me of a remark I think Eliezer made
awhile back to the effect that religious belief puts an upper bound on
rationality. I'm not sure the reverse isn't the case. It is true of course
that some people hold irrational beliefs for religious reasons, the most
notorious example in modern times being the creationism versus evolution
debate. On the other hand it may be for some people that having a place to
stand, so to speak, outside our world, may free them to look more
objectively at empirical propositions within this world.

I'm reminded of the remark, related to the popularity of New Age memes:
"when people stop believing in God, it's not that they believe nothing -
it's that they'll believe anything". That is, in the absence of religion,
people may find they have to choose a place to stand within this world:
their _empirical_ beliefs may have to be modified away from the function of
reflecting reality in order to serve this second purpose. This can have
significantly negative effects.

Consider for example hard takeoff. Stepping back and taking a cold look at
it, the belief that such power can be obtained by shuffling bits in a RAM
chip is in the same ballpark as the belief that you can summon demons by
chanting phrases in Latin: it's something whose source is very clearly the
internal workings of the human mind rather than external reality. Even
without applying domain-specific knowledge, it's not something an objective
observer would ever have ended up with.

So why did we, such smart rational people, ever take something like that
seriously? Because in the absence of traditional religion we needed a place
to stand; and my purpose here is to offer an alternative.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:57 MDT