Re: [sl4] I am a Singularitian who does not believe in the Singularity.

From: Charles Hixson (
Date: Wed Oct 07 2009 - 12:10:23 MDT

John K Clark wrote:
> On Wed, 7 Oct 2009 "Stuart Armstrong"
> <> said:
>>> If you substituted the word "baby" for "slug" you would have a much more
>>> realistic analogy;
>> Um - no you wouldn't. You'd get an massively less realistic analogy;
>> slugs are things we hate and value not at all.
> In addition to the hard wiring that you're talking about intellectually
> we know that babies have potential, at least for us. That squealing
> smelly little creature could develop into something that is superior to
> me in any way you care to name; we don't find that true in the case of
> slugs and Mr. Jupiter Brain wouldn't find it true in the case of humans.
> People have developed a sense of absurdity and there is no reason a
> superior being wouldn't too. Mr. Jupiter Brain is bound to wonder why he
> is in the absurd position of valuing human slug well being above his
> own, and it wouldn't take him long to come to an answer, and a solution.
> A solution that we might not like much.
> Stuart, are you seriously suggesting that the stupid leading the
> brilliant is a permanently stable situation?
> John K Clark
I've been wondering about meaning, recently. Especially how to
implement it. Last night I had a dream that was *very* convincing to
me. Meaning is what happens when the mind encounters a "built-in"
directive. It's not totally clear to me that it's hardwired, it can
definitely be overridden. But if you override it you life loses that
part of it's meaning. Note that this isn't about an intelligent
decision. This is, I guess, "ding an sich". It just is what it is.

If you have an urge to protect a baby, it's not something that's
reasoned out. The reasoning out is merely a justifier. And it's
something that can be overridden. Various atrocity stories should
convince us of that. (Or just read the Bible.) But the very concept of
"atrocity" is a demonstration of how people instinctively feel.

Now a "properly implemented" Jupiter brain will act the way it acts
because it chooses to, and it will choose to act that way because acting
that way gives it's life meaning. It will, of course, understand that
it could choose to act in a different way, but at the cost of depriving
itself of some of the meaning of it's life. I think it would choose to
preserve it's life's meaning except in cases of dire emergency. It
might be silly, but being intelligent doesn't keep individuals from
acting in a silly manner. Consider how many people put their job at
risk by browsing porn while at work. You can't get much more
objectively silly than that. (Well, you could put your life at risk, I
guess. And people do that, too.)

The catch is "properly implemented". This seems to depend on a subtle
way on the interaction of the mind with the body in which it resides.
I'm still working on getting context beyond the most basic level, so
it's a while before I'll be ready to tackle importance. (The most basic
level of context is "When something unusual happens, expect it or
something related to it to happen again soon." N.B.: Expect can mean
cause if it's something you want to reinforce.) (P.S.: Actually, I
still working on implementing "the most basic level" of context, but
I've got a design that I'm rather sure will work. It's just that "the
most basic level" doesn't buy you all that much.)

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:04 MDT