Re: answers I'd like from an SI

From: Matt Mahoney (
Date: Sun Nov 11 2007 - 15:31:48 MST

--- Wei Dai <> wrote:

> One of the things I'd like most from a friendly SI is answers to certain
> questions. If I meet an SI and it can't answer these questions or at least
> explain why they can't be answered, I would be really disappointed. But it's
> hard to imagine how an optimization process can answer them (unless the
> answers are programmed into it, or it's doing something like "search for the
> string that maximizes intellectual satisfaction for the questioner"), which
> makes me think that intelligence != optimization process. Does anyone have a
> counterargument?
> Here are my questions:
> How does math really work? Why do we believe that P!=NP even though we don't
> have a proof one way or the other?

1a. Math is the reason the universe exists.
2b. To the human mind, three examples are more convincing than a proof.

> How does induction really work? Why do we intuitively know that, contra
> Solomonoff Induction, we shouldn't believe in the non-existence of
> halting-problem oracles no matter what evidence we may see?

2a. By Hebb's rule.
2b. See question 1b.

> Is there such a thing as absolute complexity (as opposed to complexity
> relative to a Turing machine or some other construct)?

3. No. Complexity is relative to the observer. The observer is a finite
state machine. A finite state machine can only model itself

> How does qualia work? Why do certain patterns of neuron firings translate
> into sensations of pain, and other patterns into pleasure?

4a. Qualia does not exist. Evolution selects for brains that are programmed
to believe in qualia. You cannot refute this belief.
4b. Pain and pleasure are input signals to a reinforcement learning algorithm.

> How does morality work? If I take a deterministic program that simulates a
> neuron firing pattern that represents pleasure, and run it twice, is that
> twice as good as running it once? Or good at all?

5a. Morality is what you believe it is. It is not a function of pain and

> Why am I me, and not one of the billions of other people on Earth, or one of
> the many people in other parts of the multiverse?

6. Because I = me. Your real question is why is this mysterious? This is due
to a conflict between your programmed belief in qualia, and logic which says
qualia does not exist. See question 4a.

If I was an SI, I would have better answers, but then I would not be able to
communicate them to you. If you are not satisfied with my answers, then take
this pill...

-- Matt Mahoney,

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:00 MDT