**From:** Eliezer S. Yudkowsky (*sentience@pobox.com*)

**Date:** Tue Sep 13 2005 - 19:28:42 MDT

**Next message:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Previous message:**Ben Goertzel: "RE: Logics at multiple levels of abstraction"**In reply to:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Next in thread:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Reply:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Ben Goertzel wrote:

*>
*

*> -- As it turns out standard probabilistic semantics and PTL say
*

*> basically the same thing about Hempel's paradox, though they
*

*> express it in different ways. This is interesting to me,
*

*> though in the big picture not surprising. PTL was created in
*

*> order to formula probabilistic ideas in a way convenient for
*

*> AGI, not to contradict probability theory.
*

The probability theory that is currently conventional, sometimes called

"Bayesian", got that way for a reason. There were some very fundamental

proofs that you *had to* do it this way in order to be consistent.

If Novamente has betting odds where it will buy gambles priced under

those odds and sell gambles priced above those odds, and those odds

disobey the laws of conventional probability theory in their relations

to one another, dutch book may be made against Novamente - I can arrange

a set of bets which Novamente will individually accept and which lead to

a guaranteed loss regardless of the real state of the world. If PTL's

two numbers lead it to bracket probabilities by two numbers, as in

Dempster-Shafer theory, so that Novamente will buy gambles priced under

one number and will sell gambles priced above a higher number, then

dutch book may not be made against Novamente so long as every bracket

contains the Bayesian value. Perhaps it is faster to compute

probabilities in this way. (Perhaps not.) However, I can still arrange

a set of gambles which Novamente will refuse and which would be a

deductively guaranteed gain, unless all lower and upper brackets

coincide with the exact Bayesian price. That's the cost of saving on

computing power, even if your approximation doesn't give absurd answers.

Then there are all the theorems which prove that reasoning consistent in

various ways - such as the final probability (betting odds) being the

same, regardless of the order in which the evidence is taken into

account - must be Bayesian. Et cetera. Et cetera.

If you haven't read the particular work of Jaynes, "Probability Theory:

The Logic of Science", read it, especially chapters 2 and 15. Or maybe

Jaynes isn't the remedy that's right for you. But what you *needed* to

do was sit bolt upright and say "Wait, something's wrong with my math!"

as soon as you thought PTL gave an answer substantially different from

the Bayesian one. This would have saved some time, even if you didn't

know whether you'd messed up the Bayesian math, the PTL math, or both.

I recently saved Michael Wilson some time by looking at a complex

formula he'd derived, which he'd sent back to me to see if there was a

more efficient way to do it. I looked at the formula and said, "This

has to be doing *something* wrong," because Bayesian probability theory

said he couldn't compute the end result from his premises. It turned

out he was, literally, dividing by zero.

*> -- My issue with probability theory as a foundation for AI has
*

*> to do with its inability to deal with some issues in a
*

*> computationally efficient way, without the addition of a
*

*> significant amount of additional concepts. The main
*

*> examples I have in mind here are attention allocation in
*

*> a complex cognitive system, assignment of credit, concept
*

*> creation, and the learning of schemata for the control
*

*> of inference trajectories. These things certainly can be done
*

*> in a way that's *consistent* with probability theory, but,
*

*> seem to require the addition of a lot of structures and
*

*> dynamics that are not suggested by probability theory.
*

The problem is that you're regarding Bayesian probability theory as a

computational library, a tool for solving problems. It's not.

Probability theory is the underlying mathematics of the thing you're

trying to do. It's how you recognize intuitively when your

approximation delivers an absurd result. If you can compute things out

by brute force probability theory, fine; if not, then approximate using

a different algorithm - but don't lose sight of the privileged status of

probability theory. Probability theory is not a computer program.

Probability theory is not a fast algorithm for AI. It's your sanity

check. It's your casting-out-nines. It's how to spot an absurd end

result, and go back and check your approximation line by line against

the equivalent Bayesian derivation, until you find the line where you

divided by zero. Probability theory tells you the price (in foregone

gains) of saving on computing power, and says what your approximation

has to do to avoid guaranteed losses.

I wouldn't have gotten on your case if you'd said, "Novamente

approximates the evidence as zero to save on computing power, though the

Bayesian probability is infinitesimally positive." It's the part where

you said Bayesian probability theory was *wrong* where I got out the tar

and feathers and started heaping firewood around the stake in the town

square. PTL doesn't pull enough weight, and intuition doesn't pull

enough weight, to contradict a theory that's only got three million

published proofs of its unique consistency. That sort of thing makes me

think that you don't respect probability theory, bro, and if I let

people go around dissing Bayes, soon Bayes won't get no respect. If you

find a case where you think PTL gives a different result from

probability theory, and you see nothing alarming about this, but instead

conclude offhandedly that probability theory must be wrong, you will end

up not noticing when you divide by zero.

-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence

**Next message:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Previous message:**Ben Goertzel: "RE: Logics at multiple levels of abstraction"**In reply to:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Next in thread:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Reply:**Ben Goertzel: "RE: Hempel's Paradox -- OOPS!"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:00:52 MDT
*