**From:** Eliezer S. Yudkowsky (*sentience@pobox.com*)

**Date:** Thu Sep 08 2005 - 10:22:55 MDT

**Next message:**Michael Vassar: "AI communities"**Previous message:**Eliezer S. Yudkowsky: "Re: The Relevance of Complex Systems [was: Re: Retrenchment]"**In reply to:**Ben Goertzel: "RE: The Relevance of Complex Systems [was: Re: Retrenchment]"**Next in thread:**Richard Loosemore: "Quick Clarification"**Reply:**Richard Loosemore: "Quick Clarification"**Reply:**Ben Goertzel: "Hempel's Paradox [ was RE: The Relevance of Complex Systems [was: Re: Retrenchment]]"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Ben Goertzel wrote:

*>
*

*> I am curious how you propose to handle Hempel's paradox of confirmation
*

*> using probabilistic semantics alone.
*

*>
*

*> There are many solutions to this problem, but I'm curious which one you
*

*> advocate...
*

Like I said, this challenge was meant for Loosemore alone; I don't have

enough time to respond if everyone piles on...

Googling on Hempel + Bayes turns up some standard responses, for example

in http://plato.stanford.edu/entries/epistemology-bayesian/

"Hempel first pointed out that we typically expect the hypothesis that

all ravens are black to be confirmed to some degree by the observation

of a black raven, but not by the observation of a non-black, non-raven.

Let H be the hypothesis that all ravens are black. Let E1 describe the

observation of a non-black, non-raven. Let E2 describe the observation

of a black raven. Bayesian Confirmation Theory actually holds that both

E1 and E2 may provide some confirmation for H. Recall that E1 supports H

just in case Pi(E1/H)/Pi(E1) > 1. It is plausible to think that this

ratio is ever so slightly greater than one. On the other hand, E2 would

seem to provide much greater confirmation to H, because, in this

example, it would be expected that Pi(E2/H)/Pi(E2) >> Pi(E1/H)/Pi(E1)."

A fine answer so far as it goes, though really it is only half the

solution. The two main points to bear in mind are that, first, a

hypothesis tells us which experiences to anticipate, not which facts to

believe. Second, evidence never confirms or disconfirms a hypothesis by

itself, it only confirms or disconfirms one hypothesis relative to an

alternative hypothesis. So I need to know what the alternative

hypothesis is to "All ravens are black", and I need to know what

probability both hypotheses assign to seeing a red lampshade given the

local sampling method. Then I'll tell you whether a red lampshade

confirms "All ravens are black" over the alternative hypothesis.

In the usual scenario, we suppose that objects are sampled randomly from

the set of all objects. "All ravens are black" does not tell us with

what probability we ought to observe a raven. It does not even imply

that any ravens exist at all. The strength of a hypothesis is what it

tells us *not* to expect. "All ravens are black" only excludes a very

tiny amount of probability mass, the possibility that we will observe a

raven-shaped object whose color is not black. Since, a priori, the

chance of seeing a raven-shaped object was 1/2 to the Kolmogorov

complexity of "raven-shaped", the hypothesis excludes only a tiny realm

of possibilities, and therefore only slightly concentrates its force

into other possibilities relative to the hypothesis of maximum entropy.

Therefore seeing a red lampshade is only infinitesimal evidence in

favor of the hypothesis that all ravens are black, as compared to the

null hypothesis. If, on the other hand, we have already seen at least

one raven, then this confirms other hypotheses which assign a

substantial probability to seeing more raven-shaped objects. And when

we do see another raven, its color will confirm subclasses of these

hypotheses which say "All ravens are black", over alternative hypotheses

that permit ravens to be more colors.

Since "All ravens are black" and "All ravens are white" exclude around

the same amount of probability mass from the maxentropy distribution

(thereby concentrating it into other possibilities), seeing a red

lampshade does not confirm one hypothesis over the over, and

infinitesimally confirms both over the null hypothesis.

But it really depends on the hypotheses that seem likely a priori, and

even more so on the sampling method. Suppose there are only two highly

probable hypotheses under consideration. The first hypothesis, that in

a group all the ravens are black and one-third of the lampshades are

red. The second hypothesis, that in a group not all ravens are black

and one-ninth of the lampshades are red. If the two hypotheses have

equal prior odds, seeing a red lampshade randomly sampled from the

subgroup of lampshades, makes it three times as probable that all ravens

are black as that not all ravens are black.

* > In PTL (Novamente's probabilistic inference component) we handle
*

* > this sort of thing via augmenting probability theory with
*

* > other mathematics.
*

I shall now demonstrate the folly of adulterating Bayes with lesser wares.

Suppose that I know that, in a certain sample, there is at least one

black raven, and at least one blue teapot, and some number of other

ravens of unknown color. I now observe an item from the group that is

produced by the following sampling method: Someone looks over the

group, and if there are no non-black ravens, he tosses out a blue

teapot. If there are non-black ravens, he tosses out a black raven.

Now observing a black raven definitely shows that not all ravens are black.

How would Novamente's "augmented" probability theory handle that case, I

wonder?

From http://www.iep.utm.edu/h/hempel.htm:

"Hempel's paradoxes are a straightforward consequence of the following

apparently harmless principles:

1. the statement (x)(Rx --> Bx) is supported by the statement (Ra & Ba)

2. if P1 and P2 are logically equivalent statements and O1 confirms P1,

then O1 also supports P2."

As the above example demonstrates, to a Bayesian, the idea that "All

ravens are black" is supported by observing a black raven, is in general

false. Though it may oft be true in the particular that observing a

black raven confirms subclasses of hypotheses which hold "All ravens are

black" over their alternatives.

-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence

**Next message:**Michael Vassar: "AI communities"**Previous message:**Eliezer S. Yudkowsky: "Re: The Relevance of Complex Systems [was: Re: Retrenchment]"**In reply to:**Ben Goertzel: "RE: The Relevance of Complex Systems [was: Re: Retrenchment]"**Next in thread:**Richard Loosemore: "Quick Clarification"**Reply:**Richard Loosemore: "Quick Clarification"**Reply:**Ben Goertzel: "Hempel's Paradox [ was RE: The Relevance of Complex Systems [was: Re: Retrenchment]]"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.5
: Wed Jul 17 2013 - 04:00:52 MDT
*