Re: The dangers of genuine ignorance (was: Volitional Morality and Action Judgement)

From: Marc Geddes (
Date: Thu May 27 2004 - 19:54:11 MDT

 --- Eliezer Yudkowsky <> wrote: >
Marc Geddes wrote:
> >
> > Ooh, ooh, can I have a go at explaining
> consciousness?
> >
> > It has to be some deep 'self-referential' property
> of
> > the universe itself I think. So I would lean
> towards
> > panpsychism: there has to be some 'ground state'
> of
> > pure awareness in everything.
> Another interesting lesson of scientific history is
> that when people are
> faced with apparent mysteries, they postulate
> mysterious substances as
> explanations, rather than specific complex mundane
> processes. Biology
> becomes "elan vital", molecular chemistry becomes
> "phlogiston".
> Only a true Bayesian will notice that these
> "explanations" have no internal
> detail and cannot concentrate their probability mass
> - cannot make advance
> predictions, even in retrospect.
> --
> Eliezer S. Yudkowsky
> Research Fellow, Singularity Institute for
> Artificial Intelligence

I'm not sure you've given a good analogy. As far as I
can make out, Panpsychism does not regard qualia as
some kind of 'substance'. It just postulates an
irreducible ground-state of 'awareness' in everything,
stemming from the hypothesis that low-level Qualia is
associated with some deep abstract invariant built
into the structure of reality which is omnipresent.
Given that Qualia seem to be some sort of deep
property arising from computation (which is indeed
omnipresent as far as we know - all known physical
processes are computable in principle), the hypothesis
seems to me to be quite likely. Of course, even if
the hypothesis is true, higher level forms of qualia
(like human awareness) would still be explained mostly
in terms of the specific complex properties associated
with brain processes.

Tell you what Eliezer,

If you can honestly say you've reached the point where
you understand what 'Qualia' are, and you understand
what 'Personhood' is, then the last of my doubts about
your approach to FAI would be dissipated.

But if you don't yet have the answers, then prudent
doubts about your approach must linger. It is all too
likely that finally understanding 'Qualia' and/or what
constitutes a 'Person' could deliver an intellectual
bomb-shell which blows your approach out of the water.

Any way, I've 'put my money where my mouth is' and
made claims in public (which are permanently lodged in
the sl4 archives). Here are my claims:

*General intelligence without consciousness is
*General intelligence without a 'Self' is impossible
*General intelligence that is totally altruistic
(totally non observer centered) is impossible

Let's see who turns out to be right about this shall
we? The great intellect of Eliezer... or the lowly
intuition of that New Zealand nobody, yours truly ;)

"Live Free or Die, Death is not the Worst of Evils."
                                      - Gen. John Stark

"The Universe...or nothing!"

Please visit my web-sites.

Science-Fiction and Fantasy:
Science, A.I, Maths : - Yahoo! Personals
New people, new possibilities. FREE for a limited time.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT