Re: Volitional Morality and Action Judgement

From: Eliezer Yudkowsky (
Date: Mon May 24 2004 - 19:48:16 MDT

Ben Goertzel wrote:
> Hi,
>> Ben? Put yourself in my shoes for a moment and ask yourself the
>> question: "How do I prove to a medieval alchemist that there is no way
>> to concoct an immortality serum by mixing random chemicals together?"
> Pardon my skepticism, but I don't believe that the comparison of
> A) your depth of knowledge about FAI, compared to mine
> with
> B) modern chemical, physical and biological science, versus the medieval
> state of knowledge about these things
> is a good one.
> Frankly, this strikes me as a level of egomania even beyond the very
> high level that you normally demonstrate ;-)

Feel free to explain how a realistic and frightened medieval alchemist can
convince a hopeful, cheerful, vaguely mystical medieval alchemist that
there is no way to concoct an immortality serum by mixing random chemicals
together. Bearing in mind that the first alchemist has to drink whatever
the second alchemist comes up with. Bearing in mind that there is in fact
no way to do it, and that being ignorant of this does not change Nature's
law in the slightest.

I do not say I am a medieval alchemist, mind you. I think there is far
more existing science that can be brought to bear on this subject than
people realize. Humanity has already accumulated vastly more math and
science than most people, including most scientists, have any inkling
exists. That is what it means to live in a world of six billion people
with printed literature, instead of an aliterate tribe of two hundred
hunter-gatherers. No one can say what science does not know. There is too
much science. Funny, how the people asserting the ignorance of science on
some subject are so rarely specialists in that *particular* field...

But the puzzle you pose is just as dangerous, and just as difficult to survive.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT