Re: SIAI's flawed friendliness analysis

From: Barkley Vowk (
Date: Thu May 29 2003 - 16:36:20 MDT

Perhaps I'm mistaken, but you are NOT nature right? I think it would have
been easier to NOT have pulled that response out of your ass and just
answered the question.

On Thu, 29 May 2003, Eliezer S. Yudkowsky wrote:

> Philip Sutton wrote:
> >
> > I would like you to explain why in language that a non-mathematican can
> > understand. If you can't get around to explaining your ideas in a form
> > that an intelligent, informed non-mathematician can understand then you
> > are commiting yourself to fail to communicate with the people you want
> > to pusuade not to adopt Bill's approach.
> Nature is not obligated to make her problems easy enough for intelligent,
> informed non-mathematicians to understand them. But of course people get
> quite indignant when told that a problem may be too difficult for them.
> Maybe, *maybe* if it's someone like a physicist, a nice
> already-established famously difficult area of science, someone might be
> willing to believe that this field is too difficult to be grasped over
> lunch. Why? Because it taps into the ready-made "witch doctor" instinct
> for understanding a field as arcane and barred to outsiders. Lacking any
> established witch doctors, of course, the presumption must be that your
> opinions are as good as anyone's and that the problem itself is, oh, about
> as simple as anything else your brain expects to run into.
> Hunter-gatherers don't confront hard scientific problems. But just
> because there isn't a field of AI with an established, confirmed theory of
> intelligence, and scientists with reputations for being in a difficult
> field, does not mean that the problem of AI will be simple. The
> difficulty is set by Nature. I might try to explain the problem to an
> intelligent, informed non-mathematician. But remember that Nature is
> under no obligation *whatsoever* to make the problem comprehensible.
> If you are doing something that will, in fact, kill you, Nature is under
> no obligation to make this obvious to you. Nature has no privileged
> tendency to avoid killing people whenever her reasons cannot be explained
> to an intelligent, informed non-mathematician.
> Now, bearing that in mind, you might start at:
> and go on from there.
> --
> Eliezer S. Yudkowsky
> Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT