Re: [agi] Two draft papers: AI and existential risk; heuristics and biases

From: Eliezer S. Yudkowsky (
Date: Wed Jun 07 2006 - 09:18:59 MDT

Michael Vassar wrote:
> I think that you are giving experiment too much credit here. In
> practice much revolutionary science (think X-Rays) happens due to new
> experimental observation, but the very most revolutionary science often
> doesn't. Darwin's theory was grounded in pure logic and inspired by a
> huge general body of pre-existing observational and experimental data.

Darwin went gallivanting around the world for more than ten years,
observing how different turtle species on nearby islands looked
suspiciously similar, before he had an insight that, in theory,
Pythagoras could have had sitting in his living room. It goes to say
something about how awful humans are at collating their evidence and
searching the space of simple hypotheses. The history of natural
selection is hardly a triumph for pure logic.

> The same is true of Newton.

If humans had been logically omniscient, or had any decent search
capability through the space of small programs, the first human to watch
an apple fall would have understood. They had all the evidence a
Solomonoff induction engine would have needed, right then.

> Gallileo's belief about equal free-fall
> velocities and his ballistics theories are basically simply assertions
> of an unappreciated null hypothesis,

A null hypothesis which he only reached after painstakingly observing
cylinders rolling down a slide and timing their exact distances
traversed using a water clock, which was the only clock accurate enough
for his purposes.

> as is the more recent work of
> Judith Rich Harris. Even Einstein's general theory of relativity is
> arguably of this type.

Einstein's general theory of relativity is the *only* case I can think
of for non-experiment-driven revolutionary science. Which is why I
sometimes use the analogy: "Imagine that you had to develop a complex
application of general relativity before anyone anywhere developed a
simple application of special relativity. Don't ask whether that would
be easy or convenient, because obviously it wouldn't be. But if the
life of the entire human species depended on it, could it be done?"

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT