Re: Recipe for Destruction - Joy/Kurzweil NYTimes Op-Ed

From: Eliezer S. Yudkowsky (
Date: Mon Oct 17 2005 - 18:53:54 MDT

This sounds a lot like an argument Leo Szilard once had with Enrico
Fermi over whether to publish the true neutron cross section of purified
graphite. Szilard, who had invented the idea of the fission chain
reaction, said that purified graphite might be useful in creating a
chain reaction someday, and that they shouldn't publish. Szilard was
one *damned* foresightful fellow. Today everyone knows that nuclear
weapons are a real problem; back then it was, literally, science
fiction. Nobody had proved a chain reaction was possible at this point.
  Graphite doesn't explode.

Fermi exploded. Fermi felt that a chain reaction was only a distant
possibility, and Szilard's suggestion went against everything Fermi
stood for: the international community of scientists, the ideal of openness.

Szilard and Rabi together voted not to publish, and Fermi felt obliged
to go along with the majority. As a direct result, the German A-bomb
project did not realize that graphite was an effective neutron
moderator, and went with the less efficient alternative of heavy water.
  This is one of the major reasons that Germany's A-bomb project, which
got started before the Allies, did not achieve a chain reaction before
the end of the war.

It's not like this kind of situation is historically unprecedented, for
all that only historians know the precedents.

I have to agree with Kurzweil and Joy. Publishing the genome of the
1918 Spanish flu in open-access databases seems, to me, stupid.

Eliezer S. Yudkowsky                
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:52 MDT