From: Eliezer Yudkowsky (sentience@pobox.com)
Date: Fri Oct 24 2008 - 10:39:19 MDT
On Fri, Oct 24, 2008 at 9:26 AM, Alex F. Bokov <alexboko@umich.edu> wrote:
> Actually, I'm not talking about individual volition. I'm talking about
> self-organizing clusters of coherent extrapolated volition. At the
> moment, it seems like Eliezer is willing to concede failure of CEV
> altogether if it fails to converge on one consensus (his Final Judge
> comments), and other parts of CEV.html. I'm just asking why not
> empirically determine the optimal number of CEVs, and follow them all?
That would strain the moral ability of the original programmers to
decide how the different human communities would fit together, how to
divide up resources, what to do if one piece thinks that everyone is
allowed to have nuclear weapons and another piece doesn't want to have
anyone on the borders who can launch nuclear attacks, etc. The whole
point of this is not that it seems like such a pleasant thing to do
with absolute power, but that we don't want to put such a huge strain
on the wisdom and foresight of a pack of mortal programmers thinking
with their own tiny brains at the uttermost dawn of time.
-- Eliezer Yudkowsky Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT