From: Stuart Armstrong (dragondreaming@googlemail.com)
Date: Sat Oct 25 2008 - 07:29:01 MDT
> On Fri, Oct 24, 2008 at 9:26 AM, Alex F. Bokov <alexboko@umich.edu> wrote:
>> Actually, I'm not talking about individual volition. I'm talking about
>> self-organizing clusters of coherent extrapolated volition. At the
>> moment, it seems like Eliezer is willing to concede failure of CEV
>> altogether if it fails to converge on one consensus (his Final Judge
>> comments), and other parts of CEV.html. I'm just asking why not
>> empirically determine the optimal number of CEVs, and follow them all?
>
> That would strain the moral ability of the original programmers to
> decide how the different human communities would fit together, how to
> divide up resources, what to do if one piece thinks that everyone is
> allowed to have nuclear weapons and another piece doesn't want to have
> anyone on the borders who can launch nuclear attacks, etc.
I agree with that - if a CEV is meant to be incomprehensible to us,
how are we going to decide the interface between many differing CEV's?
> The whole
> point of this is not that it seems like such a pleasant thing to do
> with absolute power, but that we don't want to put such a huge strain
> on the wisdom and foresight of a pack of mortal programmers thinking
> with their own tiny brains at the uttermost dawn of time.
I fear the strain may be inevitable, in some form or other, and that
the CEV is an invalid shortcut. That said, it's most attractive aspect
is precisely its universality, as you mentioned there; that a bunch of
evil programmers (ie those I don't agree with) couls implement it as
easily as a bunch of "good" one.
Stuart
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT