From: Eliezer S. Yudkowsky (firstname.lastname@example.org)
Date: Wed Jun 26 2002 - 07:06:51 MDT
Eugen Leitl wrote:
> Coupling, as interacting. A runaway Singularity machine operating down
> here is not a colibri. I understand the whole point of building an
> ethics-aware Power is not that it tiptoes out of everybody's way. Orelse
> why build it?
> Warping as changing the course of evolution. I've described why this must
> occur regardless of whether moral evaluation is completely externalized,
> or completely internal. The Power asserts the delta between world's and
> own metric is small at each iteration, but this doesn't constrain longterm
> drift. Since it's not a passive player, it drives the drift. Longterm
> results are undecidedable.
> What I meant with the above is that it doesn't matter. An iron fist in a
> velvet glove is still an iron fist. All the problems are intrinsic to the
> power gradient, which the system must maintain to keep ahead of subject
Eugen, please use smaller words or define the terms above. Or as you would
"Sentences too terse. The explanation creates imagery but does not
constrain it. Binding to the model in the original agent is possible but
not guaranteed; internal forces may supervene, causing the overall course of
system development to be internally reflective rather than externally
indexive. A finer-grained communications model is needed as a fix."
-- Eliezer S. Yudkowsky http://singinst.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Fri May 24 2013 - 04:00:21 MDT