Re: A position

From: Eliezer S. Yudkowsky (
Date: Wed May 23 2001 - 17:34:12 MDT

Jimmy Wales wrote:
> "Respecting the wishes of others" can't be an ethical primary,
> as it leads directly to unanswerable questions -- the wishes of
> others will typically be in conflict.

I don't see why you think this makes volition-based Friendliness
impossible as an ethical primary. If you have a quantitative (or
comparative) measure of how well a local event violates or matches
"respecting the wishes of others", then that plus standard intelligence
allows for a good stab at global optimization.

Under volitional Friendliness, your volition only holds sway over the
things you own, including yourself. Making a coercive alteration to
someone else's property or self, no matter how much you want it, is not
moral. This renders volitional Friendliness a strictly local property
almost all of the time, at least as far as I can see. Not that optimizing
an intricate global property *would* present a problem for a transhuman;
we humans flinch away from that sort of thing just because our own
consciousnesses can only track a very small number of objects

-- -- -- -- --
Eliezer S. Yudkowsky
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT