Re: Domain Protection

From: Russell Wallace (
Date: Tue May 10 2005 - 02:23:02 MDT

On 5/10/05, Sebastian Hagen <> wrote:
> The last judge is more of a final check to protect against unexpected
> failures; it shouldn't be necessary to avoid likely failure modes. Why
> do you think that this specific one is sufficiently likely to warrant
> specifically worrying about it?

Now I've cited several more that are also sufficiently likely to
warrant specifically worrying about, if that answers your question? :)

> Either 'sentient life' matters to humanity in general, in which case I'd
> expect the Extrapolated Collective Volition of humanity to ensure that
> it continues to exist (or fail harmlessly in case of nonconvergence,
> computational intractability, etc)

Or humanity cares about the continued existence of sentient life and
takes decisions it _thinks_ will ensure that; and the RPOP flashes a
picture on the screen that lets the Last Judge convince himself it
will do so; and it turns out otherwise.

> or it doesn't, in which case I don't
> have any particular reason to care if it is optimized away.

So you don't personally care if everything of value is destroyed?

> The conventional assumption in this case is that unupgraded humans, even
> with some predictive help from a RPOP, are not capable of reliably
> making good decisions about rules that are supposed to stay in force
> forever.

Conventional assumption? What convention would this be?

> This applies to the rules for individual domains, but it also
> applies to the whole idea of domain protection.

If you have a better idea, it's not too late.

> If there are critical differences in the values of different subsets of
> the population the CV still has the option implement an individual or
> domain protection system with permanently fixed rules; it would be in a
> much better position to make that decision than present-day humans.

[1] Is there an emoticon for humor of the "Nah, I was only having you
on, it's crucifixion really!" [2] variety?

[2] Monty Python, 'The Life of Brian'. (Do not die without having seen
this movie.)

- Russell

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT