Re: Universalising an AGI's duty of care

From: Joel Pitt (
Date: Mon Jul 18 2005 - 19:45:19 MDT


I think that these golden rules are not something I want to have alien
species use in judging how they treat me. For instance - a species may
believe in life sacrifice and that it is the highest honour, however
unless I believe in their spirituality I don't want an FAI deeming it
okay because it fits these golden rules (presumably any of the aliens
would volunteer and *want* to be the sacrificial individual). Those that
have read the Ender's saga by Orson Scott Card may recognise this scenario.

When other sentient beings actions *intentionally* put the primary
interests (i.e. existance) of other individuals at risk, then they
should be prevented from doing so.


Philip Sutton wrote:
> I don't think it's a coincidence that golden rules are common in the most
> widespread human philosophies eg. "do unto other as you would have
> them do unto you" (golden rule 1) or "don't do unto others what they would
> have you not do unto them" (golden rule 2). Things seem to work
> reasonably well when diverse cultures have contacts that are governed by
> these sorts of rules. I imagine it wouldn't be impossible to implement these
> ideas in any galaxy where there are sentients clever enough to create AGIs
> exist. Golden rule 1 can guide action even if you know nothing about the
> other sentients you are contacting (which means it is not a fail safe rule).
> Golden Rule 2 means you need to build understanding of any sentients you
> contact before taking action that could have a potential to violate the rule.
> Golden rule 2 would require an attitude of forberance, and patience and of
> careful learning.
> By the way, the goldern rules were invented to inject a bit of friendliness into
> natural general intelligences where contact between non-kin or out-groups
> occurred - especially in larger rural and early urban communities.
> My guess is that AGIs on earth that adhered to both these rules would have
> at least a basic level of friendliness.
> Cheers, Philip

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT