RE: External reference semantics

From: Philip Sutton (
Date: Thu May 27 2004 - 12:30:53 MDT

Hi Ben

> Who defines the voting scheme across all sentients?

Across the whole universe? Unlikely to be one. So the AGI with the
'external referent' "Your supergoal content is an approximation to [
friendliness as would be maximally acceptable to all sentients
everywhere ] would have to try to infer what get this meta goal best -
through observation, consultation, empathy, tapping evidence of views
expressed via democratic processes and via reactions to dictatorships,
etc. It's a big powerful brain - so I'm sure it will think of other good
ways over time - its probably got plenty of that too.

> The manipulability of democratic systems is pretty clear...

Sure - that's the whole idea (up to a point) - everyone gets a chance to
influence the system. Dictatorships are still open to manipulation - just
that the number of players is dramatically reduced - and the rules are
more opaque.

> Whatabout the potential for tyranny of the majority?

Its possible but usually less dangerous than the tyranny of a minority
(which is oppression in anyone's book and possibly fully blown

Cheers, Philip

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT