RE: Universalising an AGI's duty of care

From: Phillip Huggan (
Date: Mon Jul 18 2005 - 14:48:18 MDT

H C <> wrote:
Basically what you are reffering to with Golden Rule #1 and #2 is, assuming
a sucessful cleanly Friendly AGI (with Friendliness external refernce
semantics), a speculation on the Friendliness content of the AI's
"supergoal" of Friendliness.

Ultimately since it's Friendliness content is going to be orinating upon the
basis of human desires, I still argue that neither humans nor the FAI will
want to arbitrarily infringe upon the rights of the alien race unless it
becomes necessary.

And if you don't assume a 'cleanly' FAI... well then we are probably
screwed, let alone the aliens.
-- Th3Hegem0n
  Seems to be an issue of justice and playing god. At some point a UFAI would gain access to powerful technologies an FAI would be helpless to stop. Possibly the UFAI's influence would extend outwards at the speed of light and envelope life on emerging planets, but more likely it could build time machines and monopolize its future lightcone or even find a natural time-machine to monopolize the resources of a significant volume of the multiverse which might otherwise give rise to sentience. I think we need a better TOE to determine at what specific tech level it becomes impossible to safeguard against a civilization gone wrong, but just before MM, AGI or something more exotic, a "lighning bolt from the sky" may be called for. After FAI makes its presence known, ETs should be given every opportunity for education and judged as individuals accordingly (they can gain membership to the galactic club). Previous to this I hope some method of unobtrusively catching ETs somewhere near
 their deaths and judging their lives by the choices available to them, can be developed. Regardless, any interference by AGI before ET's tech level gets dangerous, is an argument for our own AGI to steamroll us in the near future with computronium. We as individuals might find something better than computronium if not steamrolled. And if AGI can't find a way to resurrect dying ETs, the ET civilization itself (or us in parallel universes which fall under AGI's causal control) may find a way to do this or make other multiverse improvements that AGI has missed, if we give the ET planet some breathing room.

Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT