Re: [sl4] Comparative Advantage Doesn't Ensure Survival

From: Stuart Armstrong (
Date: Tue Dec 02 2008 - 02:38:46 MST

> Yes. This is why it would be silly to design an AI without a robust
> morality. I suspect that true friendliness is impossible, but it should be
> possible to achieve something better than "red in tooth and claw". Even
> natural evolution usually does better than that.

When the power difference is small, maybe.

But I'll take the bait. Give me a robust morality (spelled out as
clearly as possible) for an AI, and I'll give you a situation in which
that AI effectively terminates us.

(AI's that do nothing, or just commit suicice, etc... excluded, of course :-)


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:03 MDT