From: Carl Shulman (cshulman@fas.harvard.edu)
Date: Wed Jul 13 2005 - 14:58:39 MDT
Faced with the first seed AI, why not ask it for a rigorous and humanly
comprehensible set of principles for designing a FAI, such that the programmers
could prove to their satisfaction that the result would be Friendly? (This would
be done in a physically isolated environment where only a handful of programmers
could interact with the AI.) If the AI denies that provable FAI is possible,
then you have no choice but to shut it down. If it delivers, then those
principles could be used to design a new seed AI from the ground up, without
any code contributed by the first AI. The second AI is released, establishes a
Friendly infrastructure, and the first can then be released as well.
The setup seems (seems!!!) secure unless:
1. The programmers are socially hacked by the first AI so that they release it
before going through the procedure. (Perhaps the AI would argue that the risk
of taking the time to build the second AI is unacceptable.)
2. The AI has magic powers, a.k.a. Eliezer's 'quantum cheat codes.'
Cheers, Carl
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT