From: Michael Roy Ames (michaelroyames@yahoo.com)
Date: Wed Jan 26 2005 - 21:16:53 MST
Daniel Radetsky wrote:
> "In any case, unless I get substantial
> or good responses to this, I will assume
> that Singularitarians grudgingly agree
> with premises (1) and (3). I will also
> assume they agree with (2), because
> nobody has actually disagreed with it
> yet."
Okay Daniel: Perhaps you are not getting clear answers because the
questions (1, 2 and 3) are poorly constructed and you do not have the same
understanding of the problems as the respondents. Maybe I can clarify by
re-phrasing and critiquing the questions.
Question (1) is poorly constructed because it asks about two completely
different things. It should be rewritten as two question, such as:
1a. In developing FAI will some form of Last Judge be used?
1b. In developing FAI will some form of AI Jail be used?
The concepts of Last Judge and AI Jail should be considered separately
because neither relies on the other to be created. Also AI Jail is not well
defined, and people will answer differently depending on their conception of
it. You may want to consider improving question (1a) by providing your
definition of AI Jail.
Question (2) shows that your concept of the Last Judge (LJ) is different
from the originator's idea, in that you expect the LJ to assess friendliness
whereas the originator specified that the LJ would assess an outcome. These
are two very different kinds of assessment. The question could be rewritten
as:
2. Will a Last Judge be used and have sole authority to determine the
acceptability of the outcome of CV?
The short answer to (2) was made by Robin Lee Powell in an earlier post.
Question (3) shows again that your concept of how to check friendliness is
different from that of those involved, in that you ask about friendliness
assessment specifically after the AI starts to improve whereas I doubt that
the originators would alter friendliness assessment procedures at that time.
Good procedures assessing friendliness should work before, during and after
any self-improvement processes. What code auditing would be used on a
self-improving process? is a question on a different level and would have a
substantially different answer. I do not know the answer at this time and
couldn't give a meaningful one until I saw a more detailed technical design.
The questions "Is it friendly" and "Is it doing what we programmed it to do"
are two quite different questions. A friendliness assessment would be
designed to answer the former question, a code audit would answer the
latter.
Michael Roy Ames
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:52 MST