Re: guaranteeing friendliness

From: Jeff Medina (
Date: Sun Dec 04 2005 - 02:17:52 MST

On 12/4/05, Jef Allbright <> wrote:
> I will suggest, as I have approximately annually, that you
> will find that there is no guaranteed *solution* to the problem of
> friendliness, but that there is an optimum *approach*,

I'm not sure how it might be said Eli would disagree with this. He may
well disagree with your proposal concerning the nature of that optimal
approach, but not with the limitation of having only optimal, and not
guaranteed, solutions to any given problem.

Bayesian epistemology requires one never to assign zero probability to
anything. Hence, where "guaranteed solution" implies certainty
(P(successful solution)=1), there are no guaranteed solutions for the
methodologically Bayesian. And should "guaranteed solution" refer to
something less than complete certainty, one might reasonably wonder
what difference there could be between a "guaranteed" solution, on
this construal, and an optimal one. The use of language like
"guaranteed" or "formally proven" or "verifiable" in discussions of
FAI do not indicate guaranteed (i.e., certain, P=1), contra 'merely'
optimal, approaches, as there are possibilities of failure
ineradicable even in principle let alone practice. For example, a
formal proof may fail due to human error in the construction of the
proof itself, no matter how many times one n-tuple-checks one's work,
or inapplicability of the chosen formal system to the problem at hand,
because, say, the formal system doesn't accurately map/model reality
in ways one has not foreseen

Jeff Medina
Community Director
Singularity Institute for Artificial Intelligence
Relationships & Community Fellow
Institute for Ethics & Emerging Technologies
School of Philosophy, Birkbeck, University of London

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT