Re: Pete & Passive AI

From: Chris Capel (
Date: Thu Dec 08 2005 - 10:54:12 MST

On 12/8/05, Phillip Huggan <> wrote:
> But if PAI spit out
> a course of action like "okay, now you have to let me online, and then all
> kill yourselves", we could blast ver servers with a shotgun. If FAI fails
> because of some fluke error in the philosophy or logic of the goal system we
> give ver, we are steamrolled. If PAI fails the same way, we have a chance
> for human intervention before we finish building the steamroller ve is
> giving us the blueprints to.

Passive AI reduces to the AI boxing problem, plain and simple. If a
superintelligent PAI fails, it's no longer passive. Any constraints we
try to enable to ensure that it remains passive are an AI box, by

Encouraging passivity is a good precaution to take, (and I presume one
that SIAI is planning on taking,) but it doesn't mitigate the pressing
need for Friendliness, not even a little.

Chris Capel

"What is it like to be a bat? What is it like to bat a bee? What is it
like to be a bee being batted? What is it like to be a batted bee?"
-- The Mind's I (Hofstadter, Dennet)

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:54 MDT