Re: Problems with AI-boxing

From: Michael Wilson (
Date: Fri Aug 26 2005 - 17:25:37 MDT

Jeff Medina wrote:
> Having the power to let the AI out directly or through a human
> intermediary are functionally indistinguishable scenarios. If a UFAI
> can convince any given human to let it out with nothing but words, it
> can convince any given human to convince any other given human to let
> it out using nothing but words.

I'm not sure that this follows. A human trying to convince another
human only has the benefit of whatever arguments/knowledge/tricks the
AI can give them in advance. They won't have the raw intelligence,
1000x thinking speed, perfect knowledge of human psychology, the
ability to model their opponent down to the neuron level etc etc
that a boxed AI could theoretically have. They may or may not have
other advantages over the AI (e.g. seeing their opponent's face
instead of communicating through a text terminal, or emotional
leverage with their opponent), but barring that the effects of a
radically superhuman AGI would be diluted if a human mind has to
be used as the tool to affect another human mind.

That said the improvement in success probability is marginal at
best and 'AI boxing' remains a stategy with very little hope of

 * Michael Wilson

P.S. To people suggesting perfect virtual worlds and EMP blasts,
please see the archives for a full discussion of all the basics
of so-called 'adversarial methods' for trying to make an FAI.

How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:52 MDT