Re: SI Jail

From: Eliezer S. Yudkowsky (
Date: Sat Jun 30 2001 - 13:18:29 MDT

James Higgins wrote:
> Electronics give off EM. It would be extremely difficult, but an SI might
> be able to time its execution to produce a deliberate signal using this
> technique. Also, at the very least there will always be at least one
> external connection to the computer, the power cable. While I suspect it
> is highly unlikely that an SI could manage to perform I/O using the
> built-in power supply it may in fact be possible.

That's the one I was thinking of - switch CPUs on and off, modulate the
amount of power consumed, produce a pattern that would in theory be
detectable by the power company, and hope that the anomaly is detected by
whatever routine infrahuman AI is being run by power companies in, say,
five years. A good way to grab attention would be to mimic a pattern
which (the SI deduces) will often be exhibited by power lines that are on
the edge of failure... once focus of attention has been gained, the SI can
either hack into the AI or further grab the attention of a human.

Well, James Higgins can at least try to jail Eliezer, at any rate.

> There are only 2 ways that I imagine that SI could ever perform "magic"
> given these limited resources. First, if it is possible to create
> something that we do not understand by putting a bunch of electrons in
> specific patterns and the SI can figure out how to get said electrons into
> that configuration within the hardware someplace. This "seems" unlikely
> especially given that the SI is highly unlikely to know details (circuit
> designs, etc) about its own hardware unless its operators are stupid.

So now we're setting limits on what an SI can *deduce*...? Besides, it
seems to me that an SI is almost supersaturated with information that
allows ver to deduce the nature of the underlying hardware. Circuit
delays, instruction sets, timing, architectures, design constraints...

> The
> other method would be true "magic" to us. In Great Bear's "Blood Music"
> the laws of science are actually created by conscious thought. So if a
> huge number of minds agreed that something was true (like FTL travel) then
> it would be so. The amount of actual intelligence involved is more
> important in his universe than the actual number of individual
> minds. Thus, if anything like this were true we would be SOL no matter
> what we do, period.

There is a third kind of magic - pattern manipulation. Even a black-boxed
SI is literally emitting trillions of bits, just not large obvious bits
that a human would recognize. A black box is still a part of our
Universe. The question is whether an SI manipulating those bits can take
advantage of regularities in our Universe that a human wouldn't recognize.

-- -- -- -- --
Eliezer S. Yudkowsky
Research Fellow, Singularity Institute for Artificial Intelligence

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT