From: Daniel Radetsky (daniel@radray.us)
Date: Mon Aug 22 2005 - 18:15:39 MDT
I decided I would respond at least once to Michael Vassar's objection before
putting it in my essay, just in case he's right. I'm writing this in a
non-conversational tone, partially because I'm practicing for the essay, and
partially because enough time has elapsed since Vassar's response that it no
longer feels like a conversation. I am also writing more verbosely than is
probably necessary to convey my response. This is because I recognize my
general ignorance about many of these topics, and writing verbosely will allow
a reader to diagnose errors of reasoning in my email, if any exist.
Vassar wants to say that despite my objections, we ought worry about exploits
but not ninja hippos because the claim "there are exploits" has a higher prior
probability than "there are ninja hippos." This is because, according to
Vassar, the Kolmogorov complexity of e, the first proposition, is greater than
the complexity of h, the second proposition.
Obviously, e and h are not bitstrings and do not straightforwardly have
complexity. To discuss the complexity of objects in the sense of rocks and
fish, we need some sort of encoding scheme which maps real world objects to
bitstrings. Vassar must hold either:
1. The encoding scheme is a matter of free choice on our part, and every
complexity is complexity-for-scheme-S.
2. There is a universal, correct encoding scheme.
I assume that Vassar holds (2), because (1) is too easy for someone holding my
position to get around. Briefly, I can pick an artificial encoding scheme where
ninja hippos have a simpler representation than exploits.
If Vassar holds (2), then what is the encoding scheme, and what is the basis
for saying that this is THE scheme? I know of only one such basis, and will
assume that it is the only basis here. This basis is the idea that the universe
is fundamentally digital: it consists of a bunch of states that are either in
one fundamental state or another. I have heard this theory repeated by a lot of
SL4 types and would not be surprised if Vassar held it. I will call the theory
the Binary Universe Thesis (BUT). One who asserts the BUT will naturally hold
that any proposition p corresponds to some binary state of affairs (BSA), and
can be expressed as a bitstring with a definite complexity. So e and h are
numbers or sets of numbers (just in case more than one BSA would count as there
being exploits or ninja hippos).
I'm not entirely sure how one gets prior probability from complexity, but I'm
willing to accept that it can be done. We'll say there is a function f such
that for some proposition p with a corresponding BSA n, f(K(n))=P(p)=x, where x
is a real number, K is the complexity of a string, and P is a probability
function (I will sometimes use the same symbol to represent both the BSA and
the proposition, but I don't think this will be confusing. I suspect many of
the proponents of the BUT would assert that the proposition and the BSA are
identical, and so no distinction need be made). Using some technique
resembling the above, Vassar holds that P(e) > P(h), and so we should worry
about e before we worry about h. This is a valid defense against my h-based
counterexample, but it doesn't actually get the job of defending a worry about
exploits done.
The first problem with Vassar's position is a problem for Orthodox Bayesianism
in general. Suppose we want to know the prior probability of p, so we calculate
f(K(p)) and get x. However, to do this, we presuppose BUT. We have not
calculated P(p), but rather P(p|BUT), since we would be wrong to claim the
probability of p is x if BUT were false. However, to find P(p|BUT) we need to
know P(BUT), but we can't find this out by using f(K(BUT)), as this would be
begging the question. So we need to use ordinary, non-formal scientific
know-how to confirm BUT. I am under the impression the BUT is far from strongly
confirmed, but rather is merely another exciting theory. Is this true? How
confident can we be that BUT is the case? If we cannot be quite confident, we
cannot make the kind of claims about prior probability that Vassar needs to
make.
The second problem has to do with the structure of an argument against my
position. If my intuition tells me that P(a) = P(b), and Vassar (or someone
else) wants to defeat my intuition, he can do it either mathematically or
intuitively. No doubt Vassar wishes he could make a mathematical argument that
P(e) != P(h), but this is not possible because to do this Vassar would have
possess knowledge which (as far as I know) he doesn't: the BSAs corresponding
to e and h. So he must defeat my intuition intuitively. Vassar simply claimed
that e was vague and h specific -> K(e) < K(h) -> P(e) > P(h). This is fine,
but it's not clear that m="There is magic" (in the ordinary sense of the word)
or l="there is a lurking horror" or g="there is a god" are more specific than e.
Vassar also points out that even if, for example, g and e are equally vague,
and hence have similar prior probability, it still may not be rational to treat
them the same way. Obviously, we need to worry about God just in case there is
something relevant about doing so. Let g'="God will send you to hell no matter
what." How should we respond to the possibility that g'? We shouldn't, because
nothing we can do will change it. On the other hand g''="God will send you to
hell unless you go to church on sunday" should be responded to by going to
church on sunday iff we are justified in believing g''. In this case, we are
justified to the tune of f(K(g'')). However, the proposition g'''="God will
send you to hell unless you avoid going to church on sunday" tells us to do the
opposite of what g'' tells us. Vassar would claim that f(K(g'')) is equal to
f(K(g''')), and so we can't use our worries about going to hell to decide
whether or not to attend church. The claim is encapsulated by the principle
that we are not justified in worrying about p if there is no evidence for p and
if, given that p will alter the utility of the world if we do X, the complexity
of p remains roughly constant for all X. If we wanted to make this principle
more mathematical, we could require that the distance between the highest and
lowest probability remain below a certain value, with the value perhaps related
to the mean probability or the disutility of the event.
Here's the problem as I see it: I claim that a world which contains exploits
is about as complex as a world which does not (or, There are two possible
worlds w1 and w2 such that both are empirically equivalent to the actual world,
and w1 contains exploits, w2 does not, and K(w1) = K(w2)) (What is the symbol
for "approximately equal to" in text?). Suppose we were to engineer humans
which, for whatever reason, could not be mind-controlled by UFAI. Now we want
to decide whether or not we should box the AI, recognizing that if there are
exploits, we're screwed. Necessarily, we cannot have evidence that there are
exploits, so we consider our complexity-based priors. But since K(w1) = K(w2),
they should have the same prior probability. If w1 were the case, then we
should not box the AI, because if it is going to be friendly it would be a
waste of time and resources to box it, and if it is going to be unfriendly,
boxing won't do any good. But if w2 were the case, then we should box it,
because if the AI is friendly, we'll just have wasted a bit of time and
resources, but if it is unfriendly we've averted disaster. Hence we cannot use
our worry that e to decide between boxing and not boxing, as with the case of
God. Unless Vassar can compellingly argue that K(w1) != K(w2), I don't see how
the complexity argument can move forward.
So, to sum up, arguing that complexity tells us to worry about exploits has two
major problems. It relies on a premise that even more controversial than the
conclusion, and it seems like the logical conclusion of the premises is the
opposite of the intended conclusion.
Daniel
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:52 MDT