From: Michael Vassar (michaelvassar@hotmail.com)
Date: Wed Jul 20 2005 - 15:31:49 MDT
I agree that no convincing argument has been made that a deceptive proof
could be made, or that a UFAI could exploit holes in our mathematical logic
and present us with a false proof. However,
a) a SI should be able to recognize any systematic cognitive flaws which
would cause us to fail to recognize errors in a proof. If we demand a
sufficiently short proof and there exists a short enough proof this may not
be a problem, but realistically humans are very faulty reasoners and if you
analyzed a particular person's (or committee's) reasoning well enough you
could almost certainly find flaws that would cause analysis to fail.
b (but related)) a long proof that had to be groked fully might open the
possibility of features of the proof hacking the analysts minds.
c) "magic" has to be accounted for. How many things can you do that a dog
would simply NEVER think of? This doesn't have to be "quantum cheat codes".
It could be something as simple as using the electromagnetic fields within
the microchip to trap CO2 molecules in Bose-Einstein condensates and build a
quantum medium for itself and/or use electromagnetic fields to guide
particles into the shape of a controlled assembler or limited assembler. It
could involve using internal electronics to hack local radio traffic. But
it probably involves doing things I haven't thought of.
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT