From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Sun Dec 09 2001 - 07:37:46 MST
Jeff Bone wrote:
>
> "science moves forward, therefore our extrapolation of the next
> billion millennia changes every fifty years".
>
> The question isn't one of whether it changes; we agree that it does. The question is how
> *much* it changes, i.e. the accuracy of predictions from current understanding vs. later
> understanding.
The problem is that ultimate physical limits are very sensitive to the
eighteenth decimal place. Apples didn't stop falling when Newton
discovered gravity, and planets didn't stop orbiting when Einstein
discovered General Relativity, but in both cases our conception of the
ultimate physical limits changed drastically.
> > This observation, I rather like. The problem, as my humorous post
> > suggests, is workarounds where the physicists stand around innocently
> > saying "Violation? What violation?"
>
> Yes, but until you demonstrate at least *some* likelihood of some non/violation, you can't
> just argue from your gut that such are possible. At least, not if we're having a rational
> discussion.
Alcubierre warp is "some likelihood" of (non)violation. So is Van Den
Broeck warp. So are wormholes, and Tipler cylinders. The point is not
that these specific methods are likely to be workable, but rather that the
fact that modern-day physicists can even attempt to produce this plethora
of workarounds should demonstrate that the character of the laws
themselves may not be so absolutely mathematically inviolable as commonly
thought; or at least such inviolability is not as obvious as commonly
thought.
> Granted. And hence, I'm less inclined to believe such "laws" that are phenomenological ---
> like the inviolability of c --- vs. things that are more abstract in both their construction
> and their application, such as Godel's, 2LT, etc. The latter transcend their
> phenomenological formalizations and speak of deep, formal, mathematical, epistemological
> concerns. As such, they are accessible to the tools of mathematics, logic, and reason ---
> and experiments related to them can be conducted in the mind, mostly independent of any
> given technological constraints or capabilities. Advancing technology may increase the
> accuracy of our predictions using such tools, but it is unlikely to overthrow the
> fundamental concepts and relations.
Aha, now I see the conflict! You are a Platonist! I trust C because C is
the universe's belief about causality. I don't trust human conceptions
about "logic" because past experience has shown that the universe often
defies our intuitive conception of logic. Shades of "Jupiter's moons are
not visible to the naked eye, and therefore can have no influence on the
earth, and therefore would be useless, and therefore do not exist", as the
Aristotelians said to Galileo.
> > For that matter, something which appears possible under our own
> > laws of physics could turn out to be impossible under the real laws of
> > physics!
>
> Aha, now I see the conflict! You are a Platonist! ;-) What, exactly, pray tell, are the
> "real" laws of physics? Where are they kept? ;-) Remember: the "laws of physics" as we
> understand them now, and as we will ever understand them at any given time, are only an
> instantaneous snapshot of a monotonically improving model.
A monotonically improving model of *what*?
Given that my own sentience is a product of the laws of physics, and was a
product of the laws of physics long before humanity ever started coughing
up physicists, I don't need to be a Platonist to assert the existence of a
physics that predates humanity. Real effects are the product only of real
causes.
The sun shone for billions of years before we discovered fusion, or so the
current evidence indicates; I don't believe that this evidence was
retrospectively invented by the universe to cover its tracks when we
discovered fusion. And even if the evidence was retrospectively invented,
there would be still higher laws, governing the rules by which universes
were brought into correspondence with explanations.
> But the only arguments you've given for long-term probability of disproof are
> methodologically inconsistent with the arguments you've bought into elsewhere to weight the
> probability of Singularity, as illustrated above!
Only if you think that "Moore's Law is likely to continue for the next ten
years" should be analogized to "no effective workaround to the second law
of thermodynamics will be discovered over the next ten billion years".
> > If, leaving workarounds aside, I had to pick one of these rules as *most*
> > likely to survive, I'd pick c - it seems to be built into the nature of
> > causality in our universe.
>
> Actually, that's the one I'd rate as least likely to survive. 2LT is *much* more intimitely
> involved in causality --- "c+1 only violates causality" only holds true if many other things
> hold true, while "systems tend from order to disorder" (pardon the simplification) *by
> itself* explains one of the most puzzling assymetries in physics ("The Arrow of Time")
> without introducing an observer bias.
Yep, 180-degree opposition here... when I look at C, I hear the universe
speaking, and it says: "There is no such thing as simultaneity; the speed
of light isn't just a speed limit, it's built into the nature of
causality." When I look at the second law of thermodynamics, I see an
emergent statistical phenomenon, an effect of the laws of physics, and not
a structuring cause of the laws of physics. Perhaps I am wrong. But
unless you have a physics degree you've been concealing, and I'm pretty
sure you would have mentioned it by now, both of us are simply listing
which laws of physics we like and dislike based on their character...
rather a silly activity, really.
> Here's a gedankenexperiment. If we were to decide between the two following scenarios,
> which would we choose: a Friendly SI and successors that or capable of carrying the
> entirety of pre-Singularity humanity for the next 10^20 years in idyllic bliss but unable to
> escape some eventual catastrophe (due to resource allocations on achieving individual
> satisfaction vs. other considerations which ensure longevity and security) or a Friendly SI
> and successors that are capable of eventually building a post-Human society that experiences
> subjective infinite time, at the cost of some (perhaps large) amount of involuntary pain,
> suffering, or termination for some (perhaps large) portion of immediately pre-Singularity
> humanity? I don't know which is preferable, and I have no bias one way or the other ---
> though I think the conversation is definitely worth having. ? I indeed believe that the
> choices we make now can impact such things; the "moral" bias we provide to the first AIs is
> very likely to determine which outcomes are possible.
If it's a decision predicated on the truth of physical law, then what a
Friendly AI requires is the ability to make the correct decision based on
the truth as known to it at that time. If it's a decision not predicated
on the truth of physical law, then this conversation is entirely
irrelevant to Friendly AI. Even if you argue that our current model of
physics will affect how we now make moral decisions that establish basic
values (supergoals) which are then not dependent on physics, a Friendly AI
with causal validity semantics would probably re-model the moral decision
we would have made at this point as if we had had accurate knowledge of
physics.
> I'm not accusing you of that. I'm merely pointing out that it's very hard for humans to
> accurately assess their priors and reason about things in a manner divorced from sentiment.
I agree, it's very hard. But you have provided no evidence supporting the
assertion that I am guilty of this flaw in any of the specific cases in
question; *first* you must establish this, *then* you may sigh over the
fallibility of humans in general and me in particular.
> > As far as I can tell, my sole crime is
> > that I attach a 20% probability to irrelevant-but-fun hypotheses to which
> > you would rather grant a 90% probability.
>
> ??? Which ones?
The eternal (not just temporary, for-us-at-the-moment) endurance of the
second law of thermodynamics, for example.
> > Yes, well, I have some experience with the bizarre matrix of
> > self-reinforcing misinterpretations that usually results in such a
> > statement. You don't appear to be an advanced case, and hopefully can be
> > extracted from whatever corner you're currently wedging yourself into.
>
> And how exactly is that not an ad hominem? ;-)
It is an ad hominem. Just not an arbitrarily chosen, one-size-fits-all ad
hominem. The phenomenon I'm referring to is the one where somebody
says... "Hm... he thinks Y, which I don't like... he must believe this
because he's an X. And if he's an X, he must also believe silly ideas A,
B, and C. Boy, ABC sure are silly! That sure confirms he's an X! What a
silly guy!" Meanwhile, I'm standing in some totally other corner of state
space, metaphorically waving my arms and shouting: "I'm over here!"
Asked for evidence for any of your assertions about flaws in my thinking,
you point not to things I have said, but to things you think I might say,
generally with some statement along the lines of "You've said Y, which I
suppose is logical, but Y is only a short distance from X, which is
unacceptable..."
-- -- -- -- --
Eliezer S. Yudkowsky http://intelligence.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:37 MDT