From: Jef Allbright (jef@jefallbright.net)
Date: Wed Jan 11 2006 - 21:45:31 MST
On 1/11/06, Eliezer S. Yudkowsky <sentience@pobox.com> wrote:
> Jef Allbright wrote:
> >
> > Sometimes a military leader is faced with the difficult choice of
> > sacrificing some of his troops in order to save the rest. Sometimes
> > an individual will sacrifice himself to allow others to survive in an
> > overloaded lifeboat. Sometimes a surgeon will advise a patient to
> > undergo radical amputation in order to have a chance at life.
> > Sometimes a politician will risk loss of popularity in order to
> > contribute to a greater good.
>
> Sometimes people can be hypnotized by difficult choices.
Eliezer, I commend you on your pursuasive writing techniques (which I
was tempted to annotate), and of course you make a valid point about
the importance, rising to the level of necessity, of striving to
discover a constructive solution to seemingly unsolvable problems.
I could support your point using real-life examples such as the
apparent paradox of the Cold War, or multiple examples from my own
life where I haven't been able to see a way forward, but survived via
perseverance and that abstract principle I've tried to communicate to
you before -- that we're the end result of a long chain of survivors,
and to a surprising extent we can trust in the environment that lead
to us being here even when we don't know what's going on.
But that wasn't the point of my post. The point was about that "moral
repugnance", operating below fully conscious awareness, that people
experience when faced with difficult decisions of the type where a
near-term sacrifice is in fact indicated to achieve broader good.
For the record, I wouldn't agree with a proposal to nuke a population
under any expected near-term conditions, but I could conceive of
hypothetical conditions under which that might be the right thing to
do. I can certainly entertain the thought and consider classes of
similar problems and what general approaches might be effective toward
achieving various goals of varying moral value.
If this topic of conversation ever evolved to a sustained level of
intelligent, respectful discussion, I would hope to discuss ways of
making such decisions rationally, but under constraints of incomplete
knowledge and time. We might talk about heuristics and biases before
moving on, and then I suspect it might become clear that
decision-making based on informed principles, rather then expected end
result, is one of the better ways to proceed in such cases. We might
even talk about what it means to have principles that are "well
informed", and so on...
- Jef
> One recalls
> Elrond, in Tolkien's prehistory to _The Lord of the Rings_, pleading
> with Isildur to throw the Ring into Mount Doom. In the movie version we
> get to see this (and as far as I know, it's faithful to Tolkien):
> Elrond and Isildur actually standing at the Crack of Doom, Isildur
> holding up the Ring, and then...
>
> Elrond: Throw in the Ring!
> Isildur: Nah.
> Elrond: Okay.
>
> So what should Elrond have done? Push Isildur screaming into the Crack
> of Doom? A fine deed that would have been, to set to the credit of the
> Ring... So Elrond let Isildur go, resulting in some untold number of
> casualties in the War of the Ring a few centuries later.
>
> Should we blame Elrond for that? Well, if it was me, I sure would blame
> myself. Just because I have ethics doesn't mean I'm not responsible for
> their consequences.
>
> Plus the Ring killed Isildur anyway.
> And Isildur was lucky. He could have ended up as Gollum.
>
> Elrond had plenty of options besides pushing Isildur into Mount Doom.
> He could have bopped Isildur on the head and then used his sword to
> nudge the Ring off the edge. Worst case scenario, Elrond bops Isildur
> on the head, calls in his lieutenants, strips off his own armor, and
> *volunteers* to be pushed into Mount Doom if he can't manage to nudge
> off the Ring, throw off the Ring, or step off the edge. If Elrond
> wasn't willing to sacrifice himself, he was *obligated* to call for
> volunteers, and if that made him feel awful that was *his* problem.
>
> Elrond was so focused on the obvious wrong way to solve the problem that
> he didn't see the creative right ways. His great failure wasn't that he
> lacked ethics, it was that he didn't know how to use them. He thought
> his ethics were supposed to be heroic disadvantages. If Elrond had just
> taken for *granted* that he couldn't push Isildur off the edge, instead
> of agonizing, he would have seen easier and better solutions.
>
> It won't always be that way. We don't live in so kind a universe. But
> for Elrond it was so, even without Tolkien intending it.
>
> Did anyone else notice this, when they read the book, or watched the movie?
>
> The theory behind the Singularity Institute is that it's possible to
> *save the entire damn world* without killing people, pointing guns at
> people, telling people what to do, or any of the usual bullying
> tribal-chief solutions that instantly pop into people's heads when they
> consider political problems. That's not idealism, it's intelligence.
> History teaches us that the "difficult" choices, the obvious wrong ways
> to solve the problem, DON'T FRICKIN' WORK. Stalin broke plenty of eggs,
> but where are the omelets?
>
> So don't make excuses in advance for ethical failures. People are so
> hypnotized by "difficult" choices that they don't look *hard* for a
> creative solution. They just go straight off and make the "difficult"
> choice. Taking the "difficult" option is not difficult, it's easy and
> convenient. That's why people spend so much time looking for excuses to
> do things the "difficult" way.
>
> So what's really difficult? Thinking. It can be frickin' hard to think
> of a good solution, you've got to, like, actually sit down and
> concentrate. And sometimes, yes, it's painful and inconvenient - for
> *yourself*, not some convenient outside victim who has to be
> "sacrificed" - to do things the right way. It's not always easy. So
> don't make your excuses in advance, or you'll shoot yourself down before
> you start.
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT