From: Eliezer S. Yudkowsky (sentience@pobox.com)
Date: Wed Jan 11 2006 - 18:52:00 MST
Jef Allbright wrote:
>
> Sometimes a military leader is faced with the difficult choice of
> sacrificing some of his troops in order to save the rest. Sometimes
> an individual will sacrifice himself to allow others to survive in an
> overloaded lifeboat. Sometimes a surgeon will advise a patient to
> undergo radical amputation in order to have a chance at life.
> Sometimes a politician will risk loss of popularity in order to
> contribute to a greater good.
Sometimes people can be hypnotized by difficult choices. One recalls
Elrond, in Tolkien's prehistory to _The Lord of the Rings_, pleading
with Isildur to throw the Ring into Mount Doom. In the movie version we
get to see this (and as far as I know, it's faithful to Tolkien):
Elrond and Isildur actually standing at the Crack of Doom, Isildur
holding up the Ring, and then...
Elrond: Throw in the Ring!
Isildur: Nah.
Elrond: Okay.
So what should Elrond have done? Push Isildur screaming into the Crack
of Doom? A fine deed that would have been, to set to the credit of the
Ring... So Elrond let Isildur go, resulting in some untold number of
casualties in the War of the Ring a few centuries later.
Should we blame Elrond for that? Well, if it was me, I sure would blame
myself. Just because I have ethics doesn't mean I'm not responsible for
their consequences.
Plus the Ring killed Isildur anyway.
And Isildur was lucky. He could have ended up as Gollum.
Elrond had plenty of options besides pushing Isildur into Mount Doom.
He could have bopped Isildur on the head and then used his sword to
nudge the Ring off the edge. Worst case scenario, Elrond bops Isildur
on the head, calls in his lieutenants, strips off his own armor, and
*volunteers* to be pushed into Mount Doom if he can't manage to nudge
off the Ring, throw off the Ring, or step off the edge. If Elrond
wasn't willing to sacrifice himself, he was *obligated* to call for
volunteers, and if that made him feel awful that was *his* problem.
Elrond was so focused on the obvious wrong way to solve the problem that
he didn't see the creative right ways. His great failure wasn't that he
lacked ethics, it was that he didn't know how to use them. He thought
his ethics were supposed to be heroic disadvantages. If Elrond had just
taken for *granted* that he couldn't push Isildur off the edge, instead
of agonizing, he would have seen easier and better solutions.
It won't always be that way. We don't live in so kind a universe. But
for Elrond it was so, even without Tolkien intending it.
Did anyone else notice this, when they read the book, or watched the movie?
The theory behind the Singularity Institute is that it's possible to
*save the entire damn world* without killing people, pointing guns at
people, telling people what to do, or any of the usual bullying
tribal-chief solutions that instantly pop into people's heads when they
consider political problems. That's not idealism, it's intelligence.
History teaches us that the "difficult" choices, the obvious wrong ways
to solve the problem, DON'T FRICKIN' WORK. Stalin broke plenty of eggs,
but where are the omelets?
So don't make excuses in advance for ethical failures. People are so
hypnotized by "difficult" choices that they don't look *hard* for a
creative solution. They just go straight off and make the "difficult"
choice. Taking the "difficult" option is not difficult, it's easy and
convenient. That's why people spend so much time looking for excuses to
do things the "difficult" way.
So what's really difficult? Thinking. It can be frickin' hard to think
of a good solution, you've got to, like, actually sit down and
concentrate. And sometimes, yes, it's painful and inconvenient - for
*yourself*, not some convenient outside victim who has to be
"sacrificed" - to do things the right way. It's not always easy. So
don't make your excuses in advance, or you'll shoot yourself down before
you start.
-- Eliezer S. Yudkowsky http://intelligence.org/ Research Fellow, Singularity Institute for Artificial Intelligence
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:55 MDT