RE: The Moral Sense Test

From: Durant Schoon (
Date: Tue Sep 02 2003 - 19:52:52 MDT

I just took this test...the test I got was the very recognizable classic
moral paradox about whether to kill one person to save five. I'd heard
it used to combat the stance of pure utilitarian thinking in a class
on moral philosophy (I must have sat in on it in college, because I
don't recall taking an entire course on it, or taking any tests, but
the paradox was very memorable). Basically, extenuating circumstances

A real Friendly AI will have to distinguish all the nuances of these
cases in the same way that we humans do. AT LEAST as well.

> -----Original Message-----
> From: Tyler Emerson []
> Sent: Tuesday, September 02, 2003 10:38 AM
> To: SL4
> Cc:
> Subject: The Moral Sense Test
> The Moral Sense Test
> Sponsored by the Primate Cognitive Neuroscience Laboratory, Harvard
> The Moral Sense Test is a Web-based study into the nature of moral
> intuitions. How do humans, throughout the world, decide what
> is right and
> wrong? To answer this question, we have designed a series of
> moral dilemmas
> designed to probe the psychological mechanisms underlying our ethical
> judgments. By putting these questions on the Web, we hope to
> gain insight
> into the similarities and differences between the moral
> intuitions of people
> of different ages, from different cultures, with different educational
> backgrounds and religious beliefs, involved in different
> occupations and
> exposed to very different circumstances. Participation in the
> study is easy,
> quick and completely confidential.
> * * * *
> Nothing captures human attention more than a moral dilemma.
> Whether we are
> soap opera fanatics or not, we can't help sticking our noses
> in other people
> 's affairs, pronouncing our views on right and wrong, permissible and
> impermissible, justified or not. For hundreds of years,
> scholars have argued
> that our moral judgments arise from rational, conscious, voluntary,
> reflective deliberations about what ought to be. This perspective has
> generated the further belief that our moral psychology is a slowly
> developing capacity, founded entirely on experience and education, and
> subject to considerable variation across cultures. With the
> exception of a
> few trivial examples, one culture's right is another's wrong.
> We believe
> this hyper rational, culturally-specific view is no longer
> tenable. The MST
> has been designed to show why and offer an alternative. Most
> of our moral
> intuitions are unconscious, involuntary, and universal,
> developing in each
> child despite formal education. When humans, from the
> hunter-gathers of the
> Rift Valley to the billionaire dot-com-ers of the Silicon
> Valley generate
> moral intuitions they are like reflexes, something that happens to us
> without our being aware of how or even why. We call this
> capacity our moral
> faculty. Our aim is to use data from the MST, as well as
> other experiments,
> to explain what it is, how it evolved, and how it develops in
> our species,
> creating individuals with moral responsibilities and concerns
> about human
> welfare. The MST has been designed for all humans who are
> curious about that
> puzzling little word "ought" - about the principles that make
> one action
> right and another wrong, and why we feel elated about the
> former and guilty
> about the latter.
> As in every modernly held view, there are significant historical
> antecedents. The origins of our own perspective date back at
> least 300 years
> to the philosopher David Hume and more recently, to the political
> philosopher John Rawls. But unlike these prescient thinkers,
> we can now
> validate the intuitions with significant scientific evidence.
> Over the past
> twenty years, there has been growing evidence for a
> universally shared moral
> faculty based on findings in evolutionary biology, cognitive
> psychology,
> anthropology, economics, linguistics, and neurobiology. This
> evidence has
> created a powerful movement directed at the core aspects of
> human nature. It
> is a movement that has the power to reshape our lives by
> uncovering the deep
> structure of our moral intuitions and showing how they can
> either support or
> conflict with our conscious, often legally supported decisions.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT