From: Peter de Blanc (
Date: Sun Aug 14 2005 - 14:00:28 MDT

I think this is a good time to identify what we mean by responsibility.

In a neural network, we use back-propagation to assign "blame" for
errors to individual neurons. The purpose of this heuristic is to
determine which weights can be changed to reduce this error while
minimizing other disruptions which would be caused by this change. Human
responsibility is a similar concept.

When Sam commits a crime, we do not blame Sam's grandmother. Sam's
grandmother giving birth to Sam's mother is part of the chain of
causality which led to the crime; without her, the crime could not have
occured. So why don't we blame her? We don't blame Sam's mother because
if we universally weaken the heuristics which led Sam to commit the
crime, then fewer crimes will be committed per person, but if we
universally weaken the heuristics which led Sam's grandmother to give
birth to Sam's mother, then all we accomplish is to reduce the world's

Note also that blame should not be assigned solely based on proximity to
the undesired event. If Sam pushes Bob off of a tall building, we do not
blame gravity for Bob's death, even though it was a more immediate cause
than Sam pushing Bob. Even if gravity had no other redeeming
characteristics, we still should not blame it, because our blaming
gravity has no bearing on what gravity will do. If we assign
responsibility to Sam, we can change the heuristics which led him to
push Bob, and fewer people will be pushed off of buildings. If we assign
responsibility to gravity, there's nothing we can do.

In the case of the AI researcher... if a hypothetical AI researcher
creates a Seed AI which destroys humanity, who is responsible? If
society universally blames the hypothetical AI researcher, then maybe
actual AI researchers will listen and watch what they're doing. If
society universally blames the hypothetical Seed AI, then actual Seed
AIs will not care in the least bit.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:51 MDT