From: Bill Hibbard (test@doll.ssec.wisc.edu)
Date: Mon Dec 09 2002 - 11:11:28 MST
On Mon, 9 Dec 2002 polysync@pobox.com wrote:
> > As social animals humans have some more altruistic values, but these mostly
> > depend on social pressure.
>
> Implying we didn't create these social pressures on our own?
Stephen Pinker does a good job of analyzing human altruistic
values in How the Mind Works. Humans evolved as social animals
and thus have some respect for social pressure (those who don't
often end up in prison). Of course, individuals often exploit
social pressure to further their own selfish values. So I guess
the source of social pressure is a mixture of evolution and
individual actions.
> > the solution is to make alteration of reinforcement learning values a
> > necessary condition for granting a human super-intelligence.
>
> I don't see this flying. I certainly wouldn't accept it. Maybe I will when
> I get to 'almost-super' intelligence?
I think regulation of the values of super-intelligent brains,
whether artificial or human, will be a subject of political
debate. Wealthy institutions and individuals will have obvious
motives for creating super-intelligent brains with values other
than universal human happiness.
> > Given that the combination of super-intelligence and human values is
> > dangerous
>
> Do you think right now, with an unaltered learning system, the average person
> has the _capacity_ to handle super-intelligence without being destructive? I'm
> not asking if you think it's likely or not, just if the average person were
> raised in the perfect environment, perfect parents, perfect culture, perfect
> schools, etc etc, could they handle super intelligence? And would a super
> intelligent society be able to create that perfect cultural environment? That
> is, would it all be self stabilizing? Or do you think the brain has
> fundamental, insurmountable, physical flaws as it stands now? I'm not saying
> improvements wouldn't help, I'm just objecting to the prospect of more
> hardwired CONSTRAINTS on how and what we think.
Even the most altruistic mind cannot escape being destructive to
someone's interests, in this world full of conflicting interests.
But the rest of us are toast in a world inhabited by super-
intelligent brains acting against our interests. The constraints
on the reinforcement learning values of super-intelligent brains
are like many other laws: a social contract to protect people (and
animals) from the actions of other people. I feel that I have
the capacity to safely drive 50 mph on city streets, but respect
the need for speeding laws to protect me from all the other nuts
on the road.
Cheers,
Bill
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:41 MDT