From: Robin Brandt (mandelum@gmail.com)
Date: Thu Mar 20 2008 - 07:46:47 MDT
Mark,
Just answer this simple question?
-- Why would it be in the self-Interest of a Super-Intelligence to be Friendly unless it already is friendly? -- We all want AI to be friendly. But an AI has only the goals/values/interests that we program into it. We have to code friendliness right there in the core from the beginning. We cant just make a rational economic agent and try to convince it that it is in itīs self-interest to be friendly by applying some circular reasoning. Humans have built in moral circuity, but they donīt work as platonic morality optimizers, they are tweaks that statistically produce more offspring. There is absolutely no rational reason to be friendly unless you have certain values. Unless you value diversity, sentience, complexity, beauty, individuality, subjectivity, preservation of information etc etc... The AI will not be produces by evolution stumbling into pro-social area. It will be built by humans that have. This is both tromendous advantage and a great risk. On Thu, Mar 20, 2008 at 1:19 PM, Nick Tarleton <nickptar@gmail.com> wrote: > On Wed, Mar 19, 2008 at 10:20 PM, Mark Waser <mwaser@cox.net> wrote: > > > Before you talk about defending a planet from a better-armed alien > > > invasion, here's a much simpler scenario: You have $1,000 worth of > > > stuff in your pocket. A robber walks up to you with a gun, and > demands > > > that you hand over the loot. How will your Friendliness theory > > > convince the robber to leave, without shooting you or stealing your > > > stuff? > > > > It probably won't. He won't stick around long enough to be infected. > > Again, this is a strawman. You're inventing a ridiculous scenario > > Ridiculous? This happens in reality ALL THE TIME. > > > and > > demanding that I defend it. I decline to do so unless you can show how > it > > is at all justified or relevant. > > An UFAI need not listen to us long enough to be infected, either! > > So let's give you more time. Someone kidnaps you for ransom, ties you > up in the basement, and for whatever reason lets you talk to them for > a few hours. How do you convince them to be Friendly and let you go? > -- ~Robin Brandt~ Emergence, Evolution, Intelligence Control the infolepsy! Love the world!
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:02 MDT