FW: Group releases "Friendly AI guidelines," Webmind goes bankrupt

From: Patrick McCuller (patrick@kia.net)
Date: Thu Apr 19 2001 - 10:18:03 MDT


        The only problem I have with the story is that Friendly AI isn't exactly
'released'. But it is an entertaining read.

        (Forwarding allowed by restricted license; see below.)

Patrick McCuller

-----Original Message-----
From: owner-politech@politechbot.com
[mailto:owner-politech@politechbot.com]On Behalf Of Declan McCullagh
Sent: Thursday, April 19, 2001 11:29 AM
To: politech@politechbot.com
Subject: FC: Group releases "Friendly AI guidelines," Webmind goes
bankrupt

http://www.wired.com/news/business/0,1367,43158,00.html

   Intelligenesis Faces Dim Future
   By Declan McCullagh (declan@wired.com)
   2:00 a.m. Apr. 19, 2001 PDT

   A pioneering New York company that once hoped to develop the first
   artificial intelligence is preparing to declare bankruptcy.

   Intelligenesis Corp., which was creating the Webmind software, has
   been evicted from its Broadway office suite and plans to file for
   Chapter 7 bankruptcy next week.

   [...]

**********

http://www.wired.com/news/technology/1,1282,43080,00.html

   Making HAL Your Pal
   by Declan McCullagh (declan@wired.com)

   2:00 a.m. Apr. 19, 2001 PDT

   Eliezer Yudkowsky has devoted his young life to an undeniably unusual
   pursuit: planning for what happens when computers become far smarter
   than us.

   Yudkowsky, a 21-year-old researcher at the Singularity Institute, has
   spent the last eight months writing an essay that's half precaution,
   half thought exercise, and entirely in earnest.

   This 750 KB treatise, released Wednesday, is not as much speculative
   as predictive. If a computer becomes sufficiently smart, the argument
   goes, and if it gains the ability to harm humans through
   nanotechnology or some means we don't expect, it may decide it doesn't
   need us or want us around.

   One solution: Unconditional "friendliness," built into the AI as
   surely as our genes are coded into us.

   "I've devoted my life to this," says Yudkowsky, a self-proclaimed
   "genius" who lives in Atlanta and opted out of attending high school
   and college.

   It's not for lack of smarts. He's a skilled, if verbose, writer and an
   avid science-fiction reader who reports he scored 1410 on his SATs,
   not far below the average score for Stanford or MIT students.

   Yudkowsky's reason for shunning formal education is that he believes
   the danger of unfriendly AI to be so near -- as early as tomorrow --
   that there was no time for a traditional adolescence. "If you take the
   Singularity seriously, you tend to live out your life on a shorter
   time scale," he said.

   Mind you, that's "Singularity" in capital letters. Even so-called
   Singularitians like Yudkowsky admit that the term has no precise
   meaning, but a commonly accepted definition is a point when human
   progress, particularly technological progress, accelerates so
   dramatically that predicting what will happen next is futile.

   The term appears to have been coined by John von Neumann, the great
   mathematician and computer scientist who used it not to refer to
   superhuman intelligence, but to the everyday pace of science and
   technology.

   Science-fiction author Vernor Vinge popularized the concept in the
   1980s, capitalizing the word and writing about whether mankind would
   approach Singularity by way of machine intelligence alone or through
   augmented mental processes. Predictions vary wildly about what happens
   at the Singularity, but the consensus seems to be that life as
   humanity currently knows it will come to a sudden end.

   Vinge is the closest thing Singularitians have to a thought leader,
   spokesman and hero. He offers predictions based on measures of
   technological progress such as Moore's Law, and sees the Singularity
   as arriving between 2005 and 2030 -- though some Vinge aficionados
   hope the possibility of uploading their brains into an immortal
   computer is just around the corner.

   One of them is Yudkowsky, who credits Vinge for turning him onto the
   Singularity at age 11. "I read True Names," he said, referring to a
   Vinge novel. "I got to page 47 and found out what I was going to be
   doing for the rest of my life."

   Since then, Yudkowsky has become not just someone who predicts the
   Singularity, but a committed activist trying to speed its arrival. "My
   first allegiance is to the Singularity, not humanity," he writes in
   one essay. "I don't know what the Singularity will do with us. I don't
   know whether Singularities upgrade mortal races, or disassemble us for
   spare atoms.... If it comes down to Us or Them, I'm with Them."

   [...]

   Like a character from science fiction, Yudkowsky sees his efforts as
   humanity's only hope.

   In an autobiographical essay, he writes: "I think my efforts could
   spell the difference between life and death for most of humanity, or
   even the difference between a Singularity and a lifeless, sterilized
   planet... I think that I can save the world, not just because I'm the
   one who happens to be making the effort, but because I'm the only one
   who can make the effort."

   ###

[Clarification: Yudkowsky just emailed me to say he received a 1600 on his
SATs when he took them again. --Declan]

-------------------------------------------------------------------------
POLITECH -- Declan McCullagh's politics and technology mailing list
You may redistribute this message freely if it remains intact.
To subscribe, visit http://www.politechbot.com/info/subscribe.html
This message is archived at http://www.politechbot.com/
-------------------------------------------------------------------------



This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:36 MDT