From: Matt Mahoney (matmahoney@yahoo.com)
Date: Fri Oct 16 2009 - 14:38:11 MDT
Pavitra wrote:
> A[ ] Develop a mathematically formal definition of Friendliness.
In order for AI to do what you want (as opposed to what you tell it), it has to at least know what you know, and use that knowledge at least as fast as your brain does. To satisfy conflicts between people (e.g. I want your money), AI has to know what everyone knows. Then it could calculate what an ideal secrecy-free market would do and allocate resources accordingly.
One human knows 10^9 bits (Landauer's estimate of human long term memory). 10^10 humans know 10^17 to 10^18 bits, allowing for some overlapping knowledge.
> A->B[ ] Develop an automated test for Friendliness with a 0% false
> positive rate and a reasonably low false negative rate.
Unlikely. Using an iterative approach, each time that a human gives feedback to the AI (good or bad), one bit of information is added to the model. Development will be slow.
> C[ ] Develop a mathematically formal definition of intelligence.
Legg and Hutter propose to define universal intelligence as the expected reward given a universal (Solomonoff) distribution of environments. http://www.hutter1.net/ai/sior.pdf However it is not computable because the number of environments is infinite. Other definitions are possible of course, e.g. the Turing test.
> C->D[ ] Develop an automated comparison test that returns the more
> intelligent of two given systems.
How? The test giver has to know more than the test taker.
However, you don't need C and D. If you solve B then you already have a model of all human minds, and therefore have already solved intelligence, at least by the Turing test.
> B,D->E[ ] Develop prototype systems and apply these tests to them
> iteratively until the Singularity occurs.
Let's keep in mind that a Singularity is *not* the goal. The goal is friendly AI. The Singularity is what happens when we lose control of it.
-- Matt Mahoney, matmahoney@yahoo.com
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT