From: Matt Mahoney (matmahoney@yahoo.com)
Date: Tue Oct 20 2009 - 18:47:58 MDT
Luke wrote:
> (1) With regard to a definition of friendly AI, and how it needs to encompass all those bits, I've got a big problem there. (a) That's impossible. So we either need to find a way around that intractable problem (i.e. a smaller definition, within the 10^5 bits range or something, 10^2 would be great but that's obviously wishful thinking), or we need to accept that we're not going to be able to proceed, and start "saying our prayers" or seeking "enlightenment" or stocking up on heroin or whatever else we need to do to face death. This is a completely serious point: if we decide we cannot hope to produce this friendly AI, it's better to accept that as quickly as possible and decide what we want to do with this short stay between the birth canal and the grave.
For another proposed definition of Friendliness, see http://intelligence.org/upload/CEV.html
This is a much more detailed definition, but it suffers from the same flaw as my simple description. It does not define "human". It's an important question in edge cases such as embryos, animals, and people who are terminally ill and in a coma. It becomes even more important when you consider future human-software hybrids and simulations. There are no "correct" answers to these questions, just opinions based on cultural beliefs. But a complete definition of Friendliness *must* answer these questions.
> (2) You said that a test-giver has to be more intelligent than a test-taker. I don't think that's necessarily the case. For instance, what if the test consisted of: "We're dealing with RSA. Here's an encrypted message, and here's the public key that encrypted it. What is the private key?"
Suppose the AI says "I can't tell you for 10^100 years, because there is no faster algorithm for solving RSA than those already known". Is this the correct answer? You don't know.
In fact there are *no* classes of problems that are known to be hard to solve but easy to check. That's true even if it turns out that P != NP, because you still don't know *which* NP-complete problems are hard, just that some of them are.
-- Matt Mahoney, matmahoney@yahoo.com
________________________________
From: Luke <wlgriffiths@gmail.com>
To: sl4@sl4.org
Sent: Tue, October 20, 2009 2:16:49 PM
Subject: Re: [sl4] to-do list for strong, nice AI
@Pavitra: thanks for reminding me of that. It's true - there's a lot of talking that needs to get done before we can throw out the haiku of FGAI design. I accept this fate, this large-scale discussion, though I can't promise I'll read everything before I respond. Not enough time for that.
@Matt Mahoney: Two points, as follows:
(1) With regard to a definition of friendly AI, and how it needs to encompass all those bits, I've got a big problem there. (a) That's impossible. So we either need to find a way around that intractable problem (i.e. a smaller definition, within the 10^5 bits range or something, 10^2 would be great but that's obviously wishful thinking), or we need to accept that we're not going to be able to proceed, and start "saying our prayers" or seeking "enlightenment" or stocking up on heroin or whatever else we need to do to face death. This is a completely serious point: if we decide we cannot hope to produce this friendly AI, it's better to accept that as quickly as possible and decide what we want to do with this short stay between the birth canal and the grave.
However, as a programmer I'm tempted to point out that often you don't need to see the bits that represent an object, but merely the bits that represent its interface. Let someone else worry about implementation.
(2) You said that a test-giver has to be more intelligent than a test-taker. I don't think that's necessarily the case. For instance, what if the test consisted of: "We're dealing with RSA. Here's an encrypted message, and here's the public key that encrypted it. What is the private key?" It might take massive computational power to "take" that test, i.e. break the code. But it takes orders of magnitude less to both generate the encrypted message, and confirm any answer the test-taker provides. This is quite similar to the problem of theorem-provers mentioned above. Another example of a test could be: "Here's a lab full of standard stock ingredients. Create something that will make me trip. I will give you your grade one hour after you deliver your answer."
As a final point: I'm going to go ahead and put the to-do list up online. I warn I'm going to lean heavily on real-world applicability, so we might see a constant resonance between mathematical definitions and what I consider "executable" actions. I'll be putting up steps like "raise $20,000,000 to fund research" and "create a computer with 700 TF to perform tests" and the like. Others can focus on the mathematical rigor necessary at various junctures. Defining waypoints as mathematical objects, and the interconnecting strategies as meatspace man-hours, may be our best bet.
- Luke
On Tue, Oct 20, 2009 at 11:22 AM, Matt Mahoney <matmahoney@yahoo.com> wrote:
Luke wrote:
>> Alright, it is no wonder you guys can't get anything done. I start a single thread, with a single, simple purpose: to trade versions of a single document: the to-do list. And you all can't resist the urge to get into the most arcane, esoteric mathematical bullshit imaginable. "Degree of compressibility". "Test giver must have more information than test-taker". wank wank wank.
>
>
>Because your checklist is wrong. Specifically, the first 3 steps are wrong. This invalidates the last 2 steps that depend on them. To quote:
>>>>
>This document implies dependencies only insofar as each step's dependencies should appear before that step. Note that other sequential orderings are possible while maintaining this constraint.
>
>
>[ ] Compile design requirements for "friendly AI". When will we know we have succeeded?
>
>[ ] Develop automated tests which will determine whether a given system is friendly to humans or not
>[ ] Develop automated tests which will determine whether a given system is intelligent or not (IQ, whatever)
> (these tests should reflect the requirements laid out in the first step: "compile design requirements")
>
>[ ] Develop prototype systems and apply these tests to them. Refactor tests as necessary in the case we find that some requirement is not specified in the tests.
>
>[ ] Continue refactoring prototypes until we have a system which passes both the intelligence tests and friendliness tests.
><<<1. The definition of "Friendly AI" has an algorithmic complexity of 10^17 bits. Roughly, it means to do what people want, with conflicts resolved as an ideal secrecy-free market would resolve them. So your definition has to describe what 10^10 people want, and how much they want it, which means your definition must describe what they know, and each person knows about 10^7 bits that nobody else knows. My definition is cheating, of course, because I am pointing to human brains instead of describing what they contain.
>
>
>Also, I haven't defined "people". Does it include embryos, animals, slaves, women, and illegal immigrants? (Don't give me an answer that depends on your cultural beliefs). Does it include future
> human-animal-robot-software hybrids? Do all people have equal rights or do we weight rights by how much money you have like in a real market?
>
>
>2. You can't test for friendliness unless you already know that the tester is friendly. How do you know it isn't lying?
>
>
>3. You can't test for intelligence unless you are smarter than the test taker. Otherwise, how do you know that it is giving the right answers?
>
>
>So the result is we will find another way to build AI. There is a US$1 quadrillion incentive to get it done. That's the value of global human labor divided by market interest rates.
>
>
>Just in case you haven't noticed, the internet is getting smarter.
>
>-- Matt Mahoney, matmahoney@yahoo.com
>
>
This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:01:05 MDT