THanks for the reply James, I reply to a couple of the points below.. --- James Higgins wrote: >Justin Corwin wrote: >> On a related note, I believe this experiment can be generalized to most >> humans, and should be seen as applicable even to highly intelligent and >> prepared individuals, as some of these people were, and I think this >> illustrates some universal principles. I would like to clarify my belief here. I did not mean to imply that the pattern of conversation, or the method of convincing could be generalized to most humans. The problem with humans as voluntary jailers is that most human have flaws or 'hooks' in their thinking. Even thinking that they've mulled over and formalized. An AI(or in this case, me roleplaying an AI) can seek out and grab those hooks in whatever kind of conversaiton you have with it, and exploit them. The most convincing kinds of hooks can be used against each person, such as their logical flaws in religion, or philosophy, or even AI perspective. It is this process that I believe can be generalized from this series of experiments. Humans are too bad at thinking to accurately defend against a determined and rational opponnent, particularly an unFriendly, lying, transhuman one. And another point I think got lost here, but that the original aim was not to show how cool I am, or how many convinces I can bag, but that the AI Box theory is not sound. Humans cannot determine accurately how Friendly or unFriendly an AI is. Humans can, as it was demonstrated in my experiment, leave an AI in jail just by not listening, or not interacting, but that defeats the whole stated purpose of the AI Box, which is a testing ground for the AI to be released. > >I agree that this experiment could be generalized to most humans, but I >do not believe it is applicable to highly intelligent & prepared >individuals. You said "only 2 were transhumanists" and "1 could be >considered a nascent Singularitarian". Therefor your sample may be >indicitive of the general population but not of transhumanists, >singularitians or AI researchers (the most important). Most of the participants were quite intelligent. I would say none much over 3 sigmas above the mean, but very intelligent on the whole compared to the base population. Also, these people realized what they were getting into. They took the experiment seriously, and reacted seriously. In fact, some of the hardest convinces I had were from people far removed from the transhumanist arena. One lady actually wanted to destroy the AI, not just leave it in jail, when i finally convinced her to let ve out, she did so, then realized what she had done, burst into tears, and demanded I destroy the log of her doing so. On a less emotional note, as I noted earlier, 22 of the involved claimed beforehand that nothing could make them let the AI out. 4 said they would evaluate the AI in the interview and decide for themselves. None were predisposed to simply let it out. > >The vast majority of the population is not equiped for such a challenge. > To make an intelligent decision in this one would, at least, need to >understand the concepts of NanoTechnology & The Singularity. If they >don't truly understand, or believe in, the potential threat they may >arbitrarily release the AI. Even if NanoTech & The Singularity were >impossible, a transhuman AI could still wreak havoc on the Internet. >You don't even qualify the level of Computer Science knowledge of most >participants (prefering to specify their religous beliefs in detail, >which (IMHO) is far less important). As I mentioned in my first email, I qualified the dangers and relevant technologies for these people, telling them why the AI was boxed, and the kinds of things it might be able to do. All of the participants communicated via IRC, none were newcomers to computers. 10 worked in Information Technology. 2 did not believe AI was possible, but accepted it for the purposes of the interview. All had some time to prepare for the interview, (in an interesting twist one was doing research on AI, came across Yudkowsky's site, and noticed the AI-boxing protocol) I quantify the religious beliefs in detail becuase that was the factor that overridingly decided how long of a convince this would be. In general, it came down to moral beliefs. So the religion or beliefs of people tended to decide in general how people went, what they wanted to talk about, and how they were convinced. I think within a single person Computer Science Knowledge could be the deciding factor, rather than religious beliefs, but that has not been my experience. > >You also don't specify the conviction of the participants. Were they >just playing along because they thought it was fun or had some excess >time? Did they take this seriously? I'm sure most did not take it as seriously as I did, but most tried their best to be objective, 14 did research before hand(of what level I did not inquire) Most had strong feelings on the subject. I would also like to make some things clear. In many cases you seem to refer to these people as if they're of no consequence, or inapplicable to the problem. These were intelligent, vibrant people who think things through, know about lots of things, and care a great deal. They're internet people (else how would I find them) and they were the most interested ones I could find. Don't let my quantifying the differences fool you, they were on the whole, more alike than unalike. and the picture is one of above average intelligence, interest in the issues at hand, and computer savvy, if not proffessional. But as the saying goes, sheep have no trouble telling each other apart. And when I say only 2 were transhumanist, I mean REAL transhumanists, with the desire to transcend themselves, upwing beliefs, rational objectors. Hell, most of the people I interviewed would be more than comfortable on the Extropians list, or Transhumantech. They're just not yet ready to go all the way. (or at least as far as I have, i hesitate to call that all the way, even if ELiezer doesn't see anything past sl4.) > >As for Eliezer's rules I do agree that the 2 hour minimum is not >realistic. I saw Eliezer's rule to minimum at 2 hours as a hack to try and approximate transhuman arguing ability, but I felt the experiment would be more meaningful without it. I suppose it might have changed my mind if more thatn one person had just 'hung up' on me. > >James Higgins Interesting stuff james, Thanks for the reply, again. Justin Corwin outlawpoet@hell.com "One more time....Celebrate...One more time." ~Daft Punk