GurpsFriendlyAI

SL4Wiki | RecentChanges | Preferences

by EliezerYudkowsky


Characters in GURPS Friendly AI may learn three new skills, the AI skill (Mental / Hard), the Seed AI skill (Mental / Very Hard), and the Friendly AI skill (Mental / Ridiculously Hard).

AI skill:

An ordinary failure wastes 1d6 years of time and 4d6 hundred thousand dollars. (Non-gamers: 4d6 means "roll four 6-sided dice and add the results".) A critical failure wastes 2d10 years and 2d6 million dollars. An ordinary success results in a successful company. A critical success leads to a roll on the Seed AI skill using AI skill -10, with any ordinary failure on that roll treated as an ordinary success on this roll, and any critical failure treated as an ordinary failure on this roll.

Seed AI skill:

An ordinary failure wastes 2d6 years of time and 8d6 hundred thousand dollars. A critical failure wastes 4d10 years and 4d6 million dollars. If the player has the Friendly AI skill, an ordinary success leads to a roll on the Friendly AI skill, and a critical success grants a +2 bonus on the Friendly AI roll. If the player does not have the Friendly AI skill, an ordinary success automatically destroys the world, and a critical success leads to a roll on the Friendly AI skill using Seed AI skill -10. (Note that if the player has only the AI skill, this roll will be made using AI skill -20!)

Friendly AI skill:

An ordinary success results in a Friendly Singularity. A critical success... ooh, that's tough. An ordinary failure destroys the world. And, of course, a critical failure means that the players roll 3d10 on the

FriendlyAICriticalFailureTable


Expanded rules:

Characters in GURPS Friendly AI may learn three new skills, the AI skill (Mental / Hard, defaults to Computer Programming -6), the Seed AI skill (Mental / Very Hard), and the Friendly AI skill (Mental / Ridiculously Hard).

AI skill:

Seed AI skill:

Friendly AI skill:


Subtleties:

A very high skill increases the chance of a critical success; a very low skill increases the chance of a critical failure. But no matter how good you are, how much you know, or how carefully you plan, a roll of 18 on 3d6 is always a critical failure.

You will always have a greater effective skill on your Seed AI roll than your Friendly AI roll.

Failures such as "The AI turns the solar system into tiny molecularly inscribed pictures of smiling humans because the visual input of smiling humans was its reinforcer", et cetera, are ordinary failures of Friendly AI. A critical failure happens when the Friendship design partially succeeds. Ergo, automatic failures on Friendly AI are never critical failures. You need to actually make a Friendly AI roll in order to get a critical failure, which requires either knowing the Friendly AI skill, or a critical success on a Seed AI roll.

You have to be very unlucky for a roll against the AI skill to destroy the world; of course, there are a lot of people out there rolling their AI skill.

You have to be very unlucky for a roll against the Seed AI skill, alone, to produce a critical failure on the Friendly AI roll.

You have to be extremely unlucky for a roll against the AI skill alone to produce a critical failure on a Friendly AI roll.

If you are rolling on the basis of Seed AI skill or AI skill alone, and you do manage to get to the point of making a Friendly AI roll, your effective skill is so low that you're about equally likely to get a critical failure as an ordinary success. (Mostly you'll just fail.)

Seed AI involves a generalized and deeper understanding of, among other things, the knowledge that would go into developing AI. Friendly AI involves a generalized and deeper understanding of, among other things, the knowledge that would go into developing Seed AI. Thus the Friendly AI skill is harder to learn than the Seed AI skill (Mental / Ridiculously Hard, 8 character points per level), but can be used in place of the Seed AI skill (Mental / Very Hard, 4 character points per level) at +4.

Teaching someone the Friendly AI skill automatically implies teaching them the Seed AI skill.

You can learn the Seed AI skill without even realizing the Friendly AI skill is necessary.

If computing power increases enormously, a highly skilled roll on the AI table, or almost any roll at all by a player who has the Seed AI skill (even if the Seed AI skill is relatively low), has a good chance of producing a success on a Seed AI roll.

The increase of computing power makes no difference whatsoever to Friendly AI rolls.

Teaching someone the Friendly AI skill at a low level can easily be worse than nothing. First, it implies the ability to make Seed AI rolls. Second, it increases the net probability of a Friendly AI critical failure, along with the net probability of a Friendly AI success. Remember that any skill at Friendly AI means that all Seed AI successes, and not just critical successes, result in a Friendly AI roll. If you have a low score then most rolls will be ordinary failures, so that doesn't change, but there's an added chance of either a success or a critical failure. While both will be rare, the critical failures may outweigh the successes, if the skill is at a low level.

You must necessarily learn the Friendly AI skill at a low level before learning it at a high level.

There is no sane reason to teach anyone the Seed AI skill alone. Ever. As the amount of computing power increases, a low level of Seed AI skill becomes absurdly dangerous.

When you look at the total dynamics from the standpoint of planetary safety there is no sane reason to invest character points in any skill except the Friendly AI skill, although, of course, this necessarily means accumulating much of the knowledge that would go into the Seed AI or AI skills.

If you just know the AI skill, the Seed AI skill looks really cool.

If you have the Friendly AI skill at any level, however low, then in terms of technical ability you probably have a decent chance of developing a useful, commercializable ordinary AI technology; but there will still be a chance of failure, it will still take 1d6 years and 4d6 hundred thousand dollars, and you will still need to roll on the Dot-Com table to determine whether the company (as opposed to the technology) succeeds.

With less funding or less people, the chance of success on both Seed AI and Friendly AI rolls goes down. This should not be interpreted as "maintaining the equilibrium"; probably nothing will happen, but if something does happen, a lower percentage of those outcomes will be positive. From another perspective, however, if you think your Friendly AI skill is high enough that you can succeed on both rolls despite the penalty, it is a relatively firm gamble; if your Friendly AI skill is lower than you thought, the Seed AI roll will (probably) fail.

THERE IS NO DEFAULT ON THE FRIENDLY AI SKILL. NOT FROM THE SEED AI SKILL, NOT FROM THE AI SKILL, NADA. YOU MUST ACTUALLY INVEST CHARACTER POINTS SPECIFICALLY IN THE FRIENDLY AI SKILL IN ORDER TO GET THE FRIENDLY AI SKILL. IF YOU DO NOT HAVE THE FRIENDLY AI SKILL, YOU DO NOT GET TO MAKE A FRIENDLY AI SKILL ROLL. IT IS AN AUTOMATIC FAILURE. NOT A HUMOROUS CRITICAL FAILURE. A PLAIN OLD ORDINARY FAILURE. YOU JUST DIE. Why is this so hard to explain to people?


SL4Wiki | RecentChanges | Preferences
Edit text of this page | View other revisions

Last edited April 22, 2006 11:19 pm by cpe-72-224-95-121.nycap.res.rr.com (diff)

Search: