Re: supergoal stability

From: Wei Dai (
Date: Sat May 04 2002 - 23:52:30 MDT

On Sat, May 04, 2002 at 09:41:25PM -0400, Eliezer S. Yudkowsky wrote:
> That there are many different incompatible systems of philosophy does not
> imply that all, or even any, of those philosophies are self-consistent.

I didn't say that any of them are. I'm trying to figure out if yours is.

> But of course this is a highly optimistic view; on the other hand, mana plus
> Minimum Living Space seems like a straightforward way of settling any
> conflict, not just insignificant ones.

What about new minds born post-Singularity? Do they get an equal share of
mana? What if ten new minds are created and then get merged into one. Do
they get ten shares or one? What about copies of minds? Also, how does it
deal with non-renewable resources, e.g., negentropy?

>From your previous answer it seems as if only minds present *at*
Singularity will get a share of mana, and minds born later will have to
fend for themselves. But that policy isn't symmetrical with respect to
time (what's so special about that moment in time?), or with respect to
the space of all possible minds.

> I believe that private property is a correct answer in this case because it
> seems to be the bedrock for determining whether a solution like communism or
> democracy is fair. Suppose that I believe in private property, you believe
> in democracy, and Damien Broderick believes in communism. Assuming we're
> all still that human at this point, there's no reason why you and 2 billion
> people can't choose to live in a democracy, while Broderick and 2 billion
> others choose to live in a communism, and I and 2 billion others choose to
> live in a private-property system.

Suppose there's a project that will use 20% of all resources in the solar
system, but will greatly benefit almost everyone. Would it be fair for the
democracy to tax itself and build the project while everyone enjoys the
benefits? What if only 1 billion join the democracy? Does the project
not get built then?

> It seems simpler for conserved resources
> go 1/3 to each of the three systems than that nobody gets any conserved
> resources (the communist solution; incidentally, how the heck does anyone
> ever make use of anything under post-Singularity communism?)

I guess they'll modify themselves to be totally and provably altruistic
and then share their resources with each other.

> or that the
> disposition of the conserved resources would be decided by majority vote.
> Of course, this just drops the question back a level and asks what criteria
> determine "simpler". I would say that private property is the solution that
> resonates most strongly with the Friendliness principles of "If a question
> is arbitrary, let people pick their own answers" and "A symmetrical solution
> is better than an asymmetrical one; at the acausal level, symmetry is
> mandatory". It's possible to view communism and democracy as voluntary
> special cases of this rule. If you can figure out a good way to ground
> communism and democracy symmetrically on the acausal level, you might be
> able to make a case that they are more basic, or more consistent with other
> moral principles, than the private-property/volition model.

Why doesn't the FAI make everyone altruistic and implement communism?
Isn't that just as symmetrical as private-property/volition? BTW, where
does "If a question is arbitrary, let people pick their own answers"
appear in CFAI? There's a lot of discussion about symmetry but I didn't
see much about freedom.

> Why, yes, what a good idea. No offense, Wei Dai, and I'm speaking strictly
> in my personal capacity as just one more poster on SL4, but gimme a break!
> If it took until now for that thought to filter into my head, I should have
> been locked up as a menace to the human species in 1998.

Sorry, I didn't express myself very well there. What I meant is, please
write down your personal philosophy in more detail so you can get more
reviews and feedback.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:38 MDT