Re: Why SIAI is a nonprofit

From: Samantha Atkins (
Date: Wed Oct 27 2004 - 00:18:38 MDT

On Oct 22, 2004, at 2:20 PM, Eliezer Yudkowsky wrote:

> And even if you succeed, you lose years. I don't think the human
> species has that kind of time. If we win, and we may not win, it's
> going to be close. We need the difficult cure before the easy
> disease. Add a delay of six years (and probably more) to start a
> corporation, assume the unlikely event of success, and by the time we
> were done it would probably be just too late. Also, *I* don't have
> time. I have to do my thinking while I'm still young enough to think.
> I have to use my annus mirabilis years on AI and nothing else. Youth
> is a non-renewable resource for solving scientific problems. I still
> live in fear of running out of youth before I run out of fundamental
> AI problems requiring basic shifts in thinking and deep mental
> retraining.

This bit of ageism is probably not that strongly justified. If you
believe it is then please provide your evidence. I seem to have gone
through several basic shifts in thinking and deep retraining in my 30s
and 40s and am on the edge of another at the ancient age of 50.
Perhaps I am just odd in that way. But I and other older folks I know
still seem quite able to think mostly as well and as deeply as when we
were in our 20s.

> Oh, and fifty-five million humans die forever every year. It's a
> piece of knowledge that sinks its way into you very slowly, because
> it's so hard to comprehend. But I understand more as time goes on.
> I'll try and stay sane (well, reasonably sane) regardless, which is
> the most critical part of this job and the part that most people would
> instantly flub.

I think you are loading yourself up with a lot of pressure that does
not serve you or the work. You are too good and have much too much to
do to play super-hero.

> It might make sense for other SIAI supporters to try and start
> companies with the goal of funding the Singularity Institute if they
> succeed. Some of them are. It does not make sense for me to do so,
> unless I am willing to entirely sacrifice my potential as an FAI
> researcher.

Entirely? Hardly. How much are you sacrificing now by spending year
after year barely scraping by churning the theoretical space but
incapable of even beginning to build the infrastructure much less the
actual AI?

> But mostly, I think people making the suggestion don't know how
> unlikely is a new company to succeed, how much energy it takes from a
> person, the cold statistics of technology startup survival rates, and
> the wry psychology experiments showing that everyone thinks they can
> beat the statistics because their plan is different.

So, don't start a company. Find a niche to do consulting in or find
people who do want to start a company using some of your ideas or
attract a better evangelism team. Just don't run so purist that you
haven't a prayer of accomplishing much except maybe the theoretical
foundation. Only so much can be done in your head. Only so much can
be done in relative isolation no matter how brilliant you are.

> As for the comment about monks holding out begging bowls... either
> this is a suggestion that all charities stay small and accomplish
> nothing, which is false-to-fact, or it is a suggestion that begging is
> beneath a proud Earth's defender. In which latter case the speaker
> does not begin to comprehend the concept of "existential risk". Pride
> counts for nothing. It vanishes like a snowflake in the ocean, a
> feather buried under the weight of lives. Yes, it hurt to ask for
> help, just like it hurt to give up my dreams of being a millionaire.
> I did it anyway because people were dying.

If SIAI doesn't find a deeper source of funding and get some actual
building the tech traction the people will continue to die.

> I suppose that sounds hokey to some people, like I'm making a big deal
> over nothing and feeling too much. Fine. Maybe someday you'll
> understand that it was all real, or maybe you'll die before then.
> Maybe you'll die one day before the bus arrives. If so, I'd like to
> apologize now for not working just a little bit faster.

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:49 MDT