RE: Flawed Risk Analysis (was Re: SIAI's flawed friendliness analysis)

From: James Rogers (
Date: Fri May 23 2003 - 11:47:37 MDT

Bill Hibbard wrote:
> I write open source visualization software for a living,
> and in the open source community it is common for people
> to take over development of other people's systems. It
> is also common for new people on a project to become
> experts in the system. Many of these systems are very
> large and complex.
> There are plenty of examples of successful analysis of
> complex software from compiled object code for espionage
> (both commercial and military), where there is no
> cooperation from the designers.

You have completely missed the point. I am not referring to the complexity
of the actual code, but the conceptual complexity of the design space. Most
code in the world is of an exceptionally simple design.

I design algorithmically sophisticated systems for a living. Many people
have access to the code and fully understand what the code does, but for
many systems I've designed they still consult me before making any design

Why? Because the code is only a single (n-k)-dimensional slice through an
n-dimensional conceptual design space. There is a vast amount of knowledge
and context that must be grokked if one is going to attempt to modify the
orientation of the "implementation slice" through the conceptual design
space and I've already taken many things into consideration which are not
expressed in the code. It is the difference between fully understanding how
a specific car engine works and being able to design an engine to any given
specification from scratch. One is a mechanic, the other is an engineer. It
is really hard for a mechanic to become a competent engineer by studying a
specific product of an engineer in detail.

It isn't enough that people have access to and understand the code. They
have to grok the entire design space the same way the designer does or they
will never *really* understand the code in a meaningful way. What you are
proposing is a dog and pony show to give "the people" warm-n-fuzzy feelings
without actually accomplishing what it claims to accomplish. It sounds like
a lot of wishful thinking and wasted time to me.

> In my experience, if one group of people can create it,
> then another group can understand it.

"Can understand it" and "do understand it" are very different things, and
the more complex the conceptual space the less likely "do understand it"
will be. Throw politicians and government bureaucracy into the mix and,
well, I don't think I need to explain where THAT goes.


-James Rogers

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:42 MDT