Re: About "safe" AGI architecture

From: Harvey Newstrom (
Date: Sun Jun 13 2004 - 10:42:28 MDT

"Safety" is just one attribute in this Information Assurance business,
that also includes security, quality, performance, reliability,
functionality, etc. The current hierarchy of assurance adjectives I
have developed is below. It derives from security definitions,
auditing definitions, safety definitions, accounting definitions, and
development definitions. They are my attempt at a synthesis of all my
certifications (those letters after my name). All of these fields seem
involved with making things work right, but I needed to combine them
into an overall architecture to work together. I am about to get
another one in Security Architecture, which is why I am think about
this stuff lately. Also, my current job is full time security
architecture. Architecting security, or safety or friendliness is very
different than just being a good programmer and making programs work
right. In the development world, the architecture is a design
workproduct that tells the programmers what to build. It must be
specific enough to do and to verify. It can't be vague philosophies or
statements that nothing bad should happen.

Each word in my hierarchy below has a technical, specific meaning which
must be verified separately and precisely. Most engineers or software
developers should already recognize these terms if they have worked on
large formal systems development. I know this list scares people,
because it is so big and exhaustive. But if you want safety or
security, (or friendliness) you better address this list. People
usually want these goals to be achieved, but if they aren't part of the
original specifications or designs, they won't be.

I would love to hear of any attributes I missed! I am not sure where
"friendliness" fits in. It seems to be the reverse flow of these
attributes. Instead of protecting a system from outside influence
disrupting one of these attributes, it seems to involve protecting
outside entities from system influences disrupting these attributes for
them. If so, it may turn out that friendliness is the same thing as
security with a reversed causality. Instead of protecting ourselves
from bad things, our system nurtures ourselves with good things. If
this is a good analogy, it may imply that friendliness can take a giant
leap forward by learning and adopting standard security techniques and
principals and using the same architecture to protect the same
attributes from a different direction of causality.

Even if the above speculation does not turn out to be true, we still
want general security and safety in any major system we design. The
attributes to be protected are:

                (Business) Continuity
                (Disaster) Recovery
                Fault Tolerance
                Access Controllability
                Custody Chain


This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:47 MDT