From: Michael Wilson (mwdestinystar@yahoo.co.uk)
Date: Thu May 19 2005 - 09:39:08 MDT
Dani Eder wrote:
> You start at the top level by describing what it
> is you want the thing to do, in some detail. We
> call it a requirements document.
Writing a requirements document for an FAI is a massive task
in itself; it's hard to pick good Friendliness content and it's
hard to define it in engineering terms rather than fuzzy
humanistic terms. I suppose there is a parallel with commercial
software design, in that it's hard to design good business
processes for systems to fit into and it's hard to get a spec
out of the customer that is both specific and what they actually
want (and this is before you even start designing the software).
> Once you have broken the system into manageable pieces, you
> can then proceed to design the pieces, build them, and then
> test that they actually meet the requirements that were
> specified at the lowest level. You assemble and test at
> progressively higher levels until finally you test the complete
> system as a whole.
Part of the difficulty of AGI design is that it doesn't modularise
easily into a set of cleanly seperated components with narrow
interfaces and distinct functionality. It will modularise into
relatively intuitive components and organisational layers with
multiple functions and fuzzy boundaries (a scheme used by the
brain), which improves the chance of an educated guesswork design
working, but doesn't help much with clean formal design.
Alternatively AGI will modularise into a dense heterarchy of
functional components with well defined boundaries and interfaces
but a complex tangle of causal relationships and many layers of
indirection from the hardware to familiar CogSci concepts. This
is much more amenable to formal analysis, but is harder to develop
and explain.
>> The SIAI approach relies on achieving complete understanding
>> of how the design works before building it. This means
>> specifying global behaviour first and breaking that down into
>> progressively more local behaviour until you have code.
>
> In the aerospace industry in which I work, this is a part of
> the methodology we call 'systems engineering'. It's not a
> perfect process... there is a feedback up and down the tree
> as the definition and design is done.
Practical FAI design involves juggling constraints, all of which
are ultimately probabilistic. At the top level there are constraints
on the range of behaviour that we want the AGI to exhibit, which we
want to show have a very high probability of being satisfied. On
the other hand we aren't required to prove that an AGI will be
computationally tractable before running it; if we build a seed AI
that is Friendly but twelve orders of magnitude too slow to do
anything useful, no harm is done. Equally if we build an AGI that
we can prove will reliably and safely shut down before departing
from Friendly behaviour, that's safe too. Obviously we'd prefer to
establish ahead of time a high probability of the design actually
working, but the acceptable probabilities for this are much much
lower than the acceptable probability that the design will be safe.
Personally I'm prepared to use empirical evidence of whether
algorithms are computationally tractable if the time it would take
to understand their exact performance characteristics doesn't
deliver worthwhile returns; more understanding is always good, but
we're on a tight schedule.
> But you have to be careful that any changes to the requirements
> and design are propagated through the tree so that (a) the overall
> system still does what you wanted it to, and (b) the pieces will
> still work together.
Definitely.
> The methodology takes a lot of manpower to implement,
True, which is why I believe we need to develop better tools as we
go to speed things up.
* Michael Wilson
___________________________________________________________
How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com
This archive was generated by hypermail 2.1.5 : Tue Feb 21 2006 - 04:22:56 MST