Re: Human Fallibility

From: Dani Eder (
Date: Thu May 04 2006 - 06:17:53 MDT

based on field audits by organisations such
> as Coopers and
> Lybrand, KPMG, and HMCE (the UK's Customs and Excise
> department). Of
> 54 spreadsheets audited between 1997 and 2000, no
> fewer than 49 were
> found to contain significant errors - a defect rate
> of 91 per cent. In
> a more recent exercise, every single one of 30
> project financing
> spreadsheets scrutinised had at least one mistake.

Okay, so we know humans are fallible. That's been
going on forever. Dionysius Exiguus (Dennis the
Short), the monk who created our system of dating
years since Christ's birth got it wrong. This
leads to the absurdity of Christ being born approx
4-7 BC. But then you try to do a long string of
calculations in Roman numerals.

Knowing that humans are fallible (and hardware too,
just not as often:,,3715_13965,00.html?redir=CORPR01

the question is what is the appropriate amount of
error checking for a given situation. The bigger
the consquences of an error, and the more likely
an error can happen, the more error checking you
should do.

In the case of "accounting errors" that lead to
restating a company's financial results, they should
probably have had a few more people checking the
numbers, and it is worth questioning whether they
even had a plan for error checking.

In the case of the Space Station's software (which
I was a tester for), the consequences of losing a
multi-billion dollar asset, the lives of the crew,
and the knowledge from future research onboard were
so great that a lot of effort went in to getting it
right. Despite that, a handful of errors made it
into orbit. Knowing that was possible, the Station
was designed with distributed redundant systems, so
no single hardware, sofware, or human operator error
was likely to lead to a disaster.

In the case of an AGI, or replicating systems
(nano-, bio-, or robo-tech), or other manmade items
with potentially existential consequences if they
go bad, what is the proper amount of effort in
avoiding errors? And are you guys working on AGI
development putting that effort in? Are the risks
in developing an AGI less than the reduction in
other risks if you have an AGI?


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around

This archive was generated by hypermail 2.1.5 : Wed Jul 17 2013 - 04:00:56 MDT