SAFEWARESystem Safety and ComputersChap18:Verification of Safety
Author : Nancy G. Leveson
University of Washington
1995 by Addison-Wesley Publishing Company, Inc.
ISBN : 0-201-11972-2
Presentation by Yean-Ru Chen at ESL of CS, CCU, Taiwan
2005/09/08
Outline(1/1)
Introduction
Hazard analysis
Dynamic
Static
Conclusions
Introduction(1/2)
Most verification techniques focus on showingconsistency between the code and thespecification, but for safety, this approach isnot enough.
We need to examine not only the relationshipbetween the computer inputs and outputs,but also that between the inputs and theeffect of the outputs on system behavior.
Introduction(2/2)
Because of considering the effect of theoutputs on system behavior ,
“Hazard analysis” is needed to serves asthe basis for this approach to safetyverification.
Hazard analysis
Hazard is a state or set of conditions of asystem that, together with other conditions inthe environment of the system, will leadinevitably to an accident. (Chapter 9)
There are two main analysis ways:
(1)Dynamic Analysis
In this analysis, the code or a model of the code isexecuted and its performance is evaluated.
(2)Static Analysis
In static analysis, the code or model is examinedwithout being executed.
Dynamic Analysis(1/6)
Testing!
Here we’ll focus on software and safetyonly.
Process Considerations
Limitations of Testing
Dynamic Analysis(2/6)
Process Considerations:
There are several aspects of the systemneeded to be considered when we use the“Testing” approach:
Critical functions and variables.
Boundary conditions.
Special features, such as firewalls or safetykernels upon which the protection of thesafety-critical features is based.
Dynamic Analysis (3/6)
Reaction of the software to system faults andfailures.
The model of the system environment that guidescritical control and safety decisions.
The operator interface.
Dynamic Analysis (4/6)
In general, there are also several aspects forthe safety engineers to pay more attention :
Review test plans
Recommend tests based on the hazard analyses,safety standards, previous accidents, and so on.
Review the test results for any safety-relatedproblems that were missed in the analysis or inother testing.
Ensure that testing feedback is integrated into thesafety reviews and analyses that will be used indesign modifications.
Dynamic Analysis (5/6)
Limitations of Testing:
The main limitation of testing is thattesting is impossible in a real setting.Most testing must be done in a simulationmode, and there’s no way to guaranteethat the simulation is accurate.
Dynamic Analysis (6/6)
But these limitations do not mean that weshould not test or include testing in asafety program.
It merely means that there is a limit tothe confidence that can be acquiredthrough dynamic analysis and that testingmay have to be augmented with staticanalysis if the limit is less than what isrequired.
Static Analysis
Static analysis evaluates the softwarewithout executing it. Instead, it examines arepresentation of the software.
There are two major means in this chapter :
Formal Verification --- formal
Software Fault Tree Analysis (SFTA) --- informal
Formal Verification (1/4)
We mainly concern about the formalverification on software in this book.
Formal verification essentially providesa proof of consistency between twoformal specifications of a system.
In safety, we have to use formal modelsand definitions of safety propertiesalong with safety policies.
Formal Verification (2/4)
A safety policy would describe the rulesfor determining whether a system issafe and would use the safetyconstrains as a basis for decisionmaking.
There are three general safety policiesfor which the specification could bechecked: (Chapter 15)
Formal Verification (3/4)
(I) There must be no paths to unplannedhazardous states.
(II) Every hazardous state must have a pathto a safe state. All paths from the hazardousstates must lead to safe states. Time in thehazardous state must be minimized, andcontingency action may be necessary toreduce risk while in the hazardous state.
Formal Verification (4/4)
(III) If a safe state cannot be reached froma hazardous state, there must exist at leastone path from that state lead to aminimum risk state.
Major drawbacks of formal verification onsoftware : Practicality and Feasibility.
Major advantages of formal verification onsoftware : Analytical ,Mathematical andFormal.
Software Fault Tree Analysis(SFTA) (1/4)
FTA focuses on hazards rather than onsoftware errors in general, and it can beapplied to assembly or high-level language.
A leaf node of the software fault treedescribes a behavior of the computer.
SFTA traces that behavior into the logic ofthe code and determines whether a pathexists through the code that could cause thehazardous output.
Software Fault Tree Analysis(SFTA) (2/4)
SFTA uses Backward Reasoningapproach.
It follows the paths backward throughthe program from the hazardousoutputs to identify any inputs thatmight cause that output.
Software Fault Tree Analysis(SFTA) (3/4)
Here is an example of backward reasoningabout variable values :
X = 9
X =10
if a variable contains the value 10 after aninstruction adds one to that variable, then thevariable must have held the value 9 beforethe instruction was executed.
X := X + 1
Software Fault Tree Analysis(SFTA) (4/4)
SFTA uses fault tree templates togenerate the software fault trees.
Here are 4 templates to be introduced:
Template for assignment
Template for IF statement
Template for WHILE statement
Template for procedure call statement
Template for assignment
Fig.18_4
Assignment causesevent
or
Operand evaluationcauses event
Exception causes event
Change in value causesevent
Ex. of assignment statementsand simple IF ( A Tree !)
(1) A := F(Y)
(2) B := X-5.0
(3) IF A>B THEN FIRE
FIRE called
A > B
F(Y) > X – 5.0
A > X – 5.0
Template for IF statement
Fig.18_6
Event in IF-THEN-ELSE
or
Condition FALSE,ELSE-part causes event
Condition evaluationcauses event
Condition TRUE ,THEN-part causes event
and
and
ConditionTRUE beforeIF-THEN-ELSEstatement
THEN-partcauses event
ConditionFALSE beforeIF-THEN-ELSEstatement
ELSE-partcauses event
Ex. of IF statements( A Tree !)analyze for the unsafe condition “X>100”
if a > b
then X := f(X) ;
else
X := 100 ;
X>100 caused byIF statement
or
a>b , X := f(X) causesX>100
and
and
a > b before IF-THEN-ELSE
X := f(X)causes X>100
a <= b before IF-THEN-ELSE
X := 100causes X >100
a<=b , X := 100 causesX>100
Template for WHILE statement
Fig.18_8
WHILE statementcauses event
or
Statement executed ntimes
Statement not executed
and
and
Event beforeWHILEstatement
Condition falsebefore WHILE
n_th iterationcauses event
Condition truebefore WHILE
Ex. of WHILE statements( A Tree !)analyze for the unsafe condition “z>100”(1/2)
while b>x {
b:=b-1;
z:=z+10;
}
WHILE statementcauses z>100
or
Statement executed ntimes
Statement not executed
and
and
z>100 beforeWHILEstatement
b <= x beforeWHILE
z + 10(b-x) >100before WHILE
b > x beforeWHILE
Ex. of WHILE statements( A Tree !)analyze for the unsafe condition“z>100”(2/2)
Why z+10(b-x) > 100 will cause unsafecondition when “while loop” has been executed ntimes?
Original value of z before loop : z
Original value of b before loop : b
The value of z after nth iteration :z’
The value of b after nth iteration :b’
z’ = z+10n, b’ = b – n
Since iteration stopped, b’ = x (at least)
So, z’ = z+10(b-x)
If z’ > 100 ,then it’s unsafe.
Template for procedure callstatement
Fig.18_9
F (p1,p2,p3,…)causes event
or
Failure of F causesevent
Execution of F withp1,p2,p3…causesevent
Evaluation ofparameters causes event
Uses and Evaluation ofSFTA(1/2)
If SFTA is used with a system simulator,the critical software interfaces with therest of the system can be identified andused to determine appropriatesimulation states and events.
However, the practicality of SFTA useon large-scale software has not beendemonstrated.
Uses and Evaluation ofSFTA(2/2)
One of the reasons is that SFTA is basicallya structured walkthrough of the code withan emphasis on hazardous behavior , so itspracticality should be similar to that ofother structured walkthrough techniques.
However, the entire code is not usuallysafety-critical, maybe small part of it needto be examined in SFTA.
Conclusions
Those using computers for safety-critical functions should be wary ofrelying too heavily on the techniquesdescribed in this chapter.
They are necessary but not sufficient!!