NASA Software Safety Guidebook - EverySpec

NASA Software Safety Guidebook Forward This document is a product of the NASA Software Program, an Agencywide
program to promote the continual improvement of software engineering within
NASA. The goals and strategies for this program are documented in the NASA
Software Strategic Plan, July 13, 1995. Additional information is available from the Software IV&V Facility on the
world-wide-web site http://www.ivv.nasa.gov
Contents
1. INTRODUCTION 12
1.1 Scope 12
1.2 Purpose 13
1.3 Acknowledgments 14
1.4 Associated Documents 14
1.5 Roadmap of this Guidebook 14
2. SOFTWARE SAFETY IN A SYSTEM SAFETY CONTEXT 17
2.1 What is a Hazard? 17
2.2 What Makes Software Hazardous? 18
2.2.1 What is Safety Critical Software? 19 2.2.2 How Does Software Control Hazards? 19 2.2.3 What About Hardware Controls? 19 2.2.4 Caveats with Software Controls 20 2.2.5 What is Fault Tolerance? 21 2.3 The System Safety Program 21
2.3.1 Safety Requirements Determination 22 2.4 Preliminary Hazard Analysis (PHA) 23
2.4.1 PHA Approach 24
2.4.1.1 Identifying Hazards 25
2.4.1.2 Risk Levels 26
2.4.1.3 NASA Policy for Hazard Elimination/Control 28 2.4.2 Preliminary Hazard Analysis Process 28 2.4.3 Tools and Methods for PHA 30 2.4.4 PHA is a Living Document 32 2.5 Software Subsystem Hazard Analysis 32
3. SOFTWARE SAFETY PLANNING 33
3.1 Software Development Life-cycle Approach 34
3.2 Scope of Software Subsystem Safety Effort 36
3.2.1 Identify Safety Critical Software 37 3.2.2 Categorize Safety Critical Software Subsystems 38
3.2.2.1 Software Control Categories 39
3.2.2.2 Software Hazard Criticality Matrix 40
3.2.3.1 Determine Extent of Effort 42
3.2.3.2 Oversight Required 43
3.2.3.3 Tailoring the Effort 44
3.2.3.3.1 "Full" Software Safety Effort 45
3.2.3.3.2 "Moderate" Software Safety Effort 45
3.2.3.3.3 "Minimum" Software Safety Effort 46
3.2.3.3.4 Match the Safety Activities to Meet the Development Effort
46 3.3 Incorporating Software Safety into Software Development 47
4. SAFETY CRITICAL SOFTWARE DEVELOPMENT 55
4.1 Software Concept and Initiation Phase 55
4.2 Software Requirements Phase 56
4.2.1 Development of Software Safety Requirements 57
4.2.1.1 Safety Requirements Flow-down 57 4.2.2 Generic Software Safety Requirements 57
4.2.2.1 Fault and Failure Tolerance/Independence 58
4.2.2.2 Hazardous Commands 60
4.2.2.3 Timing, Sizing and Throughput Considerations 61 4.2.3 Formal Methods - Specification Development 63
4.2.3.1 Why Is Formal Methods Necessary? 64
4.2.3.2 What Is Formal Methods? 65 4.2.4 Model Checking 66
4.2.4.1 How Model Checking Works 66
4.2.4.2 Tools 67
4.2.4.3 Challenges 68 4.2.5 Formal Inspections of Specifications 68 4.2.6 Test Planning 69 4.3 Architectural Design Phase 70
4.3.1 Safety Objectives of Architectural Design 70
4.3.1.1 Fault Containment Regions 71
4.3.1.2 N-Version Programming 72
4.3.1.3 Redundant Architecture 73 4.3.2 Structured Design Techniques 73
4.3.2.1 Object Oriented Analysis and Design 75
4.3.2.2 Unified Modeling Language (UML) 77 4.3.3 Selection of COTS and Reuse 78 4.3.4 Selection of development tools and operating systems 78 4.3.5 Coding Standards 78 4.3.6 Test Plan Update 79 4.4 Detailed Design Phase 79
4.5 Software Implementation 81
4.5.1 Coding Checklists 81 4.5.2 Defensive Programming 82 4.5.3 Refactoring 82 4.5.4 Unit Level Testing 83 4.6 Software Integration and Test 84
4.6.1 Testing Techniques 86 4.6.2 Test Setups and Documentation 91 4.6.3 Integration Testing 92 4.6.4 Object Oriented Testing 92 4.6.5 System Testing 93 4.6.6 Regression Testing 94 4.6.7 Software Safety Testing 95 4.6.8 Test Witnessing 96 4.7 Software Acceptance and Delivery Phase 97
4.8 Software Operations & Maintenance 97
5. SOFTWARE SAFETY ANALYSIS 99
5.1 Software Safety Requirements Analysis 100
5.1.1 Software Safety Requirements Flow-down Analysis 100
5.1.1.1 Checklists and cross references 101 5.1.2 Requirements Criticality Analysis 101
5.1.2.1 Critical Software Characteristics 103 5.1.3 Specification Analysis 105
5.1.3.1 Control-flow analysis 106
5.1.3.2 Information-flow analysis 106
5.1.3.3 Functional simulation models 106 5.1.4 Formal Inspections 107 5.1.5 Timing, Throughput And Sizing Analysis 107 5.1.6 Software Fault Tree Analysis 109 5.1.7 Conclusion 109 5.2 Architectural Design Analysis 110
5.2.1 Update Criticality Analysis 110 5.2.2 Conduct Hazard Risk Assessment 111 5.2.3 Analyze Architectural Design 111
5.2.3.1 Design Reviews 112
5.2.3.2 Prototype/Animation/Simulation 112 5.2.4 Interface Analysis 113
5.2.4.1 Interdependence Analysis 113
5.2.4.2 Independence Analysis 113 5.2.5 Update Timing, Throughput, and Sizing Analysis 113 5.2.6 Update Software Fault Tree Analysis 113 5.2.7 Formal Inspections of Architectural Design Products 114 5.2.8 Formal Methods and Model Checking 114 5.3 Detailed Design Analysis 114
5.3.1 Design Logic Analysis (DLA) 115 5.3.2 Design Data Analysis 115 5.3.3 Design Interface Analysis 116 5.3.4 Design Constraint Analysis 117 5.3.5 Design Functional Analysis 117 5.3.6 Software Element Analysis 118 5.3.7 Rate Monotonic Analysis 118 5.3.8 Dynamic Flowgraph Analysis 118 5.3.9 Markov Modeling 119 5.3.10 Measurement of Complexity 119
5.3.10.1 Function Points 120
5.3.10.2 Function Point extensions 121 5.3.11 Selection of Programming Languages 122 5.3.12 Formal Methods and Model Checking 123 5.3.13 Requirements State Machines 123 5.3.14 Formal Inspections of Detailed Design Products 123 5.3.15 Software Failure Modes and Effects Analysis 123 5.3.16 Updates to Previous Analyses 124 5.4 Code Analysis 124
5.4.1 Code Logic Analysis 125 5.4.2 Code Data Analysis 126 5.4.3 Code Interface Analysis 126 5.4.4 Update Measurement of Complexity 126 5.4.5 Update Design Constraint Analysis 126 5.4.6 Formal Code Inspections, Checklists, and Coding Standards
127 5.4.7 Applying Formal Methods to Code 127 5.4.8 Unused Code Analysis 128 5.4.9 Interrupt Analysis 128 5.4.10 Final Timing, Throughput, and Sizing Analysis 129 5.4.11 Program Slicing 129 5.4.12 Update Software Failure Modes and Effects Analysis 129 5.5 Test Analysis 130
5.5.1 Test Coverage 130 5.5.2 Formal Inspections of Test Plan and Procedures 130 5.5.3 Reliability Modeling 131
5.5.3.1 Criteria for Selecting a Reliability Model 131
5.5.3.2 Issues and Concerns 132
5.5.3.3 Tools 132
5.5.3.4 Dissenting Views 133
5.5.3.5 Resources 133 5.5.4 Checklists of Tests 134 5.5.5 Test Results Analysis 134 5.5.6 Independent Verification and Validation 134 5.5.7 Resources 135 5.6 Operations & Maintenance 135
6. SOFTWARE DEVELOPMENT ISSUES 136
6.1 Safe Subsets of Languages 137
6.2 Insecurities Common to All Languages 138
6.3 Method of Assessment 139
6.4 Languages 139
6.4.1 Ada83 and Ada95 Languages 140 6.4.2 Assembly Languages 143 6.4.3 C Language 144 6.4.4 C++ Language 148 6.4.5 C# Language 151 6.4.6 Forth Language 153 6.4.7 FORTRAN Language 154 6.4.8 Java Language 155 6.4.6 LabVIEW 157 6.4.7 Pascal Language 158 6.4.8 Visual Basic 159 6.5 Miscellaneous Problems Present in Most Languages 159
6.6 Programming Languages: Conclusions 161
6.7 Compilers, Editors, Debuggers, IDEs and other Tools 162
6.8 CASE tools and Automatic Code Generation 164
6.8.1 Computer-Aided Software Engineering (CASE) 164 6.8.2 Automatic Code Generation 166
6.8.2.1 Visual Languages 166
6.8.2.2 Visual Programming Environments 167
6.8.2.3 Code Generation from Design Models 167 6.9 Software Configuration Management 169
6.9.1 Change control 170 6.9.2 Versioning 170 6.9.3 Status Accounting 171 6.9.4 Defect Tracking 172 6.9.5 Metrics from your SCM system 172 6.9.6 What to include in your SCM system 173 6.10 Operating Systems 174
6.10.1 Types of operating systems 174 6.10.2 Do I really need a real-time operating system (RTOS)? 174 6.10.3 What to look for in an RTOS 175 6.10.4 Commonly used Operating Systems 177 6.11 Distributed Computing 178
6.12 Programmable Logic Devices 181
6.12.1 Types of Programmable Logic Devices 182 6.12.2 "Program Once" Devices 182 6.12.3 "Reprogram in the Field" Devices 183 6.12.4 Configurable Computing 183 6.12.5 Safety and Programmable Logic Devices 184 6.13 Embedded Web Technology 186