IV&V Using Executable Specifications Developed by Programs Steve Rogers June, 2010
1
Presentation Outline • IEEE 1012 objectives • Executable Specification(ES) Definition & Overview • ES Analyses Process in IV&V • Example of IV&V Analyses using an ES • Conclusions & Recommendations 2
IEEE 1012 Objectives & Benefits • • •
•
The IV&V NASA center uses the IEEE 1012 standard for software V&V V&V should demonstrate whether the software requirements and system requirements (i.e., those allocated to software) are correct, complete, accurate, consistent, and testable. The software V&V processes should determine whether the development products of a given activity conform to the requirements of that activity and whether the software satisfies its intended use and user needs. Benefits should include: – – – – – – –
•
Facilitate early detection and correction of software anomalies Enhance management insight into process and product risk Support the life cycle processes to ensure conformance to program performance, schedule, and budget Provide an early assessment of software and system performance Provide objective evidence of software and system conformance to support a formal certification process Improve the software development and maintenance processes Support the process improvement for an integrated systems analysis model
An Executable Specification as developed by the programs contains source code and will satisfy many of the objectives & provide many of the benefits described above.
3
Presentation Outline • IEEE 1012 objectives • Executable Specification(ES) Definition & Overview • ES Analyses Process in IV&V • Example of IV&V Analyses using an ES • Conclusions & Recommendations 4
Executable Specification V&V System Tool Overview - 1
• The Executable Specification (ES) is a matlab/simulink V&V/design/analyses tool – Developed by the programs – Used for V&V of many system requirements – Used as GN&C executable prototype & flight application code generation. Generates flight code to the System Integration Lab (SIL) or equivalent. The ES contains source code covered by IEEE 1012. – System performance requirement (qualified) V&V – Runs on desktop PC or laptop – Maintained as a ‘truth’ model throughout the design & operational life of the program as part of the SIL. (Updated with current information when available) 5
Executable Specification V&V System Tool Overview - 2 • The ES is one of the system performance V&V tools developed and used by the program. A Formal Methods approach will be used for V&V. • System performance GN&C requirements verified by the ES include: – Guidance: 1st phase trajectory tracking accuracy by ES (verification will eventually be done by the SIL as it accumulates high fidelity prototype avionics). These are time-based requirements checked at multiple operating conditions. – Navigation: 1st phase measurement estimation and state estimation accuracy by ES (verification will also eventually be done by the SIL). These are time-based requirements. – Control: stability metrics (verification of phase & gain margins will be done with the ES for the life of the program). These are frequency-based requirements checked at multiple flight conditions. – Control: 1st phase attitude tracking (verification will eventually be done by the SIL). These are time-based requirements checked at multiple operating conditions. 6
Executable Specification V&V System Tool Overview - 3
• Validation of system stability and performance GN&C for time/frequency based linear analyses issues is done with the ES. These include: – Stability and sensitivity to disturbances due to manufacturer’s variability, design errors, model errors, unmodeled phenomena, changes during the project life, new/routine maneuvers, flexible structural modes, fuel slosh, sensor/actuator faults, and sensor noise, – Step & impulse responses, – Robustness analyses, – Monte Carlo or dispersion analyses, – 1st phase operational maneuver testing, – Analyses such as Nichols, bode, and root locus. An ES test bed can accommodate massive numbers of tests that are too expensive to do elsewhere.
7
Presentation Outline • IEEE 1012 objectives • Executable Specification(ES) Definition & Overview • ES Analyses Process in IV&V • Example of IV&V Analyses using an ES • Conclusions & Recommendations 8
Executable Specification Analysis Verification
Obtain Model Components & Documentation
Linear Analyses
Validation
Verification Simulation Validation • •
• • • •
GLORY, GLAST, ARES, & nearly all of the newer projects have a simulink-based model or an executable specification for GN&C development and analyses. The executable specifications are usually maintained by the program and consist of a vehicle 6 DOF model and GN&C algorithms written in simulink and matlab scripts. Obtaining the executable specification model & documentation will consist of: 1. Vehicle model (actuators, sensors, thrust, mass properties, flexible modes, fuel slosh, statistical dispersions of all parameters, etc. ) 2. GN&C component (autopilot parameters, guidance parameters, filter parameters, documentation showing derivations, etc.) 3. Initial conditions, dispersions, and nominal values for attitudes, actuator positions, filter states, 4. Flight conditions during launch (altitudes, velocities, alpha, beta, air density) & later, on orbit. Linear analyses/simulation are action items to produce various performance metrics that determine flight capability. Linear analyses produces model-based Formal Methods stability metrics, such as phase/gain margins. Simulation produces positions, rates, and accelerations needed to ensure set point tracking performance bounds are maintained throughout the mission. Verification and validation both make use of linear analyses and simulation. Verification is testing specific performance requirement thresholds against nominal models using a suite of verification indicators. Validation will use the same verification indicators as well as additional indicators to assess suitability of the design considering 9 anticipated model dispersions.
Executable Specification Analysis Process Outputs
Actions
Frequency-based Analyses
Model Integration
Time-based
Obtain Model Components & Documentation Model Initialization
Design Updates
Requirement List
Collect Environmental/vehicle Parameter nominal and Variations Environmental
Vehicle
Temperature profiles Etc.
actuator flexible structure fuel slosh solid fuel variations
Typical Requirement List: Frequency: Rigid Body Mode Stability Margins Flexible Body Mode Stability Margins Tracking Error: True Heading Roll Rate Roll Orientation Yaw Rate at Separation Pitch Rate at Separation Roll Rate at Separation Roll Orientation at Separation Vehicle state estimation accuracy
10
Executable Specification Requirement Verification Actions Frequency-based
Time-based
Requirements
Rigid Model linearization
Rigid Body Mode Stability Margins
Flexible Model linearization
Flexible Body Mode Stability Margins
Flexible Model Simulation
Note: Current practice as in GLORY document AI1-SYS-CAP-V1.00 is to only use nominal values. As an IV&V group we must ‘break’ the system, i.e., find its limitations or robustness bounds to disturbances or model errors. GLAST has done numerous failure studies.
True Heading Roll Rate Roll Orientation Yaw Rate at Separation Pitch Rate at Separation Roll Rate at Separation Roll Orientation at Separation Vehicle state estimation accuracy requirements GLORY general frequency requirement of 10-dB gain margin, 40 degrees phase margin, and 10 db for flexible mode attenuation for nominal conditions.
11
Executable Specification Requirement Validation Actions
Validation Indicators Rigid Body Mode Stability Margins (more data points)
Frequency-based
Rigid Model linearization
Stability Margin/sensitivity studies using additional indicators Flexible Body Mode Stability Margins (more data points)
Input appropriate dispersion distribution models, such as Gaussian or hard bounds,
Time-based
Flexible Model linearization Stability Margin/sensitivity studies using additional indicators
Flexible Model Simulation
True Heading Roll Rate Roll Orientation Yaw Rate at Separation Pitch Rate at Separation Roll Rate at Separation Roll Orientation at Separation Vehicle state estimation accuracy requirements Monte Carlo studies using variable parameters
Note: Use both nominal & dispersed values In Monte Carlo studies Flexible model includes rigid model
12
Presentation Outline • IEEE 1012 objectives • Executable Specification(ES) Definition & Overview • ES Analyses Process in IV&V • Example of IV&V Analyses using an ES • Conclusions & Recommendations 13
Satellite Pitch Control with Fuel Slosh 1 contsys
Thetsys
controller
Sat&FuelSloshmdl
Bode Diagram Gm = 13.9 dB (at 1.9 rad/sec) , Pm = 46.1 deg (at 0.628 rad/sec) 150
With large satellites a mass of fuel is required for maneuvers. It may be up to 40% of the mass. This may be a useful example. Will look for an actual project.
Magnitude (dB)
100
Typical System Requirement List:
50 0
Frequency: Rigid Body Mode Stability Margins Flexible Body Mode Stability Margins
-50 -100
Tracking Error: True Heading Roll Rate Roll Orientation Yaw Rate at Separation Pitch Rate at Separation Roll Rate at Separation Roll Orientation at Separation Vehicle state estimation accuracy
Phase (deg)
-150 0
-90
-180
-270 -3
10
-2
10
-1
10
0
10
1
10
2
10
Frequency (rad/sec)
Testing of all of the requirements exercises the simulink flight code. phase margin = 46.1 deg (good) Gain margin = 13.9 dB (good)
14