ADOPTING THE QUADRATIC MEAN PROCESS TO QUANTIFY THE QUALITATIVE RISK ANALYSIS
Accepted for publication at PMI Global Congress 2013 – North America New Orleans – Louisiana – USA – 2013
2
Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis
Abstract Related Podcasts
♫♫
Using the Quadratic Average (Mean) in the Qualitative Risk Analysis http://rvarg.as/bd
♫♫Understanding the Risk Attitude http://rvarg.as/6j
♫♫RBS – Risk Breakdown Structure http://rvarg.as/be
♫♫Risk Management Special Playlist http://rvarg.as/bc
The objective of this paper is to propose a mathematical process to turn the results of a qualitative risk analysis into numeric indicators to support better decisions regarding risk response strategies. Using a five-level scale for probability and a set of scales to measure different aspects of the impact and time horizon, a simple mathematical process is developed using the quadratic mean (also known as root mean square) to calculate the numerical exposition of the risk and consequently, the numerical exposition of the project risks. This paper also supports the reduction of intuitive thinking when evaluating risks, often subject to illusions, which can cause perception errors. These predictable mental errors, such as overconfidence, confirmation traps, optimism bias, zero-risk bias, sunk-cost effect, and others often lead to the underestimation of costs and effort, poor resource planning, and other low-quality decisions (VIRINE, 2010).
ricardo-vargas.com
Qualitative X Quantitative Risk Analysis One of the main challenges during the analysis of a risk is to define the right approach to assess the amount of the exposure/opportunity. The two basic steps to determine the right level of risk are based on the qualitative and quantitative analysis (Exhibit 1). Output
Qualitative Assessment of Risk Level
Risk Prioritisation
Risk Analysis
Updated Risk Register
Quantitative Exhibit 1 – Analysis of Risk Process Flow (ROSSI, 2007).
A qualitative risk analysis prioritizes the identified project risks using a predefined scale. Risks will be scored based on their probability or likelihood of occurrence and the impact on project objectives if they occur (Exhibit 2 and 3). LEVEL
SCORE
High Medium Low
Very High
5
High
4
Medium
3
Low
2
Very Low
1
FOR THREATS
Red Yellow Green
Exhibit 2 – Example of scales used in the qualitative risk analysis
Qualitative Risk Matrix
High
3
Medium
2
3’
4’
1
2’
3”
Low
Frequency
Risk Levels are Relative to Regions Connected by Arrows
4
5 Highest Risk
Lowest Risk
Low
Medium Consequence
High
Exhibit 3 – Example of Qualitative Risk Matrix with 3 x 3 Levels (ALTENBACH, 1995)
3
4
Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis
A quantitative risk analysis is based on simulation models and probabilistic analysis, where the possible outcomes for the project are evaluated, providing a quantitative, numeric and many times financial risk exposure to support decisions when there is uncertainty (PMI, 2013). Some quantitative processes are simple and direct like rolling a dice (Exhibit 4), but most of them involve very complex simulation scenarios like the Monte Carlo Simulation.
Exhibit 4 – Diagram showing the deterministic probability of rolling a dice
“Monte Carlo” was a nickname of a top-secret project related to the drawing and to the project of atomic weapons developed by the mathematician John von Neumann (POUNDSTONE, 1993 and VARGAS, 2013). He discovered that a simple model of random samples could solve certain mathematical problems, which couldn’t be solved up to that moment. The simulation refers, however, to a method by which the distribution of possible results is produced from successive recalculations of project data, allowing the development of multiple scenarios. In each one of the calculations, new random data is used to represent a repetitive and interactive process. The combination of all these results creates a probabilistic distribution of the results (Exhibit 5 and 6). Start
A
B
C
E
%
Finish
Forecast of Final Cost
D
Exhibit 5 – Construction of model of distribution of costs and activities or work packages making up a final distribution from random data of the project (PRITCHARD, 2001).
ricardo-vargas.com 
Exhibit 6 – Example of Monte Carlo simulation to assess the cost impact of a potential threat to the project
Because quantitative analysis is based on mathematics and statistics supported by objective metrics, such analyses are considered to be more rigorous (SMOCK, 2002). The main challenges of a solid quantitative analysis are the time and effort it requires to be executed and the required technical background in statistics to make the proper parameterization of the data. The main advantages and disadvantages of each method are presented in the Exhibit 7. QUANTITATIVE METHODS
Facilitates the cost benefit analysis Advantages
Gives a more accurate value of the risk More valuable Results of the method may not be precise
Disadvantages
Numbers can give a false perception of precision More expensive and time consuming
QUALITATIVE METHODS
Relatively simple to be implemented Easily determine risk categories with greater impact in the project Visually impactful A lack of understanding of the parameters used in the scale can lead to different interpretations Results can be biased Less valuable
Exhibit 7 – Example of Monte Carlo simulation to assess the cost impact of a potential threat to the project (based on ROT, 2008)
The risk model proposed hereafter is a qualitative process with numerical results, reducing the ambiguity of the qualitative process without adds the time and effort to determine with precision the probability and the impact of uncertain events in the project.
Assessing Probability The proposed qualitative probability assessment is based on a scale with their respective scores (Exhibit 8).
5
6
Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis LEVEL
SCORE
DESCRIPTION
Very High
5
It is expected that the event will occur. If it does not occurs it will be a surprise.
High
4
The event has a great chance of occurring.
Medium
3
The event can occur.
Low
2
It will be a surprise if the event occurs.
Very Low
1
Very remote chance of the event to happen. Practically impossible.
Exhibit 8 – 5 Scales to assess the risk probability
For each identified risk a score from 1 (one) to 5 (five) should be determined.
5 Dimensions of the Impact The impact of the event, in case it occurs, can be perceived in different dimensions of the project objectives. For example, one risk can have a major impact on costs but not necessarily an important impact in quality. It is very important to highlight that threats and opportunities should be analyzed separately. The basic groups where impact should be evaluated are (Exhibit 9):
• impact on time and deadlines • impact on costs • impact on quality • impact in safety and security • other impacts TIME
OTHER
SAFETY & SECURITY
COST
QUALITY
Exhibit 9 – Basic impact groups showing the different impact dimensions of one specific risk
Each project may develop different impact groups based on the nature of the project, including groups like: impact on reputation, regulatory impact, environmental impact, social impact, and stakeholder’s impact, among several others. Following is the presentation of the 5 basic groups.
ricardo-vargas.com
Impact on Time and Deadlines One should assess the level of impact on the conclusion of the project. It can be positive or negative for opportunities and threats, respectively. Threats that impact the conclusion of the project must be considered as a priority if compared to other events. Because each project differs in size, complexity and several other factors, the project team needs to agree on the level of tolerance that they consider appropriate for each level of impact, like the example shown in Exhibit 10. LEVEL
SCORE
DESCRIPTION
Very High
5
Delays/Anticipation above 180 days or 6 months.
High
4
Delays/Anticipation between 120 and 180 calendar days.
Medium
3
Delays/Anticipation between 60 and 120 calendar days.
Low
2
Delays/Anticipation between 15 and 60 calendar days.
Very Low
1
Less than 15 calendar days of delays/ anticipation.
Exhibit 10 – Example of impact scale and score for time and deadlines
Impact on Costs One should also assess the level of impact that the event may bring to the total project cost. It can be positive (savings) or negative (additional expenditures) for opportunities and threats, respectively. Like mentioned for time and deadlines, the project team needs to agree on the level of tolerance that they consider appropriate for each level of impact, like in the example for costs presented in the Exhibit 11. LEVEL
SCORE
DESCRIPTION
Very High
5
Variation (positive or negative) above $1,000,000.
High
4
Variation (positive or negative) between $500,000 and $1,000,000.
Medium
3
Variation (positive or negative) between $250,000 and $500,000.
Low
2
Variation (positive or negative) between $100,000 and $250,000.
Very Low
1
Variation (positive or negative) lower than $100,000.
Exhibit 11 – Example of impact scale and score for costs
7
8
Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis
Impact on Quality Assesses the level of impact on the quality required for the project. It can be positive or negative for opportunities and threats, respectively. As presented in the other groups, the project team needs to agree on the level of tolerance that they consider appropriate for each level of impact like in the example in Exhibit 12 for negative risk events. LEVEL
SCORE
DESCRIPTION
Very High
5
Client rejects the delivery or product.
High
4
Client asks for immediate corrective actions.
Medium
3
Client perceives and asks for action/ information.
Low
2
Client perceives but forgives and no action is needed.
Very Low
1
Imperceptible impact (most of the time not even perceived by the stakeholders).
Exhibit 12 – Example of impact scale and score for quality (only negative events)
Impact in Safety and Security Assesses the level of impact that the event can incur in safety at work and security. This impact group could include or not aspects related to environment, physical security of the work in the project, data security (IT), and reputation, among others. In the Exhibit 13, an example of scale is presented to assess impacts in safety and security. LEVEL
SCORE
DESCRIPTION
Very High
5
Crisis. Impact is so evident and public that the project could not proceed as planned.
High
4
Evident impact on environment/reputation.
Medium
3
Impact is perceived and raises concerns.
Low
2
Perceived impact on environment/ reputation but without relevance.
Very Low
1
No impact on environment and reputation.
Exhibit 13 – Example of impact scale and score for safety and security, with focus in environment and reputation
Other impacts This group is an optional group and aims to include any other specific impact of a risk that was not covered in the previous groups. It is important that the score of the other impacts, if it exists, should be from 1 to 5 like the other impact groups.
ricardo-vargas.com
Proximity: The 6th Impact Dimension Another dimension of the impact is the time horizon or proximity of the event (Exhibit 14). An event that may happen in hours requires different actions than another event that could impact the project in 2 years. If an event is close to happen, it has a higher priority if compared with future events (in the proximity aspect).
M ER
NEAR
MEDIUM T
IM EDIATE M
ONGͳTER RY L M E V
LONGͳTERM
Exhibit 14 – Understanding the time horizon
The proximity scale should be compatible with the other impact groups (1 to 5 score for different time horizons). It is important that the project team defines what are immediate events, short-term events, medium-term events, long-term events and very long-term events (Exhibit 15). It is important to highlight that immediate events will score higher than very long-term events when assessing their proximity. LEVEL
SCORE
DESCRIPTION
Very High
5
Event can happen anytime in the next 15 days.
High
4
Event can happen between 15 days and 3 months.
Medium
3
Event can happen between 3 and 6 months.
Low
2
Event can happen between 6 months and 1 year.
Very Low
1
Event can happen more than 1 year ahead.
Exhibit 15 – Example of proximity scale and score
9
Tabela 1. Exemplo dos resultados do impacto e a fórmula de cálculo. Copyright 2014 by Área Ricardo Viana Vargas. All Rights Reseved
10
Tempo
Custos
Qualidade
Proteção e Segurança
Outras
Proximidade
3
2
1
1
1
4
impactada Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis Risco A
Tabela 1. Exemplo dos resultados do impacto e a fórmula de cálculo.
Calculating the expected value and final risk assessment
Área impactada Risco A
√
Tempo
Custos Qualidade 3 Proteção 32 + 2 + e1Segurança +used 1 + to 1 assess +Outras 4 and Proximidade The expected value is a = risk measurement prioritize = 2, 31risk events Impact = 6 6 (Exhibit 16).2 3 1 1 1 4
Impact =
2
2
2
2
2
2
Expected Value = Probability x Impact 32 32 + 2 2 + 1 2 + 1 2 + 1 2 + 4 2 = 2, 31 = 6 Mean2 Exhibit 16 6 Quadratic = Arithmetic Mean2 + Variance
√
Using the qualitative the probability Expected method, Value = Probability x Impactwill range from 1 to 5 (Exhibit 8).
Copyright 2014 by Ricardo Viana Vargas. All Rights Reseved
Impact =
Quadratic Mean Arithmetic + Variance The impact is based on=the impactMean in different aspects of the project and the 2 + Imp. on Quality 2 + Imp. on S&Security 2 + Imp. on Other 2 + P roximity Imp. on Tproximity ime2 + Imp. on Costs using a quadratic mean (root square mean) calculation (Exhibit 17). 2
2
6
Tabela 1. Exemplo dos resultados do impacto e a fórmula de cálculo. Imp. on T ime2 + Imp. on Costs2 + Imp. on Quality 2 + Imp. on S&Security 2 + Imp. on Other2 + P roximity 2 Impact = Área de cálculo da média quadrática. 6 Figura 3. Fórmula Tempo Custos Qualidade Proteção e Segurança Outras Proximidade
impactada
A da média quadrática. 3 Figura 3. FórmulaRisco de cálculo
2
1
Exhibit 17
1
1
4
Thequantitativas decision for the quadratic insteadcomo of therolar arithmetic mean(Figura is based??), on mas a maioria e Alguns processos são simplesmean e diretos, um dado √ 2+ 2 + 1add 2 + 1additional 2 + 42 the concept different levels impact exposure toa the project 32 32 + como 2of 1rolar Alguns processos quantitativas sãothat simples e=diretos, dado (Figura maioria envolve uma complexa simulação de cenários como aum Simulação Monte "Monte Carlo" foi um =mas 2, 31Carlo. Impact = de ??), and this variance should be considered6 as a risk factor to the6 project. volvealcunha uma complexa simulação cenários comorelacionado a Simulação ao de Monte Carlo. Carlo" foi umade armas atômic para um projetodeultra-secreto desenho e ao"Monte desenvolvimento Theultra-secreto relationship relacionado between theaoquadratic mean and the arithmetic alcunha para um projeto desenho e ao desenvolvimento de mean armas isatômicas Value = Probability x Impact criadas pelo matemático John vonExpected Neumann (POUNDSTONE, 1993 e VARGAS, 2013). Ele descobriu q
criadas pelo matemático John von Neumann (POUNDSTONE, 1993 e VARGAS, 2013). Ele descobriu que Quadratic Mean2 poderia = Arithmetic Mean2 + Variance um simples modelo de amostras randômicas solucionar certos problemas matemáticos, que n um simples modelo de amostras randômicas poderia solucionar certos problemas matemáticos, que não podiam ser solucionados até aquele momento. Exhibit 18 podiam ser solucionados até aquele momento.
where the variance is a measure of how far a set of numbers is spread out A simulação se refere, no entanto, a um método pelo qual ados distribuição dos resultados possíveis é p A simulação se refere, no entanto, a um método pelo qual a distribuição resultados possíveis é proImp. on T ime2 + Imp. on Costs2 + Imp. on Quality 2 + Imp. on S&Security 2 + Imp. on Other2 + P roximity 2 Impact = The variance concept is directly related to of the different impact duzidas através de sucessivos recálculos dos dados do projeto, o desenvolvimento de múltiplos 6 dispersion duzidas através de sucessivos recálculos dos dadospermitindo dothe projeto, permitindo o desenvolvimento de múltip Copyright 2014 by Ricardo Viana Vargas. All Rights Reseved groups. If the impact ranges are very wide, the variance will also be high and the cenários. Em cada um dos cálculos, novos dados randômicos são usados para representar um processo
cenários. Em cada um dos cálculos, novos dados randômicos são usados para representar um proces
difference between the proposed quadratic mean and the traditional arithmetic Figura 3.e interativo. Fórmula de cálculo da média quadrática. repetitivo A combinação de todos estes resultados cria uma distribuição probabilística dos
repetitivo e interativo. combinação de the todos resultados cria uma distribuição probabilística d mean will A increase, increasing riskestes impact.
resultados e 6). Tabela 1. (Figuras Exemplo 5dos resultados do impacto e a fórmula de cálculo.
resultados (Figuras 5 e 6). of the impact results is presented in the Exhibit 19. One example Alguns processos quantitativas são simples e diretos, como rolar um dado (Figura ??), mas a maioria en Área impactada
Tempo
volve uma complexa
Custos
IMPACT TIMEde simulação
QualidadeIMPACT Proteção e Segurança IMPACT OTHER
IMPACT COST cenários
Outras
Proximidade
QUALITY S&S IMPACT como a Simulação de MontePROXIMITY Carlo. "Monte Carlo" foi um
Risco A 3 2 1 1 1 4 A 3 2relacionado 1 1 4 alcunha umRisk projeto ao desenho e 1aooudesenvolvimento de armas atômica **Figura 5** para – Construção de umultra-secreto modelo de distribuição de custos e atividades, pacotes de trabalho,
e2001). √ aleatórios gerando umapelo distribuição final a partir do projeto (PRITCHARD, criadas John von Neumann (POUNDSTONE, 1993 2013). Ele descobriu qu 2+ **Figura 5**matemático – Construção de de umdados modelo distribuição atividades, ou pacotes de trabalh 32 + 2de 12 + 12 + 12 +de 42custos 32eVARGAS, Impact = = = 2, 31 6 6 problemas matemáticos, que nã um simples modelo amostras poderia solucionar certos **Figura 6** – uma Exemplo dade simulação dearandômicas Monte para avaliar o impacto no custo de uma possível gerando distribuição final partirCarlo de dados aleatórios do projeto (PRITCHARD, 2001). Exhibit 19 ameaça ao projeto. podiam ser solucionados até aquele Expected momento. Value = Probability x Impact
**Figura 6** – It Exemplo da simulação Carloand para avaliar o can impacto no custo de uma possív is important to highlightde thatMonte the threats opportunities be calculated
2 Em razão da análise quantitativa ser baseada na matemática e na estatística Quadratic Mean2 = Arithmetic Meansuportadas + Variancepor métricas obje-
Aameaça simulação se refere, nosame entanto, a um pelo signals qual a (+ distribuição dos resultados using the formula, butmétodo with different for opportunities and – for possíveis é pro ao projeto. tivas, tais análises são threats). consideradas mais qualitative rigorosas (SMOCK, 2020). of Os the principais desafios de uma by análise The total exposure project is determined the duzidas através de sucessivos recálculos dosrisk dados do projeto, permitindo o desenvolvimento de múltiplo Em razão da são análise quantitativa ser baseada na matemática e na estatística suportadas quantitativa sólida o tempo e o esforço requeridos para a sua execução e também o requerido conhecsum of the expected values of all threats and opportunities. An example of this por métricas ob cenários. Em cada um dos cálculos, novos dados randômicos são usados para representar um process process is presented on the Exhibit 20. tivas, Imp. tais análises consideradas Os principais desafios deP uma anál 2 + Imp. on 2 on T ime2são + Imp. on Costs2 +mais Imp. rigorosas on Quality 2(SMOCK, + Imp. on2020). S&Security Other2 + roximity 3 Impact = repetitivo e interativo. A combinação de todos estes resultados cria uma distribuição probabilística do quantitativa sólida são o tempo e o esforço requeridos6para a sua execução e também o requerido conh resultados (Figuras 5 e 6). Figura 3. Fórmula de cálculo da média quadrática.
ricardo-vargas.com 
TYPE
PROBABILITY
PROXIMITY
IMPACT TIME
IMPACT COST
IMPACT QUALITY
IMPACT SAFETY AND SECURITY
OTHER IMPACTS
TOTAL IMPACT
EXPECTED VALUE
Threat
1
3
2
1
1
1
4
2,31
(2,31)
Threat
2
4
4
3
3
4
3
3,54
(7,07)
Threat
2
3
5
4
4
5
1
3,92
(7,83)
Opportunity
3
2
4
3
5
4
2
3,51
10,54
Opportunity
4
1
3
2
4
3
2
2,68
10,71
Threat
5
2
2
1
3
1
1
1,83
(9,13)
Total Risk Expected Value Exhibit 20 – Example of a project expected risk value considering opportunities and threats.
The results from the process will be always a number between 1 and 25. In the example of Exhibit 20, the value -5,10 is equivalent to 20,4% negative exposure (5,10/25) for the Project. Based on this result and the tolerance thresholds (HILSON & MURRAY-WEBSTER, 2007), the total exposure can be compared with other projects and the corporate limits to define potential risk response plans.
Conclusions The qualitative risk method is always a simplified model if compared with the quantitative methods. The approach of this paper suggests an alternative model that can be tailored to include different kinds of impacts and scales in order to produce a reliable quantitative result. This result allows opportunities and threats to be compared in order to determine the total risk exposure. The concept that an opportunity can cancel a threat of the same level is not possible with the traditional qualitative risk management approach.
References ALTENBACH, T. J. (1995). A Comparison of Risk Assessment Techniques from Qualitative to Quantitative. Honolulu, ASME Preassure Vessels and Piping Conference. Available at http://www.osti.gov/bridge/servlets/purl/67753-dsZ0vB/webviewable/67753.pdf HILSON, D. & MURRAY-WEBSTER, R. (2007). Understanding and Managing Risk Attitude. London: Gower Publishing. PMI (2013). The Project Management Body of Knowledge: Fifth Edition. Newtown Square: Project Management Institute.
(5,10)
11
12
Adopting the Quadratic Mean Process to Quantify the Qualitative Risk Analysis
POUNDSTONE, W (1993). Prisoner’s Dillema. Flushing: Anchor Publishing Group. PRITCHARD, C. L. (2001). Risk Management: Concepts and Guidance. 2ª Ed. Arlington: ESI International. ROSSI, P. (2007). How to link the qualitative and the quantitative risk assessment. Budapest: Project Management Institute Global Congress EMEA. ROT, A. (2008). IT Risk Assessment: Quantitative and Qualitative Approach. San Francisco: World Congress on Engineering and Computer Science. SMOCK, R. (2002). Reducing Subjectivity in Qualitative Risk Assessments. Bethesda, SANS Institute. Available at http://www.giac.org/paper/gsec/2014/reducing-subjectivity-qualitative-risk-assessments/103489 VARGAS, R. V. (2013). Determining the Mathematical ROI of a Project Management Implementation. New Orleans, Project Management Institute Global Congress North America. VIRINE, L. (2010). Project Risk Analysis: How Ignoring It Will Lead to Project Failures. Washington: Project Management Institute Global Congress North America.