Uniwersytet Gdaski
Summary
The paper is targeted at the comparative analysis of practical applications of risk assessment methods of projects in IT industry. The study was based on an IT Project comprising 349 activities, wherein 8 were selected which were the most likely not to comply with the scheduled time, scope and budget resources. Next, four techniques were described and apply to provide the assessment of the same part of the project: 2x2 Matrix Method, Probability and Effects Matrix Method, Heeg’s Method, Failure Mode Effects Analysis. The obtained results were discussed and on the basis thereof, conclusion were formulated to select the method depending on the specificity of an IT Project.
Keywords: IT projects, risk management, risk assessment techniques
1. Introduction
IT projects implementation as described in software development methodologies is usually dis-tributed into several stages. It begins with requirements specifications and ends with operation and maintenance of the implemented product [1]. Throughout the course of software development cy-cle the events affecting its course may take place. In practice, they are described by probability of their occurrence and the potential scope of damage they may create in the implemented project. The product of these quantities is defined as the risk measure [3]. It is used for forecasting negative impacts on the course of the implemented individual activities within the task or the entire project. Project management is the approach oriented at the accomplishment of the set objectives within the assumed time and budget [4]. It is an art of maintaining the project failure risk at a possible lowest level throughout the entire project cycle. Risk management is one of many elements of the project management process. In principle, it is distributed into the following stages [2]: identification and distribution of risk sources, identification of exposed project tasks, risk assessment, planning of response to the risk and risk and risk monitoring in the course of project implementation.
2. Failures in implementation of IT projects
There are scores of institutions acting in the field of risk analysis in IT projects. These are mainly academic centres but there are also many organizations associating experts with know-how. One of the most often cited publications are reports called The CHAOS Chronicles, published regularly by The Standish Group International, an American institution dealing with monitoring of IT projects implemented in the USA. The analysis of documents published on the Web fosters de-tailed specification of the results of IT projects monitoring in the form of statistics.
2/3 of IT projects fail to end in a full success. Throughout their implementation (according to the statistics in every second project), there are deviations from time schedule or budget assumptions, failure to develop all the designed functions of the programme or abandonment of the entire pro-ject. The main objective of the risk assessment at the planning and implementation stages of IT projects is to reduce the number of projects which are likely to exceed the scheduled resources or be eventually terminated [9].
The risk assessment techniques presented in the paper compel the IT Project Managers to con-tinuously update the information on potential hazards. One of the frequent mistakes made by IT Project Managers is to assume that the risk of successful accomplishment of the project remains the same throughout the entire project of its implementation [6]. It is to the contrary. The risk is inces-santly variable, thus constant monitoring as per a carefully designed schedule is of essence. As shown in practice, the meticulousness and thoroughness of the risk assessment contribute consid-erably to the end success of the project.
3. Characteristics of the selected IT project
The study was based on a project of system development aimed at servicing the Lending Li-brary of Higher Education School. The undertaking in question was designed as a set of 349 tasks to be carried out by a group of 5 IT Specialists within 64 days. The accomplished and implemented software is to facilitate students’ access to the book catalogue by means of web browsers.
Table 1. Selected risk-prone projects tasks ID Task name: Duration Probability of non-compliance with the
scheduled resources Implementation costs [PLN] 40 Requirements Specifi-cation 14 days 0,4 4500
43 Definition of classes 3 days 0,3 800
52 Code development 30 days 0,5 10 300
67 Code testing 5 days 0,6 4 400
104 Code adjustment 10 days 0,5 7 300
109 Software installation 30
min-utes 0,3 500
110 Preparation of user documentation
10 days
0,3 5 500
111 User training 2 days 0,2 4 800
Source: own computations
The detailed comparative analyses were based on eight tasks from the entire project exposed to the risk of failure, taking into account the scheduled time, scope and budget (Table 1).
4. Examples of application of risk assessment techniques in the selected IT Project – case study
The popular risk assessment methods applied to IT Projects comprise: 2x2 matrix method, probability and effects matrix method, Heeg’s method, failure Analysis of failure effects. Less popular techniques encompass: sensitivity analysis, spot techniques, probability analysis (using e.g. Monte Carlo simulation), flow diagrams (e.g. critical path analysis) or decision tree analysis
(e.g. PERT, VERT, GERT analyses).
4.1 2x2 Matrix Method
One of the primary tools fostering risk management process in project works is the so-called 2x2 matrix. It defines risk as the probability function for the occurrence of harmful event and the effect thereof [7].
Table 2. Risk assessment using 2x2 Matrix method Impact
Probability Small Large
Large Quarter 1 40’ Quarter 2 40 Small Quarter 3 40’’ Quarter 1
Source: own computations
2x2 Matrix should be completed following the previous identification of potential hazards and preparation of the list thereof. Next, they are filled (depending on the probability of occurrence and the scope of potential loss they may cause) in their respective matrix quarters. The parts of the ta-ble specify [2]:
• Quarter 1 represents the area of hazards of high probability of occurrence and inconsider-able negative effects for the project,
• Quarter 2 represents the area of hazards of high probability of occurrence and simultane-ously substantial negative effects for the project implementation process,
• Quarter 3 represents the area of hazards of small probability of occurrence and inconsid-erable negative effects for the project implementation process. This is the least risk-prone area.
• Quarter 4 represents the area of hazards of small probability of occurrence and consider-able negative effects for the project,
If the 2x2 table is not completed with hazards but with the tasks exposed to the risk of non-compliance with the scheduled resources, then after the analysis of the possibility of reallocation thereof along the directions recommended by the technique in question, precautions will be devel-oped with the view of diminishing the risk for the accomplishment of particular tasks and thus the entire project.
Task 40 – requirements specification was placed in the 2nd quarter by the project risk manager. Next, the Project Manger suggested that Task 40 should be performed in compliance with the forms adopted in PRINCE2 method. The suggestion caused the task to be moved to the first quar-ter 40’ (prim variant). Next, another Project Manager decided to carry out additional audit of the prepared requirements specification by an external expert. This decision caused the task 40’’ to be moved to the third quarter 40’ (bis variant). All project tasks may be examined individually in the same manner.
4.2 Method of Matrix of probability and effects
Another more complex tool is the so-called matrix of probability and effects. It is an elabora-tion of the 2x2 matrix concept. It is more detailed as far as the probability of estimates and the ef-fects of hazard occurrence are concerned. Like previously, particular hazards are filled in the re-spective fields in the extended version of the table. After all hazards identified in the project have been filled, preventive measures are designed to eliminate the risk sources allocated in the second quarter [2]. The ultimate objective of the risk management process is the reallocation of the most probable and most perilous hazards top other areas of the matrix. The measures provide the basis for the risk management in the project planning process.
The table used for the method in question may upon its modification serve for calculation of the measurable total scope of project risk. The aforesaid modification provides for allocation to par-ticular cells of hazard weights representing the scope of probability of occurrence of a given hazard and the potential effects thereof (Table 3).
Each identified hazard, which may occur during performance of the project shall be allocated to particular cells of the table. As a next step, the weight of a given cell should be multiplied by the number of hazards allocated thereto and sum up all the achieved numbers. The sum shall be di-vided by the total number of hazards in the analysed project. The end result is a measurable quan-tity of the total project risk.
Table 3. Risk Assessment Method of Matrix of Probability and Effects Effects
Probability
Minimal Minimal Minimal Minimal Minimal
Extremely high (0.8 – 1) 43 (2,0) 40 (3,5) (7,0) 67 (8,0) (9,0) High (0.6 – 0.8) (1,5) 52’ (2,0) 52 (5,0) (7,0) (8,0) Average (0.4 – 0.6) 43’ (1,2) 67’ (1,8) 104 (4,0) (5,0) (7,0) Low (0.2 – 0.4) 40’, 104’ (1,0) 110, 109 (1,5) (3,0) (4,0) (5,0) Extremely low (0 – 0.2) 110’,109’, 111’ (0,5) 111 (1,0) (1,5) (3,0) (4,0)
Source: own computations
With respect to the project, the entire introductory risk of the project was 3.31. After the pre-ventive measures have been recommended and applied, the total project-related risk was reduced to 1. The presented example, like 2x2 matrix method was based on the study of risk-prone tasks and not the tasks themselves.
4.3 Heeg’s method
The method recommended by Heeg in [2] comprises three stages. These include: • risk identification,
• risk assessment, • selection.
the author of the method, the risk sources may be the identified in several ways. One of the com-monly used techniques is the analysis of task packages described by means of e.g. Work Break-down Structure - WBS. It may be presented in the form of a table 4.
Table 4. Risk assessment using Heeg’s method
ID Task name Potential risks
Probabil-ity of occur-rence Costs of neutrali-zation [PLN] Probable costs [PLN] 40 Requirements Specification Omission of required functionalities 0,4 8000 3200 43 Definition of classes Incomplete classes 0,2 1000 200 52 Code development Syntactic errors 0,01 500 5 67 Code testing Data transfer errors 0,3 4000 1200 10
4 Code adjustment Semantic errors 0,04 300 12 10
9 Software installation Incompatibility 0,1 2000 200 11
0
Preparation of user
documentation Deadline 0,01 3000 30
11
1 User training Deadline 0,01 1500 15
Source: own computations
Following identification of the risk-prone tasks and detailed specification of potential risk sources, which may affect the implementation process, it is necessary to determine the probability of occurrence of detailed hazards (Table 4, Column 4). Next, the planned costs related to elimina-tion of potential losses are to be estimated (Table 4, Column 5). The last column of Table 4 com-prises probable costs i.e. product of probability and foreseen costs of loss compensation (Table 4, Columns 4 and 5).
Thus computed quantities of probable costs must be sorted in descending order and the group of tasks for which the sum of quantities in Column 4 Table 4 will be 75% of the total probable costs of the analysed project [2] must be specified (starting from the highest values). In the exam-ple in question, these include tasks 40 and 67 (amounting to 90.5% of the total costs). Thus identi-fied set of task groups shall be given a particular attention from the Project Managers. The possi-bility to undertake protective measures for these groups must be taken into consideration. The sum of total probable costs shall be 4862 PLN.
4.4 Failure Mode Effect Analysis
Failure mode effect analysis was proposed by Maylor and described in [1]. This method ana-lyzes three parameters describing all tasks within the project. Each of these parameters must be expresses as a number on a scale from 0 to 10. The author adopts one point scale for all parame-ters. The requested quantities include:
• meaning of failure of implementation of a given task (failure), • probability failure oversight,
• probability of failure occurrence during performance of a particular task.
Each of the parameters must be examined individually. The objective of the presented analysis is to calculate a given total risk task constituting a function dependant on the aforementioned pa-rameters. The risk is calculated on the basis of the following dependence:
Risk = failure significance * probability of failure omission * probability of failure occurrence
The higher the risk values, the more serious hazard is related to a particular task. With respect to activities exposed to the highest risk, additional measures alleviating potential losses should be proposed.
Table 5. Risk Assessment Through Failure Effects Analysis ID Task name Failure
sig-nificance Probability of failure oversight Probability of failure occurrence Risk 40 Requirements Specification 8 3 4 96 43 Definition of classes 3 2 3 18 52 Code development 7 1 5 35 67 Code testing 8 2 6 96 104 Code adjustment 7 2 5 70 109 Software installa-tion 5 1 3 15 110 Preparation of user documentation 4 7 3 83 111 User training 5 1 2 10
Source: own computations
For each task examined in the project by Failure Mode Effect Analysis, two additional parame-ters must be provided. These include: failure significance and failure oversight probability. Having performed the calculations illustrated in Table 5, one may discern that the highest risk pertains to tasks 40 and 67, while the least to the tasks no. 109 and 18. The total project risk is the sum of val-ues in the last column, which is 423.
5. Comparative analysis of applications of risk measurement methods in IT projects
Applications of the results obtained by means of 2x2 Matrix Method are not vast. This method is suitable for presentation of risk mitigation issues, since it clearly illustrates the required trends of preventive measures. IT project risk assessment based on this method may occur vague, general and eventually not yielding satisfactory results for the Managers. The concept of 2x2 matrix is fo-cusing on risks, not the risk-prone tasks, which definitely affects the profile of the analyses being carried out. 2x2 matrix does not allow risk quantification, with respect either to a part or the entire project. This technique is an easy-to-use tool for risk assessment in small projects. Its application supports the strategy of compensating potential effects of identified risks.
assumptions of the method have been applied, the effect of its use shall consist in development of preventive measures plan targeted at reduction of risks for individual tasks, hence for the entire project. Thus applied 2x2 matrix method will facilitate the assessment of the proposed preventive measures (after twice risk quantity measurement, before and after the preventive measures have been taken). Those who wish to use this technique shall be good experts in risk management, as this technique is based on the intuition, which determinant for the usability of the obtained results.
Probability and effects matrix is the extended version of 2x2 matrix method. It has two advan-tages, which differentiate it significantly from the original pattern. One advantage is the fact that it fosters calculation of risk for individual tasks, task groups or the entire project, before and after the preventive measures have been taken. The other advantage is the clarity of results, not only for small but also for medium-sized IT projects.
Heeg’s method is the first from among the presented techniques, which assigns individual risks to the planned project tasks. Each of the tasks may be assigned more than one risk. This technique requires specification of the probability of occurrence of all identified risks. It is an interesting pa-rameter, since within the framework of the risk definition another quantity is searched for, which is the probability of task non-performance [5]. In Heeg’s method it is indispensable to specify poten-tial costs of the reduction of effects of the identified risk occurrence. In practice, both quantities are identified on the basis of experience and intuition of the researchers. The method facilitates a de-tailed identification and analysis of risk sources, which may occur during the implementation of the project. By means of these method, the sources may be easily identified and assigned to particular tasks. This possibility is an essential advantage of the method in question. Unfortunately, this method also employs heuristic quantities in final risk assessment.
In failure mode effects analysis it is necessary to specify further parameters. These parameters are not required in any other risk assessment techniques. It is indispensable to determine: failure significance, failure oversight probability as well as failure occurrence probability. The last pa-rameter is identical to the scope of risk of a non-performed task taking into consideration sched-uled time resources, scope and budget. The person managing the risk must express all these values on a scale from 0 to 10.
Failure mode effects analysis does not employ mathematical tools facilitating objectivity of data use for calculations. Like the aforementioned methods, the values used are based on the intuition. It will prove, however, in comparative analyses. From among the presented methods, this one al-lows the largest number of details to be used in the study. It occurs that Managers value the possi-bility of taking into account the probapossi-bility of failure oversight. Failure mode effects analysis may be easily used for the risk assessment in large and middle-sized IT projects.
Table 6. Comparative applications of risk assessment applications IT projects Required resources Results
Method name Co st s D u ra ti o n o f im p le m en ta -ti o n E as in es s o f ap p li ca ti o n T im e in v o lv em en t P re ci si o n U sa b il it y 2x2 Matrix Method l s e s l l
Method of Matrix of Probability and Effects l s e m m m
Heeg’s Method l s m h m h
Failure Mode Effect Analysis l s d m h h Key: low (l), medium (m), high (h), short (s), easy (e), difficult (d)
Source: own computations
6. Summary
The paper depicted four methods of risk assessment methods applied in IT projects implementation. The same part of the IT project was examined through a comparative analysis. The obtained results foster concept that matrix techniques focus predominantly on the analysis of identified risks. They only indirectly examine the project tasks, for which it is probable, not comply with the scheduled time resources, scope or budget. As a consequence, although these methods mark out the directions of preventive measures aimed at reduction of the risk, they seem not to be useful for large or complex IT projects. They are suitable for rough analyses in small projects.
Further two methods foster a more detailed risk assessment. Based on the identified potential risks assigned to particular project tasks, Heeg’s Method specifies probable costs, the company will have to incur in case of an anticipated risk. This project does not foster total risk calculation. This technique is suitable for comparative analyses for several variants of implementation of the same task, while it is not appropriate for the entire IT projects.
The last presented technique – Failure Mode Effect Analysis introduces two additional (crucial) parameters: failure significance and failure oversight probability. Thanks to these parameters, the method enables calculation of risk for implementation of all individual tasks as well as the entire project. This technique is commonly used in IT project risk management as the calculations involving two new parameters are very useful.
The Project Manager’s ultimate decision on the choice of the method for risk assessment in planning and implementation of an IT project will depends first and foremost on the specificity of project requirements (scope and innovativeness), funds and selected implementation methods.
7. Literature
1. Chapman Ch., Ward S. (1997): Project risk management processes, techniques and in-sights, J Wiley & Sons, Chichester.
2. Chong Y,Y, Brown M.E. (2001): Zarzdzanie ryzykiem projektu, Oficyna Ekonomiczna, Dom wydawniczy ABC, Kraków.
3. Frczkowski K. (2003): Zarzdzanie projektem informatycznym, Oficyna Wydawnicza Politechniki Wrocławskiej, Wrocław.
4. Kaczmarek T.T. (2005): Ryzyko i zarzdzanie ryzykiem. Ujcie interdyscyplinarne, Di-fin, Warszawa.
5. Knight F. (1933): Risk, uncertainty and profit, London.
6. Pakowska M. (2001): Zarzdzanie zasobami informatycznymi, Difin, Warszawa. 7. Pritchard C.L. (2002): Zarzdzanie ryzykiem w projektach. Teoria i praktyka, WIG –
PRESS, Warszawa.
8. Stabryła A. (2006): Zarzdzanie projektami ekonomicznymi i organizacyjnymi, Wydaw-nictwo Naukowe PWN, Warszawa.
9. Szyjewski Z. (2004): Metodyki zarzdzania projektami informatycznymi, Wydawnictwo PLACET, Warszawa.
10. Winiarski J. (2007): Analiza metod zarzdzania ryzykiem w pracach projektowych z dzie-dziny informatyki, Pienidze i Wi , Nr 2 (35), Gdask.
Jacek Winiarski Uniwersytet Gdaski
81-824 Sopot, ul. Armii Krajowej 119/121 e-mail: Jacek.Winiarski@univ.gda.pl http://ekonom.ug.gda.pl/
