Next Article in Journal
Evaluating the Thermal Pollution Caused by Wastewaters Discharged from a Chain of Coal-Fired Power Plants along a River
Previous Article in Journal
Restoring Soil Quality to Mitigate Soil Degradation
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Risk Leveling in Program Environments—A Structured Approach for Program Risk Management

School of Economics and Management, Beijing University of Posts and Telecommunications, No. 10 Xitucheng Road, Haidian District, Beijing 100786, China
Federal University of Para, Campus of Abaetetuba, Para 68440-000, Brazil
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Sustainability 2015, 7(5), 5896-5919;
Received: 10 December 2014 / Revised: 10 April 2015 / Accepted: 14 April 2015 / Published: 13 May 2015


Programs are demarcated as administrative structures established to realize planned organizational strategies through multi-project activities. Programs occupy a distinct locus in organizational hierarchy, so therefore necessitate specialized management approaches. Risks in programs tend to widen the gap between the organizational plans and the actual program realizations. However, effective risk management can minimize these gaps. This research frames a structured approach for program risk management, called Risk Leveling in Program Environments (RLPE), which suggests (a) a deliberate shift of risks to the right organizational level where they can be addressed most effectively; and (b) a unique procedure for risk management, which attempts to stabilize the risky contexts in programs. RLPE tracks the standard risk management process, preserves distinct program locus, and employs certain qualitative and quantitative measures to achieve risk leveled environments for program success. It has been demonstrated how certain tools and concepts, such as Analytical Hierarchy Process (AHP), As Low As Reasonably Practicable (ALARP), standard deviation, etc. can be employed for risk oriented decision making in programs. RLPE is an instrumental approach, which can help the policy makers in controlling the risky contexts thereby providing sustainable growth for development programs. The offered approach can be particularly advantageous for risk management in large-scale (development) programs.

Graphical Abstract

1. Introduction

The presence of risk and its effects on the business context are increasingly discussed, given the growing number of publications on the subject in recent years. A common belief recognizes risk as the impact of uncertainties on achievement of objectives [1]. Aven [2] presents a comprehensive review of literature that demonstrates a diversity of uses of the term “risk” in the professional, scientific and academic environment; he outlines its connotation with varied perspectives including expected value (loss), probability of an (undesirable) event, potential of loss, and consequences of uncertainty. It is understood that risk is inevitable [3], and eliminating risks is not an easy task. However, establishing and executing suitable response plans, continually helps in mitigating and minimizing the negative upshots; such doings also nurture enhanced chances to exploit the opportunities [4]. Although risk is also considered positive in some contemporaneous developments about risk management, for reader’s clarity, this paper considers risk in the perspectives of bearing negative influences on business objectives, only. At this point, it is useful to highlight the difference between the terms risk and uncertainty. In the context of risk management, uncertainty is defined as the lack of knowledge about future, while risk is described as an uncertain event that can affect (to some extent) the objectives of certain activity (e.g., program management) [5]. Nonetheless, quite often the terms risk and uncertainty are used interchangeably.
A program is an organizationally established structure to give strategic direction to a set of related projects through coordination and other activities [6,7]. In this context, Project Management Institute (PMI) explains risk as an occasion or series of occasions or circumstances that may (positively or negatively) affect the accomplishments of the program, in case they happen [8,9]. International Organization for Standardization (ISO) describes risk management as coordinated activities to direct and control an organization with regard to risk, and standardizes the risk management process to essentially embrace communication, context building, identification, assessment, treatment, and monitoring and review of the potential risks [10]. Programs vary in nature, sizes, and types. For instance, some programs are commercial in nature while some others are governmental. Similarly their sizes may range from small scales to medium and large scales. Likewise, they bear different types based on how the programs are triggered; for example the vision lead programs, which start with a clear strategic vision; the emergent programs, which evolve from current uncoordinated initiatives; and the compliance programs that are a “must do” for the organization without any choice [11]. Moreover, the deliverables in programs may range from developing certain products or services to achieving organizational change and social transformations [11,12]. Risks in the external (macro) and internal (micro) environments continually challenge the sustenance and development of all these kinds of endeavors [10].
Since program management is still an emerging domain [13] lacking a clear consensus on its fundamentals (including definitions) and practices, the risk management function inside programs is even less established [1]. Leading efforts on risk management are devoted to projects, while the work done in area of program risk management is quite insufficient [7]. For example, program risk management tools and methodologies are not well explored even in recent literature (see e.g., [1,3,9,14]. Despite the individual characteristics of programs (for example their strategic personality and the hierarchal emplacement), not enough is pursued to conceive specialized approaches for this specific field.
In the relevant studies we observe certain deficiencies on the part of risk management approaches in programs.
Risk management methods in general do not maintain the distinct locus of the programs in organizational hierarchy. Program risk management is traditionally observed and treated as an extension of the project risk management discipline [15,16]. This initial perception is common because of the similarity in their development stages [9]; however, such concepts do not sustain longer as the goals of programs are different from those of projects [15]. Since the programs observe a well-defined level (in between the projects and the organizational strategy), program risk management justifies being a distinct domain; the project mindsets for them should be avoid as such.
Majority of the methods obtainable on risk practices are either industry-specialized or project-specific; it is hard to validate them in other disciplines. For example, the methodology presented by Alises is decidedly specific to port infrastructures and overtopping risk assessments [17]. Likewise, all the risk management methods reviewed by Stern [18] are software development or IT centric.
The risk methodologies and processes found in the literature are sometimes too philosophic or conceptual to be used or followed in practice. Some examples for reference are the work of Ahmed [19] on a risk analysis tool at the conceptual level, a theoretical research by Kwan [20] in which he investigates risk interdependencies among multiple projects within a program, etc. Resultantly, the practitioners lack credence in terms of hands-on compliance.
Finally, the mainstream methods hardly touch the end-to-end risk management cycle. The majority of the accessible studies treat risk recognition, assessment or treatment in a piecewise formation. Integrated methods that can address risk management holistically remain scarce. Some integrated efforts demonstrate good insight (see for example Dey’s work [21]) but their scope is restricted to project levels, and they cannot be applied in program environments as such.
All such deficiencies build a research question for this paper and necessitate an organized approach, which provides an integrated way to exercise risk management in programs. This study provides a procedural sequence of actions to profile identification, assessment, evaluation, and control of risks in programs in a systematic way; we name this structured procedure “Risk Leveling in Program Environments (RLPE)”. A distinguishable feature of this approach is that it safeguards the distinct standing of programs inside management hierarchy and creates the mechanism of “Risk Routing”. Risk Routing transfers pertinent risks to appropriate level up or down in the organizational layers for desirable mitigation at the right level. RLPE also suggests “Risk Leveling”, which attempts to level the risks mutually and reduces the risks to match them with the tolerance levels of program components; this way it creates risk-settled environments for program success. In addition to using simple, comprehensible and extensively recognized tools, this proactive approach follows the standard risk management process during its course to control program risks.
The rest of the paper is organized as follows. Section 2 discusses the characteristics of risk and risk management in programs, highlights the unique standing of program in organizational hierarchy, and provides the concept of Risk Routing. Section 3 details Risk Leveling as a structured approach for risk management in programs before Section 4 outlines a relevant case study. Section 5 presents some discussion and possibilities. Finally Section 6 summarizes and concludes the overall work presented.

2. Implications of Risk Management for Programs

Programs, specifically the large-scale and multifaceted ones, frequently encounter (unanticipated) threats on account of deprived risk management and demonstrate impotency in meeting the envisioned objectives. Implications of risk management in programs are signified from the cross-industry survey, concluded by joint MIT-PMI-INCOSE (Massachusetts Institute of Technology-Project Management Institute-International Council on Systems Engineering) Community of Practice, where they evaluate almost 120 (large-scale) programs and finally declare “lack of proactive program risk management” as one of the “top 10 themes of challenges in managing (engineering) programs” [12].
A program is a framework that gathers a set of projects with related objectives under a single management unit. It intends to achieve two kinds of goals: efficiency related goals and business focus goals [22]. The efficiency related goals not only ensure that the component projects achieve their desired outputs in isolation, but they also guarantee that mutual synergies between the projects are exploited to conserve the organizational resources [6,9,23,24], On the other hand, the business focus goals tend to create benefits for the organization through better alignment of these projects with the requirements, drivers, culture, and strategy of the wider organization [9,22]. Project Management Institute (PMI) maintains that the ultimate goal of a program is to realize the organizational strategic benefits through aligning its components’ goals with the overall organizational goals [9].
Programs as business implementation vehicles are exposed to risks of several extents [9,11,25] originating from organizational, technical, political, economic, social, and market sources. Risks stemming from these diverse forces hamper the program’s manageability and jeopardize the program goals. For similar reasons, different professional bodies (e.g., Project Management Institute USA, Office of Government Commerce UK, Project Management Association of Japan, International Project Management Association Netherland, etc.) categorize risk management to be an essential ingredient of program management.

2.1. Sources of Risk in Programs

The mainstream of researchers agree that programs are means of implementing organizational strategies and obtaining benefits through projects and actions (see for example [14,16,22,25,26]). On the other hand, program results are influenced by the contextual factors surrounding its component projects [16]. The existence of internal and external risk factors and the fact that a program is embraced by uncertainties over its entire life cycle, affects the ability of the program to achieve its goals. Figure 1 below shows the link between the contextual risks and the program goals.
Figure 1. Risks and the program.
Figure 1. Risks and the program.
Sustainability 07 05896 g001
ISO categorizes these contexts in two broad ranks: the External and the Internal [10,27]. The external context is founded on the macro environment factors in which the programs operate, whereas the internal context comprises indigenous features affecting the organization. Both the external and the internal environments manifest risk forces, which continually challenge the sustainability of programs. Risk management is poised to compete with both these challenges, concurrently. Dominant streams of researchers highlight the greater importance of the environments surrounding the multi-project components and lay emphasis on managing contextual factors [22,28]. In related literatures, program context has been defined as the dynamic political, cultural, and business environment in which program operates [16,29].
Pellegrinelli categorizes risk management as one of the essential skills to manage complex programs and upholds that in the contextual perspective, the program directors provide shelter to the projects from turbulent and uncertain environments [28]. Similarly, Shao describes that the stability of program context is a prime source for the program success [29,30]. It is thus concluded that risk controlled environments in programs can provide an ideal basis for bridging the gap between planned strategy and benefit recognition. In views of PMI [8], for a successful program proposition, a balance between the risk bearing capacities of the components and the risk levels surrounding them should be sought. One possible way to proceed with is to enhance the risk bearing capacities of the program entities enabling them better contend the challenges, yet another option may be to achieve the desired balance by normalizing the risky environments itself.

2.2. Distinct Program Locus and Risk Routing

Since program sits between component and strategy levels, besides other sources, the program may receive risks from the contagious management layers as well. D. Hillson [31] recognizes risk as a dynamic challenge inside programs since it has tendency to emerge from multiple directions. He recommends regular risk reviews at the program level and warns that the scopes of these reviews necessitate meticulous controls to avoid deliberating component project risks or strategic risks exterior to the scope of program. In addition to explicit responses (avoid/transfer/mitigate/accept) aimed at threats, further risk response choices accessible at program level comprise escalation of certain risks to strategic level and delegation of certain others to component levels [1,9] (p. 189). We term this transference phenomenon as “Risk Routing”. Risk Routing essentially conserves distinct program locus, which leads to an improved focus on the pertinent program issues. Figure 2 portrays distinct locus of program, the interfacing between contagious layers and the concept of Risk Routing.
Figure 2. Program locus and risk routing.
Figure 2. Program locus and risk routing.
Sustainability 07 05896 g002
Risk Routing is not an independent (or discrete) activity but a complex phenomenon where the decisions depend upon numerous inter-related factors. Routing decisions are predominantly led by the (overall) program contexts, including (but not limited to) the expected risk values (i.e., Composite Risk Index), the respective treatment capabilities, the resource dependencies, the tended relevancies, the frames of time, and managerial preferentialism, etc. The routing decisions are thus not only based on the expected damages alone but are also influenced by the other factors (described above). Moreover, it is highly desirable to understand the connotation of a risk before transferring it to pertinent management layer. At times, “one risk-one layer” trait may not be the right fit, as it may be judgmental to assign a particular risk to a precise layer (i.e., component, program or strategy). In practical belongings there could be overlaps where risks may fall under multiple (organizational) layers with respect to nature, associations and ownerships, and there are thin lines in-between these levels. In such cases, exploitation of multiple response choices is commended where, for example, a part of risk may be treated at program level and outstanding parts may be escalated to either strategic level or delegated to component level or even dealt in both directions concurrently.
To understand the Risk Routing concept and its significance, we sketch the following example. Suppose that the provincial government of a country has assumed a multi-component and multi-stage telecom construction program under which 70% of the divisional population will be served with super-fast wireless Internet service within a five-year period. One part of this (public sector engineering) program is dedicated to wireless infrastructure developments (e.g., base stations, towers, etc.) within administrative divisions, while the other part focuses on laying fiber optic cables (Out Side Plant (OSP)), which will connect all inter-division and intra-division components. The fiber cable will finally be energized through a transmission system (In Side Plant (ISP)). The provincial body (at the program level) has decided to execute the ISP part centrally (due to certain risks, e.g., lack of expertise at project levels, etc.), for which it engaged an international vendor. At the same time it “delegated” the OSP projects to each divisional body (at the project level) for execution (in order to exploit certain opportunity, e.g., abundant human resource at target level). While running an environmental analysis in which projects are being developed, the emergence of slums (riots) in areas surrounding the fiber optic lines (OSP) in certain division(s) (i.e., project level) is detected. Although development of the individual projects is least affected, such risk event can heavily interrupt the stage of system energizing and may jeopardize reaching program objectives. After thorough risk analysis, the program body may have several options to follow. If the risk priority is low, the obvious decision by the provincial body (i.e., program level) is to “delegate” it to the respective divisional (i.e., project) level(s) in order to improve the security of facilities. However, if the risk is potentially serious, in addition to the divisions (i.e., projects), the provincial (i.e., program) body may decide to treat a part of the risk at its own level. In more severe cases, besides the aforesaid responses, a likely decision will be to “escalate” it to higher management level (i.e., the strategy level) where the issues may be resolved though political engagements. A sensible routing, in this manner, can help in managing the risks (at multiple levels) in a systematic way. It is relevant to comment that Risk Routing in itself does not alleviate the risks but provides a fair rationale for mitigating the pertinent risks at the apt organizational level(s).

3. Risk Leveling Approach for Program Risk Management

As discussed earlier, one prime function of program management is to balance the risky environments surrounding the program and its multi-project components. In this pursuance, this section presents Risk Leveling idea, which attempts to accomplish well-balanced, risk-settled, and sustained environment for the program constituents. After having a dedicated focus achieved on program risks through Risk Routing, this approach addresses them in two distinct perspectives synchronously: “leveling down” and “leveling mutually”; “leveling down” focuses on alleviating the risks in absolute terms, whereas “leveling mutually” emphasizes on diminishing their mutual disparities. In the resulting regime, no risks are set to continue to be too big or too small in comparison to the others; also, the residue risks become fairly tolerable for the program entities. Ultimately, the program finds all the risks indifferent as well as within the tolerable limits.

3.1. Risk Leveling in Program Environments (RLPE)—The Methodical Structure

In this segment, we lay the structured methodology that tracks the standard risk management process (shown in Figure 3) and apply simple decision tools to accomplish risk management in programs (through Risk Routing and Risk Leveling).
Figure 4 presents overall scheme of this methodical approach. This tactic provides an organized line of action for program managers on how to spot and measure the risks; how to draw their relative priorities; and how to pacify risks to the desired levels. It also tailors what should be the reference criteria for initiating each explicit measure.
Figure 3. Risk management process (based on ISO [10]).
Figure 3. Risk management process (based on ISO [10]).
Sustainability 07 05896 g003
Figure 4. The methodical approach—risk leveling in program environments (RLPE).
Figure 4. The methodical approach—risk leveling in program environments (RLPE).
Sustainability 07 05896 g004
This procedure employs certain instruments like Brainstorming, Interviewing, Delphi Method, Three Point Estimates Technique, Analytical Hierarchy Process (AHP), As Low as Reasonably Practicable (ALARP) notion and Standard Deviation constructs to draft a decision approach. Notably, the adaptability of this approach is sought by recruiting friendly tools, widely known and practiced in project management disciplines.

3.2. Overview of the Instruments Deployed

Before we may proceed with details of this scheme, it is desirable to equip our readers with the methods used. Each sub-procedure of risk management process accomplishes its goals through certain instruments, which are described below under the relevant (sub-process) headings.

3.2.1. Risk Identification

Risk Identification primarily recognizes and documents which threats may possibly affect a particular program. For risk recognition this approach endorses the formal and informal interviewing and brainstorming techniques, which are comparatively relaxed, widely available, and exceedingly exercised in various management disciplines [32,33]. Application of these instruments leads to a detailed risk breakdown structure [34] for the program. In addition to risk recognitions, said surveys also help in learning possible counter measures, as professed by the experts.

3.2.2. Risk Estimation

For estimation and preliminary risk analysis, which is essentially meant for segregating the higher primacy elements from the rest, this procedure relies on “Three Rounds” blended with “Three Views” schema. Delphi Technique with three rounds attains a possible consensus on probability of happening of a particular risk event, whereas Three Point Estimates provide three diverse views of experts, interpreting impact of each risk on program objectives. Finally their weighted average serves valuation of the severity of each risk.
Delphi has been used by several researchers for risk related assessments (see for example [35]), and many of its variants are already available [36]. Consensus-building, subject anonymity, impartiality, controlled feedback, possibility to use jointly with statistical techniques and flexibility to accommodate geographically dispersed panelists (without any needs of meetings) are counted as advantageous characteristics of Delphi [37,38]. We believe in Delphi for risk assessments since it is a well-established and widely used technique. The typical Delphi technique follows a set of organized, anonymous, multi-round written surveys, where opinions on prospective risks are recorded (from experts) and synthesized. After each round an anonymous summary as well as the accompanied reasoning for the judgments, is provided to the contributors. This encourages improved revisions for next round under influence of the collective judgments of other panelists, and results in converged consensus and better forecasts.
Three Point Estimates is yet another well-known and largely practiced technique. It uses a simple mode of estimation from three dissimilar guesstimates known as O (optimistic), P (pessimistic) and M (most likely) views. It is broadly accredited for constructing an approximate probability distribution built on limited evidence; so it can be used for risk estimations. Project managers particularly rely on PERT (Project Evaluation and Review Technique) estimates, which are based on the same practice. Final estimation is sought by (O + 4M + P)/6.

3.2.3. Risk Analysis

To analyze the assessed risks into deeper levels and to examine risk gravities on truly relative scales, the Multi Criteria Decision Making (MCDM) technique Analytical Hierarchy Process (AHP) is adopted. AHP is a well-established decision technique that provides both subjective and objective approach to risk analysis through collection of expert’s verdicts. Since its foundation (1980) by Prof. Saaty, AHP and its variants have been widely used in finance, education, engineering, government, defense, and numerous other fields (See for example [39,40,41,42,43]). Such extensive consumption is owing to its straightforwardness, ease of use, and rich flexibility [44]. AHP relies on the process of determining the weights of the criteria and the final solution weights of the alternatives with respect to the criteria [45,46]. In the current paper it is used to prioritize risks.
Three main steps adopted by AHP to reach a decision are the hierarchy construction and representation, priority analysis and synthesis, and logical consistency test application. After a problem is decomposed into multi-level hierarchy describing goal, criteria, sub criteria and so forth, the judgments of decision makers are sought to pairwise compare elements within each cluster with respect to parent aspect. Since the pairwise contrasts are based on subjective judgments, a final consistency test is applied to evaluate any degree of inconsistency that might have occurred during opinion formation. Once all levels within hierarchy are compared, a composite judgment matrix is formed, which reveals he comparative standing of each criterion and its attributes. AHP process flow is depicted in Figure 5.
Figure 5. Analytical hierarchy process flow (based on [44]).
Figure 5. Analytical hierarchy process flow (based on [44]).
Sustainability 07 05896 g005

3.2.4. Risk Evaluation, Treatment and Review

In order to address the evaluation, treatment and review desires, AHP blended with As Low as Reasonably Practicable (ALARP) Principle and Standard Deviation (SD) concepts are applied. The ALARP is a subjective connotation that can be used under the context of risk. It is encouraged as footing approach to setting tolerable risks, predominantly rightful for regulatory determinations [47]. The ALARP philosophy balances cost-benefit proposition for risk management [48,49] and suggests avoiding overt efforts that might flash back (as additional costs) in the shape of unjustifiable secondary risks. Figure 6 paints a conceptual association of risks and the ALARP. The offered approach uses the ALARP connotation for reaching tolerable risk levels.
Figure 6. Levels of risk and the ALARP (based on [48]).
Figure 6. Levels of risk and the ALARP (based on [48]).
Sustainability 07 05896 g006
Finally this design makes use of the Standard Deviation (SD), which is a simple and well-known statistical tool. It serves as indicator of variation or dispersion from the mean value; the greater its value is, the larger is the spread of data points from their average. Standard Deviation, termed “Standard Risk Deviation (SRD)” in subject case, has been used to measure risk-associated fluctuations.

3.3. Risk Leveling Procedure—A Step-by-Step Formation

Under this procedure a risk steering committee is established which undertakes the process, collects expert’s opinions and facilitates the decision making function. Step-by-step courses adopted are elaborated henceforth.

3.3.1. Scanning the Program Environment and Establishing the Context

Risk management activity embarks with scanning the program contexts. First and foremost, the initiative should qualify the characterization of a Program; for Projects at component level and Portfolios at strategic level, the approaches to risk management may differ. This stage examines the composition and objectives of the program, probes its external and the internal environments, and recognizes key stakeholders and their influence on the program. It also outlines the early risk criteria and ascertains the phase of life cycle of the program (formulation, definition, organization, deployment, appraisal, and dissolution [50]). Expert panelists for onward collection of risk opinions are also targeted. Later, their views are manipulated through certain instruments for risk decisions.

3.3.2. Identifying Risk Factors

Together with the help of expert panelists identified an exhaustive directory of program risks is established. This index—formally known as risk breakdown structure—outlines risk definitions, risk factors, risk groups, stakeholders, potential risk owners and other risk relevant information. Formal and informal interviews, brain storming sessions and literature reviews are employed to create such indexes. At this juncture, the risks received (delegated, escalated) from cross management layers (strategy, component) are also recognized as inputs. In order to expedite this process use of online tools and computerized versions of mentioned information gathering techniques are recommended.

3.3.3. Assessing Risk Probability

For preliminary risk analysis, and to gauge the likeliness of happening of each risk, this methodology sanctions Delphi Survey from the experts. We recommend three rounds for increased accuracy; however, a two round line may also be observed with slight compromise on fallouts. As an output of this process, we acquire a list where the factors are ordered in terms of risk probability. To improve on efficiency of this step, online applications, such as real-time Delphi on the Internet [51] or Web Based Dynamic Delphi [52], are encouraged.

3.3.4. Assessing Risk Impact

Impact of a risk is also characterized as the potential severity or consequences of risk on the program objectives. This stage recommends a Three Point Estimates survey to determine possible risk impacts. The end result is a list of risks ordered with respect to the consequences of each.

3.3.5. Assessing “Risk Value”

The next step fundamentally assesses “Risk Value” for each factor by simple multiplication of results obtained from last two stages. We define “Risk Value” as risk magnitude or Composite Risk Index (CRI). Risk values reflect the expected values and provide us spotting of each risk group in a primary notion. They do not “truly” reveal the relative strength or comparative standing of risks; nonetheless they help us to filter the leading risks for further analysis. Since programs—especially the large-scale programs—are complex disciplines, they demand comprehensive risk analysis for which risk matrices alone are not sufficient, for a number of limitations [53].

3.3.6. Performing Risk Routing

Next, in this tactic, Risk Routing (discussed under Section 2) is observed to pass the belonging of each risk to the pertinent layer for an apt treatment. This step extricates program level risks from the whole bunch. The others, dealt with routing treatment, are handed over to cross management layers, with due clarity and instructions. Risk Routing decisions are governed by the best judgments of risk experts; however, for such considerations, the factors discussed in Section 2.2 are observed. After doing this, the program body is left with “purely” program risks, clearly addressable at subject level.

3.3.7. Risk Filtering - Qualification for Detailed Analysis

At this stage of procedure, with help of experts, the criteria for Risk Filtering i.e., qualification of a particular risk for advanced and comprehensive analysis is drawn. The filtering criterion are based on several factors such as program phase, timeframe, available resources, managerial preferential, expected returns, regulatory implications, the assessed risk magnitudes, etc. For enhanced efficiency, selection of medium, high and critical rank risks for thorough analysis is recommended. If the risk values are based on numerals, a sample benchmark may be to handpick certain leading percentile (e.g., top 50 percentile) members. The left over minors put on watch list construct an ultimate input for the risk identification stage (Section 3.3.2).

3.3.8. AHP Application for Constructing Risk Rankings

In order to analyze the risks deeper, AHP is utilized which mutually contrasts the risks to expose their comparative masses, so that risk responses of corresponding intensities may be initiated. First of all, related risks are combined together to form risk groups, and a multi-layer hierarchy is constructed. In a typical case, three layers AHP hierarchy is formed which consists of Risk Ranking as the Goal, Risk Groups as Criteria, and Risk Factors as Sub-criteria. A graphical representation of risk hierarchy is exposed in Figure 7.
Figure 7. Representation of multi level risk hierarchy.
Figure 7. Representation of multi level risk hierarchy.
Sustainability 07 05896 g007
Initially, Inter Group risk weights are calculated through AHP survey, using specialized questionnaire. The experts are asked to pair wise compare the risk groups with respect to the goal; the final facts are determined by (taking geometric mean of all) the responses. The results are arranged in Eigen matrix form and relative weights of each risk group are worked out. Consistencies on judgments are checked to fall within permissible limits. Inter Group risk weights tell us priority of each group in a relative standing with respect to all the other groups. In the subsequent step, comparative risk weights for the risk factors falling within each group—called Intra Group risk weights—are formulated through another AHP survey. Intra Group risk weights tell us comparative priority of each risk factor in relation to all other factors falling under the same group. Finally, the multiplication of each pair of weights under same hierarchical leg gives birth to a conclusive value. This outcome, called Inter Factor risk weight concludes individual risk factor’s priority within holistic set of factors.
Since exercising AHP on a manual note is a lengthy and time intensive job, we propose web surveys and software based AHP application programs, such as MakeItRational, Expert Choice, Decision Lens, etc., to speed up the process. Such instruments also curtail likely inaccuracies that might occur during manual operations.
A careful observation on relative weights conceived, (in AHP ranking) reveals that the SRD among risks (weights) is large enough with several outliers falling far off from the mean value (which is 1/n in case of n number of risks). This depicts an imbalanced posture of program risks (where certain risks seem too big or too small comparative to the others), and indicates the demand of Risk Leveling to accomplish risk settled fields.

3.3.9. Planning and Implementing Risk Mitigation

This stage of the process receives a prioritized list of risks (to be mitigated at the program level). As such, mitigation is exercised in situations where a risk is both sizable and unavoidable. Though obtained risks are all sizable, for the sake of mitigation, we conceive them in “multiple tiers” with respect to their evaluated weights; for example, Top-Tier (with critical primacy), Intermediate-Tier (having middle weightage) and Lower-Tier (bearing lower primacy). If the risk numbers are large enough, even more classifications (tiers) may be explored. In order to reduce the risks and to bring them within tolerable levels, different risks (tiers) are dealt with different mitigation efforts, which are comparable to their risk values. The required mitigation is either accomplished by reducing risk probability, decreasing the impact, or exercising them both in parallel.
The risk mitigation strategy follows the ALARP principle and relies on assumption that all risks will progressively diminish in proportionate to the alleviation efforts observed. It starts addressing Top-Tier factors with a profound exertion, rolls down to Intermediate-Tier elements with a mediocre focus and finally deals the Lower-Tier factors, steadily. Each round of such mitigation adopted brings in sinking deviation of outliers from the mean value (in the AHP ranking chart). This “multi-tier” approach helps in achieving risk-leveled field in the program environments. While doing so, we are convinced that one goal of risk management is to equilibrium the distribution of resources in such a manner that the least sum of resources may bring the maximum risk mitigation benefits. To what degrees the risk levels ought to be condensed discretely, and to what bounds them should be leveled conjointly, nevertheless, rests on the risk expert’s judgments. Such rounds are continued until all the risks become fairly tolerable by the program entities. Risk mitigation often breeds secondary risks, which form input for the risk identification stage (Section 3.3.2).

3.3.10. Developing Inter Residual Risk Weights

After the measures on mitigation are exercised in the delineated way, the comparative risk weights (between residual risks) are developed (through AHP instrument) yet again. The goal is to achieve a program setting where the disparities between each pair of risks are reduced as compared to the initial values. As the mutual disparities reduce, the resulting weights (in AHP table) tend to concentrate towards the mean value, which reflects the mutual leveling process. Relative weights not falling within the equation indicate insufficiency or inefficiency in mitigation. In such cases, additional rounds are needed to bring the residual risks within control limits.

3.3.11. Risk Leveling and Approval Criterion

In theory, the said spells of risk mitigation may (perhaps) lead the residual risks to optimal zero. However, since the mitigation strategies always bear limited success, such posture is hardly achievable in practice. Had such idealism been achieved, all the program risks would be equally leveled in relative terms (having relative weights as 1/n, and SRD equal to zero in the AHP ranking table) and approaching zero in absolute terms, at the same time. Engaging the ALARP, however, leads us to compromise on certain residual risk levels that are fairly achievable, and tolerable for the program components. Given the characteristics of a program, and the associated contexts, the experts are required to design clear tolerable limits for risk acceptance, in line with the ALARP principle. In parallel to achieving the ALARP levels, a mutual leveling target (for example) can be the reduction of SRD (from pre to post scenario) by a set percentage value. Risk Leveling strives to bring the risk levels within risk bearing capacities of the component entities. On a conceptual view, Risk Leveling and its connotation to the ALARP is portrayed in Figure 8.
Figure 8. Risk leveling—a conceptual display.
Figure 8. Risk leveling—a conceptual display.
Sustainability 07 05896 g008
Describing in a mathematical notion, if there are a number of “program risks” in a turbulent environment represented by letter n , their risk vales are denoted by R and the standard deviation of their risk weights (in the AHP chart) are given by σ R ( W ( 0 ) ) and σR(W) for pre-mitigation and post-mitigation scenarios, respectively, then we tend to say that Risk Leveling emphasizes achieving the following two-fold motives, synchronously.
min i = 1 n R i
And σR(W) < σ R ( W ( 0 ) )

3.3.12. Contingency Planning and Active Acceptance

At this stage, this course recommends contingency plans for each residual risk. The fallback plans are also tailored to face the times when the primary risk responses might prove inadequate. Triggers and thresholds for contingency as well as fallback plans are also determined. The program is now equipped to face the challenges with genuine readiness.

3.3.13. Implementing the Program

Risk acceptance with tolerable residual risks, risk settled environment, and suitable contingency plans in place, is now an active decision. This structural approach treats it as judgmental point and validates implementing the program.

3.3.14. Risk Monitoring and Review

Since risks are not static [11], it is firmly required to continually monitor and evaluate them against what is planned and implemented. Over the course, fresh risks emerge and once existent fade out or change their profiles. Risks that we had accredited become more or less recurrent, serious or applicable to the program. Setting the thresholds (acknowledged under step 3.3.11) as benchmarks, we may find utilization of AHP pairwise comparison technique and SRD metric for risk review and control purposes as well.

4. Risk Leveling in Program Environments—An Illustrative Case Study

To elaborate the Risk Routing and Risk Leveling ideas, here we present a case study of a program underway by USF Company Ltd. (USFCo), Pakistan. USFCo acts on behalf of government and provides funding for the execution of Information and Communication Technology (ICT) program developments in underserved areas of the country. In a bid to promote rural community services and tele-education, the government plans to institute and sustain thousands of Universal Tele-Centers (UTC) in underserved areas [54,55]. As part of this large initiative, USFCo launched a pilot program for establishment and operation of 500 UTC in rural areas for a five-year term. Accordingly, the Terms of Reference (TOR) were floated and expression of interests (EOI) was solicited. Apparently, the drive was unworkable, as the areas involved were highly turbulent, while the TOR quoted were quite stringent. In a pre-bid conference, prospective contestants raised serious concerns and unanimously refused to participate until the TOR was revised, and the grievances were addressed. Given the situation, the services of a consultancy firm were hired to find a resolution. The consultants conducted multi-stage meetings with stakeholders to address the potential risks. A small study group was framed with joint membership of the primary stakeholders; it included the program experts, bidder’s representatives, subject specialists, and the consultants.
In the initial sittings, brain storming sessions (as described in Section 3.3.2 of this study) were held to recognize the key risks perceived by the stakeholders; as a result, twelve (12) significant risks were recognized, which can be seen in Table 1. In the subsequent stage a two round Delphi survey was conducted (according to Section 3.3.3 of this study) to assess the probability of each risk; the scores obtained against each risk are also available in the same table (i.e., Table 1). Further, in order to assess the possible risk impacts, a Three Point Estimates Survey (as given by Section 3.3.4 of this study) was assumed; the relevant findings are produced in the same table (i.e., Table 1) as well. Having this done, the risk values (i.e., CRIs) were worked out by simple multiplication of probability and impact readings obtained against each risk (please refer to Section 3.3.5 of this study). Given the intensity of results, the program’s contexts, and the judgments of the program experts, it was revealed that while some risks were essentially pertinent to the program (level), some others were either components centric or strategy relevant (please refer to Section 2.2 for details). For example, the lack of education of target consumers or the deprived access facilities at remote sites were beyond the (immediate) scope of the program and were meant to be addressed at the strategic levels (e.g., government.). Similarly the lack of materials or logistic supplies needed to be resolved at the component level (e.g., bidders). Consequently, six key (program level) risks were scrutinized for further addressing at the program level whilst the others were either delegated or escalated (as given by Section 2.2 and Section 3.3.6 of this study). The Risk Routing decisions observed may also be seen in Table 1.
Table 1. Universal tele-centers (UTC) risks—Delphi and three points estimate surveys, and routing decisions.
Table 1. Universal tele-centers (UTC) risks—Delphi and three points estimate surveys, and routing decisions.
Sr. NoRisksDelphi Results [Risk Probability] (Average Scores out of 100)Three Points Estimate [Risk Impact] (Average Scores out of 10)Risk Value (P×I)Risk Ownership after Routing Decision
First RoundSecond RoundOMP(O+4M+P)/6
1Risk of locations unviability for UTC establishments78.
2Risk of missing users due to poverty of target populations67.
3Risk of commercial electric power unavailability on target sites49.348.
4Physical security threats on sites60.761.
5Troubles in accessibility to remote sites15.
6Risk of impractically long operating times41.440.
7Hardship of weathers and atmospheric hazards on some sites9.610.
8Failure in user retentions due to illiteracy of target inhabitants38.641.
9Risk of broadband services unavailability or unviability36.435.
10Risk of uncontrollable and unforeseeable costs26.425.
11Risk of mounting regulatory burdens and govt. taxes26.
12Lack of material and logistics supplies on sites14.615.
The final (purely program level) risks are reproduced hereunder with numbering tags:
  • R-1. Risk of locations unviability for UTC establishments;
  • R-2. Risk of broadband services unavailability or unviability;
  • R-3. Risk of impractically long operating times;
  • R-4. Risk of commercial electric power unavailability on target sites;
  • R-5. Physical security threats on sites; and
  • R-6. Risk of mounting regulatory burdens and government taxes.
Since the remnant risks were a few in numbers, it was decided to analyze them all further for possible mitigations (see Section 3.3.7 of this study). In the next phase, an AHP survey (as mandated by Section 3.3.8 of this study) was conducted to rank the intensity of mentioned risks; the AHP results concluded are tabulated in Table 2 below.
Table 2. UTC risks—pre mitigation comparison matrix and relative risk weights.
Table 2. UTC risks—pre mitigation comparison matrix and relative risk weights.
(R-1) Risk of locations unviability for UTC establishments1.003.002.912.
(R-2) Risk of broadband services unavailability or unviability0.331.
(R-3) Risk of impractically long operating durations0.340.831.000.560.322.890.099
(R-4) Risk of commercial power unavailability on target sites0.493.691.801.000.844.870.212
(R-5) Physical security threats on sites0.953.323.
(R-6) Risk of mounting regulatory burdens and govt. taxes0.200.520.350.
λmax = 6.110, CR = 0.018
Mean of Risk Weights = 0.167
Standard Deviation of Risk Weights (Pre-Leveling) = 0.094
To address each potential risk, several mitigation tactics were discussed and the best suitable were agreed. Each of the top-level risks (listed above) were responded to by counter measures of corresponding strength (in line with Section 3.3.9 of this study). Some of the resultant measures agreed to are listed below.
  • M-1. Provision of free of cost UTC locations in government schools or public buildings.
  • M-2. Fifty-percent subsidy on broadband charges and provision of satellite broadband where required.
  • M-3. Curtailment in operating time from 24 h to 12 h per day.
  • M-4. Free of cost commercial power arrangements and 50% subsidy grant on solar power installations.
  • M-5. USFCo funded financial insurance against physical security threats.
  • M-6. 100% tax waivers on infrastructure and 50% waiver on operating services.
The resulting scenarios recorded added comfort levels and willingness of the bidders for a potential take. Such willingness of players symbolizes that the risks were well mitigated to match their tolerability; this depicts “leveling down” process in our analogy (please refer to Section 3 of this study). Later on, the revisions were incorporated in TOR and the revised bid was floated. A number of operators competed well in the race. The same AHP survey was repeated to gauge the perceptions of potential participants (as mandated by Section 3.3.10 of this study). The results of repeat survey synthesized over respondent’s evaluation are tabulated in the Table 3 below.
Table 3. UTC risks—post mitigation comparison matrix and relative risk weights.
Table 3. UTC risks—post mitigation comparison matrix and relative risk weights.
(R-1)Risk of locations unviability for UTC establishments1.000.581.490.490.460.870.118
(R-2)Risk of broadband services unavailability or unviability1.741.001.920.820.741.570.192
(R-3)Risk of impractically long operating durations0.670.521.000.520.490.700.100
(R-4) Risk of commercial power unavailability on target sites2.031.221.921.001.001.670.223
(R-5) Physical security threats on sites2.161.352.
(R-6) Risk of mounting regulatory burdens and govt. taxes1.150.641.430.600.581.000.133
λmax = 6.023, CR = 0.004
Mean of Risk Weights = 0.167
Standard Deviation of Risk Weights (Post-Leveling) = 0.052
A careful comparison of the survey results exposes that the mutual disparities among risks) went on a decrease and the SRD of risk weights reduced by 45% in the latter case (please refer to Section 3.3.11 of this study). Such decrease in mutual disparities of risks reflects “mutual leveling” phenomenon (please refer to Section 3 of this study). This leveling development made it possible for the components to comfortably join and contribute towards the accomplishments of the subject program.

5. Discussion and Possibilities

In practice, the risk assessments and the resulting choices depend upon the risk perceptions and risk attitudes of the stakeholders. Risk perception is the way in which stakeholders view a risk, based on set of values, needs, concerns, concepts and assumptions [10,11]. Risk attitudes, on the other hand, are the chosen responses of the individuals or the organization and are driven by the perceptions of stakeholders [10,56]. The perceptions about risk and the resulting attitudes may significantly vary from person to person and group to group. For example, a set of program stakeholders may use objective basis to assess risks, while the others may prioritize them according to their individual interests. Such conflicting perceptions may compromise the program’s sustainability. It is so highly desirable to achieve uniformity in risk perceptions by educating the stakeholders about risks. Fortunately it can be realized through active engagement of the stakeholders, establishing effective risk communication channels, using the common language in risk communications, etc. [11].
The methodology presented in this paper adopts “by them, for them” philosophy and attempts to bridge the perception gap between the stakeholders through engagement of experts from within the program and its constituent components. Such formations instigate improved involvement of the primary stakeholders, result in better decision-making and bear greater potential for decision buying. It is relevant to point that this approach repeatedly engages AHP for risk judgments. As AHP prioritization metrics are derived from expert’s verdicts, which represent multidisciplinary areas, we sincerely trust that interdependencies among risks and multiple components of the program are inherently addressed in proposed course. However, to further address the risk dependencies and improve on risk rankings, utility of the Analytical Network Process (ANP) may also be explored. ANP is an evolution of AHP and takes care of interdependence and feedback concepts in an enhanced fashion [57].
We understand that any risk management methodology conceived ought to be friendly to practice and should demand limited time and efforts to produce results; else it will not be widely used. Illusorily, some procedures used in this approach might seem overlong; such constraints can be overcome by employing automated versions and computerized forms of the prescribed tools. We have already suggested some automated alternates at the relevant steps in this draft. Going the extra mile, the delineated scheme—on the whole—may be burnt into a computerized tool. It may also be transformed into online application package for added comfort of the users. Moreover, in order to save on workmanship, a selective sub-group of experts within risk management function may assume the position of entire set of panelist, but with implicit compromises on outcomes.
The illustrated scheme demonstrates certain strengths, which are summarized as follows: (1) While addressing the risk management function, it truly addresses the distinct standing (i.e., locus) of program in the organizational hierarchy; (2) In contrast to routine risk management mechanisms, which are usually industry-specific, this approach is generic in nature, and can be applied in several domains; (3) Opposite to many philosophic and theoretical methods, it is grounded in a mix of industry renowned tools, which are relatively easier to practice; and (4) Contrary to a large number of prevailing tactics that address risk management process in piecewise (or partial) formations, this methodology embraces the entire essential stages of risk process. The mentioned advantages position RLPE as a suitable choice to fit into a variety of situations and programs. Such headways can be very beneficial for management of programs in underdeveloped and developing economies. Since public sector development programs usually suffer worse, due to absence of decent practices, we particularly urge adopting such progressions for sustainable management of these large-scale programs.

6. Summary and Conclusions

Program risk management is yet an emerging research domain; far off from the levels of development attained by project risk management. Paucity in specialized approaches in programs and multi project environments has led project practices to dominate and continue, subduing the distinct identity of programs. One cause backing such diffuseness is lack of risk management methodologies conceived for this distinct function. To realize program benefits in true spirit, there is stringent need to balance the risky environments in programs. Given the unique placement of programs within (multi-layered) organizational hierarchy and the significance of the program contexts, we have conceived a structured methodology for program risk management. This (proactive) approach is easy enough to be well accepted by the practitioners; it may also attend the risk management requirements in (complex and large-scale) programs. It emphasizes on “Risk Routing” tactic to unburden the program layer from exogenous risks. Further, it attempts to provision leveled risk environments to the program (components) through “Risk Leveling” mechanism. This approach tends to establish an environment where the program (constituents) envisions all the surrounding risks symmetric, yet laying well within their tolerable limits. Risk Leveling (procedure) achieves the desired balance by treating program risk in two simultaneous scopes i.e., “leveling down” and “leveling mutually”. Underlying motive is that, in risk-leveled environments, the programs may achieve sustainability in growth and success; in other words, the attainments may optimally comply with respective plans. This novel approach may occupy an integral habitation in the overall program management course. Such adoption may particularly be helpful for large-scale (development) programs, which are often composed of multiple components, and their environments are also increasingly complex and risky.


This work was supported by the National Natural Science Foundation of China (71271031) under the project name, “The Establishment and Control and Empirical Research of Security Risk Pre-warning Index System in Mega Projects based on Optimal Control Theory”.

Author Contributions

Shahid Rasheed designed and documented the entire research while the other authors reviewed and provided suggestions to improve. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

References and Notes

  1. Hillson, D.; Simon, P. Practical Project Risk Management: The ATOM Methodology; Management Concepts Press: Virginia, VA, USA, 2012. [Google Scholar]
  2. Aven, T. The risk concept—Historical and recent development trends. Reliabil. Eng. Syst. Saf. 2012, 99, 33–44. [Google Scholar] [CrossRef]
  3. Jordan, A. Risk Management for Project Driven Organizations: A Strategic Guide to Portfolio, Program and PMO Success; J. Ross Publishing: Fort Lauderdale, FL, USA, 2013. [Google Scholar]
  4. Dafikpaku, E.; Eng, M.B.A.B.; Mcmi, M. The Strategic Implications of Enterprise Risk Management: A Framework. Available online: (accessed on 14 April 2015).
  5. Ward, S.; Chapman, C. Transforming project risk management into project uncertainty management. Int. J. Project Manag. 2003, 21, 97–105. [Google Scholar] [CrossRef]
  6. Turner, J.R.; Müller, R. On the nature of the project as a temporary organization. Int. J. Project Manag. 2003, 21, 1–8. [Google Scholar]
  7. Sanchez, H.; Robert, B.; Bourgault, M.; Pellerin, R. Risk management applied to projects, programs, and portfolios. Int. J. Manag. Projects Bus. 2009, 2, 14–35. [Google Scholar] [CrossRef]
  8. PMI. A Guide to the Project Management Body of Knowledge, PMBOK, 5th ed.; Project Management Institute Inc.: Pennsylvania, PA, USA, 2013. [Google Scholar]
  9. PMI. The Standards for Program Management, 3rd ed.; Project Management Institute Inc.: Pennsylvania, PA, USA, 2013. [Google Scholar]
  10. ISO. Risk Management—Principles and Guidelines (ISO 31000: 2009); ISO: Geneva, Switzerland, 2009. [Google Scholar]
  11. OGC. Management of Risk (M_O_R®)-Guidance For Practitioners, 3rd ed.; Office of Government Commerce: London, UK, 2010. [Google Scholar]
  12. Oehman, J. The Guide to Lean Enablers for Managing Engineering Programs, Version 1.0; Vasa. Joint MIT-PMI-INCOSE Community of Practice on Lean in Program Management: Cambridge, MA, USA, 2012. [Google Scholar]
  13. Morris, P.W.G.; Pinto, J.K.; Söderlund, J. The Oxford Handbook of Project Management; Oxford University Press: Oxford, UK, 2011. [Google Scholar]
  14. Levin, G. Program Management: A Life Cycle Approach; CRC Press: Boca Raton, FL, USA, 2012. [Google Scholar]
  15. Pellegrinelli, S. What’s in a name: Project or programme? Int. J. Project Manag. 2011, 29, 232–240. [Google Scholar] [CrossRef]
  16. Pellegrinelli, S.; Partington, D.; Hemingway, C.; Mohdzain, Z.; Shah, M. The importance of context in programme management: An empirical review of programme practices. Int. J. Project Manag. 2007, 25, 41–55. [Google Scholar] [CrossRef]
  17. Alises, A.; Molina, R.; Gómez, R.; Pery, P.; Castillo, C. Overtopping hazards to port activities: Application of a new methodology to risk management (POrt Risk MAnagement Tool). Reliabil. Eng. Syst. Saf. 2014, 123, 8–20. [Google Scholar] [CrossRef]
  18. Robert, S.; Arias, J. Review of Risk Management Methods. Bus. Intell. J. 2011, 4, 59–71. [Google Scholar]
  19. Ahmed, A.; Kusumo, R.; Savci, S.; Kayis, B.; Zhou, M.; Khoo, Y.B. Application of Analytical Hierarchy Process and Bayesian Belief Networks for Risk Analysis. Complex. Int. 2005, 12, 1–10. [Google Scholar]
  20. Kwan, T.W.; Leung, H.K.N. Measuring Risks within a Program Consisting of Multiple Interdependent Projects. In Proceedings of the 2009 International Conference on Computational Intelligence and Software Engineering, Wuhan, China, 11–13 December 2009; pp. 1–7.
  21. Dey, P.K. Managing project risk using combined analytic hierarchy process and risk map. Appl. Soft Comput. 2010, 10, 990–1000. [Google Scholar] [CrossRef]
  22. Lycett, M.; Rassau, A.; Danson, J. Programme management: A critical review. Int. J. Project Manag. 2004, 22, 289–299. [Google Scholar] [CrossRef]
  23. Pellegrinelli, S. Programme management: Organising project-based change. Int. J. Project Manag. 1997, 15, 141–149. [Google Scholar] [CrossRef]
  24. Maylor, H.; Brady, T.; Cooke-Davies, T.; Hodgson, D. From projectification to programmification. Int. J. Project Manag. 2006, 24, 663–674. [Google Scholar] [CrossRef]
  25. PMCC. P2M: A Guidebook of Project and Program Management for Enterprise Innovation; Project Management Association of Japan: Tokyo, Japan, 2008. [Google Scholar]
  26. Thiry, M. Combining value and project management into an effective programme management model. Int. J. Proj. Manag. 2002, 20, 221–227. [Google Scholar] [CrossRef]
  27. IEC/ISO. DRAFT INTERNATIONAL STANDARD IEC/FDIS: Risk Management-Risk Assessment Techniques; ISO/IEC 31010:2009; ISO: Geneva, Switzerland, 2009. [Google Scholar]
  28. Pellegrinelli, S. Shaping context: The role and challenge for programmes. Int. J. Project Manag. 2002, 20, 229–233. [Google Scholar] [CrossRef]
  29. Shao, J.; Müller, R.; Turner, J.R. Measuring Program Success. Project Manag. J. 2012, 43, 37–49. [Google Scholar] [CrossRef][Green Version]
  30. Shao, J.; Müller, R. The development of constructs of program context and program success: A qualitative study. Int. J. Project Manag. 2011, 29, 947–959. [Google Scholar] [CrossRef]
  31. Hillson, D. Towards programme risk management. In Proceedings of the 2008 PMI Global Congress Proceedings, Denver, CO, USA, 18–21 October 2008; Volume 12, p. 2011.
  32. Kvale, S.; Brinkmann, S. Interviews: Learning the Craft of Qualitative Research Interviewing; SAGE: California, CA, USA, 2009. [Google Scholar]
  33. Litchfield, R.C. Brainstorming reconsidered: A goal-based view. Acad. Manag. Rev. 2008, 33, 649–668. [Google Scholar] [CrossRef]
  34. Hillson, D. Using a risk breakdown structure in project management. J. Facil. Manag. 2003, 2, 85–97. [Google Scholar] [CrossRef]
  35. Markmann, C.; Darkow, I.-L.; von der Gracht, H. A Delphi-based risk analysis—Identifying and assessing future challenges for supply chain security in a multi-stakeholder environment. Technol. Forecast. Soc. Chang. 2013, 80, 1815–1833. [Google Scholar] [CrossRef]
  36. Mullen, P.M. Delphi: Myths and reality. J. Health Org. Manag. 2003, 17, 37–52. [Google Scholar] [CrossRef]
  37. Hsu, C.-C.; Sandford, B.A. The Delphi technique: Making sense of consensus. Pract. Assess. Res. Eval. 2007, 12, 1–8. [Google Scholar]
  38. Balasubramanian, R.; Agarwal, D. Delphi Technique-A Review. Int. J. Public Health Dent. 2013, 3, 16–25. [Google Scholar]
  39. Ramanathan, U. Aligning supply chain collaboration using Analytic Hierarchy Process. Omega 2013, 41, 431–440. [Google Scholar] [CrossRef]
  40. Kong, W.-H.; Fu, T.-T. Assessing the performance of business colleges in Taiwan using data envelopment analysis and student based value-added performance indicators. Omega 2012, 40, 541–549. [Google Scholar] [CrossRef]
  41. Rasheed, S.; Changfeng, W.; Yaqub, F.; Rafique, K.; Zhao, D. Risk Balancing in the Programmes—An Application in the Telecom Domain. Int. J. U- E- Serv. Sci. Technol. 2014, 7, 175–194. [Google Scholar]
  42. Markelj, J.; Kuzman, M.K.; Grošelj, P.; Zbašnik-Senegačnik, M. A Simplified Method for Evaluating Building Sustainability in the Early Design Phase for Architects. Sustainability 2014, 6, 8775–8795. [Google Scholar] [CrossRef]
  43. Simane, B.; Zaitchik, B.F. The Sustainability of Community-Based Adaptation Projects in the Blue Nile Highlands of Ethiopia. Sustainability 2014, 6, 4308–4325. [Google Scholar] [CrossRef]
  44. Ho, W. Integrated analytic hierarchy process and its applications—A literature review. Eur. J. Oper. Res. 2008, 186, 211–228. [Google Scholar] [CrossRef]
  45. Saaty, T.L. Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 2008, 1, 83–98. [Google Scholar]
  46. Bodin, L.; Gass, S.I. On teaching the analytic hierarchy process. Comput. Oper. Res. 2003, 30, 1487–1497. [Google Scholar] [CrossRef]
  47. Li, J.; Pollard, S.; Kendall, G.; Soane, E.; Davies, G. Optimising risk reduction: An expected utility approach for marginal risk reduction during regulatory decision making. Reliabil. Eng. Syst. Saf. 2009, 94, 1729–1734. [Google Scholar] [CrossRef][Green Version]
  48. Marszal, E.M. Tolerable risk guidelines. ISA Trans. 2001, 40, 391–399. [Google Scholar] [CrossRef] [PubMed]
  49. Melchers, R. On the ALARP approach to risk management. Reliabil. Eng. Syst. Saf. 2001, 71, 201–208. [Google Scholar] [CrossRef]
  50. Thiry, M. ‘For DAD’: A programme management life-cycle process. Int. J. Project Manag. 2004, 22, 245–252. [Google Scholar] [CrossRef]
  51. Gnatzy, T.; Warth, J.; von der Gracht, H.; Darkow, I.-L. Validating an innovative real-time Delphi approach—A methodological comparison between real-time and conventional Delphi studies. Technol. Forec. Soc. Chang. 2011, 78, 1681–1694. [Google Scholar] [CrossRef]
  52. Yao, J.; Liu, W.-N. Web-based dynamic delphi: A new survey instrument. Available online: (accessed on 14 April 2015).
  53. Anthony, L.; Cox, T. What’s wrong with risk matrices? Risk Anal. 2008, 28, 497–512. [Google Scholar] [CrossRef] [PubMed]
  54. MOITT. One Year Performance Report. 2014. Available online: (accessed on 14 April 2015). [Google Scholar]
  55. USFCo. News: Consultative session held with telecom operators regarding establishment of Telecentres, 2014. Available online: (accessed on 14 April 2015).
  56. Hillson, D.; Murray-Webster, R. Using risk appetite and risk attitude to support appropriate risk-taking: A new taxonomy and model. J. Project Progr. Portf. Manag. 2011, 2, 29–46. [Google Scholar]
  57. Saaty, T.L.; Vargas, L.G. Decision Making with the Analytic Network Process; Springer: New York, NY, USA, 2006. [Google Scholar]

Share and Cite

MDPI and ACS Style

Rasheed, S.; Wang, C.; Lucena, B. Risk Leveling in Program Environments—A Structured Approach for Program Risk Management. Sustainability 2015, 7, 5896-5919.

AMA Style

Rasheed S, Wang C, Lucena B. Risk Leveling in Program Environments—A Structured Approach for Program Risk Management. Sustainability. 2015; 7(5):5896-5919.

Chicago/Turabian Style

Rasheed, Shahid, ChangFeng Wang, and Bruno Lucena. 2015. "Risk Leveling in Program Environments—A Structured Approach for Program Risk Management" Sustainability 7, no. 5: 5896-5919.

Article Metrics

Back to TopTop