The VUCAlity of Projects: A New Approach to Assess a Project Risk in a Complex World

Round 1

Reviewer 1 Report

Thanks for the authors to address the comments from the previous reviews in the new submission. The structure of the paper is much better now. There are few minor remarks:

• Basically what you mentioned in lines 236-241 is that projects 3, 4, and 5 basically share their characteristics. Can these description be combined with each other?
• Again numbering of the tables in page 7 is not correct, both of them are table 3.
• Maybe also highlight in the limitation of the  study that due to the small sample size, the application of statistical analysis for finding a significant result is also limited. Further research could examine the generalizability and applicability of the results in other projects/contexts.

Author Response

Dear reviewer

Thank you for you observations. We hope we have complied to them.

Reviewer 2 Report

I see no benefit in this paper beyond the statement “It would be worth managers reading Bennett & Lemoine and applying the idea to their projects”.

The initial literature on “risk” is so brief that I don’t think it is helpful. A couple of minor comments:

• On Black Swan events - “It is doubtful if a conventional risk assessment method would capture such an event and even so, the risk coefficient could be low due to how rare the event is, even in spite of its severe impact.” But nuclear power risk assessments have been dealing with this sort of event for decades and has no problem. There is plenty of literature on low-probability, high-impact risks.
• Flybjerg is cited twice (with, lazily, a whole list of dates rather than distinguishing his quite different works). It should be stated that this is not uncontested, and the work of Love and co-authors particularly is vehemently opposed to Flyvbjerg’s work (see particularly Ika with Love).

The literature on project complexity is also very sparse and doesn’t have some of the main references (see below). It doesn’t add anything to the literature. The idea of “complexity” that the authors are using isn’t clear.

The definition of VUCA is rather curious

• Volatility here is ONLY on resource cost and availability. Everything else about the project is not covered in this statement.
• “Complexity” here at first looks like “Structural” complexity (in the terms of the oft-cited Williams 1999) as it talks about “Many interconnected parts, multiple components and parts”. But “complex regulatory/political environments” are quite a different thing as this is about the political environment, to do with people, and influence etc – quite a different sort of ‘complexity’; see Geraldi et al (2011).
• Traditionally, there has been a division between talking about aleatoric risk (uncertainty that is inescapable) and epistemic risk (uncertainty because we don’t know things). It needs to be clarified which sorts of uncertainty we’re dealing with

The results give five sets of results, but what are to make of these, beyond the users saying they were helpful? What is the significance of the “statement score range” and “project score range”? Where did the “accuracy” come from – is this self-assessed, and why is almost entirely “high”?

Incidentally, a spider-diagram is an obvious thing to do, as in the original Master’s thesis from which this was taken.

As I said last time “I don’t really see what this paper offers beyond other papers which describe VUCA.” The authors’ response was “in our search in the literature we haven´t come across anything that is similar to this study?” which I don’t think answers the lack of content in this paper, but a 1-minute Google Scholar threw up a book on “Program management in VUCA environments”, a paper on “Using VUCA matrix for the assessment of project environment risk”, “On Stupidity in Project Management - A critical reflection of PM in a VUCA world” and so on. Some of these will be relevant.

The Google Scholar search also threw up the Masters thesis from which this paper is taken:

https://skemman.is/bitstream/1946/33540/1/How-VUCA-is-your-project-VUCA-meter-to-assess-volatility-uncertainty-complexity-ambiguity.pdf

and it is disappointing to see no more in this link than in the paper.

Author Response

I see no benefit in this paper beyond the statement “It would be worth managers reading Bennett & Lemoine and applying the idea to their projects”.

We agree that mangers might benefit from this, but we also show that the benefits that project managers believe that they might gain from taking a closer look on actual complex projects from a VUCA perspective. Furthermore, the meter will be developed further ant tested on large and complex projects and the authors have hopes it will in the end provide the industry with a tool that can add to the identification of risk.

The initial literature on “risk” is so brief that I don’t think it is helpful. A couple of minor comments:

• On Black Swan events - “It is doubtful if a conventional risk assessment method would capture such an event and even so, the risk coefficient could be low due to how rare the event is, even in spite of its severe impact.” But nuclear power risk assessments have been dealing with this sort of event for decades and has no problem. There is plenty of literature on low-probability, high-impact risks.

Flybjerg is cited twice (with, lazily, a whole list of dates rather than distinguishing his quite different works). It should be stated that this is not uncontested, and the work of Love and co-authors particularly is vehemently opposed to Flyvbjerg’s work (see particularly Ika with Love.

We are thankful for the comment and have dealt with it in teh text. With regards to Flyvberg, yes, there was a mistake in holding on to two references, that hold not reference to this paper. They have been taken out. We do not find the Flyveverg / Ika debate to be relvant fot this paper.

The literature on project complexity is also very sparse and doesn’t have some of the main references (see below). It doesn’t add anything to the literature. The idea of “complexity” that the authors are using isn’t clear.

 This has been dealt with in the current version

Complexity” here at first looks like “Structural” complexity (in the terms of the oft-cited Williams 1999) as it talks about “Many interconnected parts, multiple components and parts”. But “complex regulatory/political environments” are quite a different thing as this is about the political environment, to do with people, and influence etc – quite a different sort of ‘complexity’; see Geraldi et al (2011).

Traditionally, there has been a division between talking about aleatoric risk (uncertainty that is inescapable) and epistemic risk (uncertainty because we don’t know things). It needs to be clarified which sorts of uncertainty we’re dealing with.

We are thankful for the comment and we have made adjustments to the text. 

The results give five sets of results, but what are to make of these, beyond the users saying they were helpful? What is the significance of the “statement score range” and “project score range”? Where did the “accuracy” come from – is this self-assessed, and why is almost entirely “high”?

Incidentally, a spider-diagram is an obvious thing to do, as in the original Master’s thesis from which this was taken.

We agree and we have added the spider-diagrams back in. 

As I said last time “I don’t really see what this paper offers beyond other papers which describe VUCA.” The authors’ response was “in our search in the literature we haven´t come across anything that is similar to this study?” which I don’t think answers the lack of content in this paper, but a 1-minute Google Scholar threw up a book on “Program management in VUCA environments”, a paper on “Using VUCA matrix for the assessment of project environment risk”, “On Stupidity in Project Management - A critical reflection of PM in a VUCA world” and so on. Some of these will be relevant.

We know these papers and, even though they are on a similar topic their approach differs. 

Reviewer 3 Report

The concept of paper is good but I have following suggestions for the improvement:

1. Cited references are too old. (No updated references are there)
2. No proper flowchart defining method and research methodology is there.
3. Please make sure you address the above two comments, and explain your method properly. 

Round 2

Reviewer 2 Report

My last review said "reject" because of the lack of contribution to knowledge, and also a number of other issues including lack of engagement with various literatures.

Some of the latter points are now dealt with - at least, the engagement with the complexity literature is much better. But I personally don't see the contribution to knowledge other than "VUCA is a useful tool to have at hand when doing risk analysis" (which probably some people use anyway). The fundamental work of the MSc thesis hasn't changed.

Perhaps this will be a contribution to the Sustainability literature - I don't think it would be appropriate for the Project Management literature, so I have to leave the decision to the editors.