Offset Obligation in Defense Projects: Schedule, Budget, and Performance Implications

Round 1

Reviewer 1 Report

The present paper entitled “Offset and defense projects: Are they related?” deals with a relevant and very interesting topic. In general, the aim of the work was appreciated. However, the proposed paper reveals a lack of relevant study details and suffers several misalignments and clarity from the research perspective. So, it is desired that the following comments can hopefully help the authors improve their paper:

 -          The title research question should be rechecked and reviewed to be more convincing. It is suggested that the authors improve the paper title alignment to the study approach and according to the achieved research results and findings. As the study follows only a partial 3 variable perspective: schedule, budget, and performance and many potentially relevant topics and variables to address the proposed original title question were left outside of the study as the recognized examples: project value and stakeholder satisfaction, please considered revise the title for something more narrow and more aligned with the explored four hypotheses in the study, as something like: “Offset obligation in defense project: schedule, budget, and performance implications”,

 -          The potential implication of the chosen “Iron Triangle,” should be also deeper discussed in the article to be more convincing. As referred to in the article the quality and other variables have been used instead of the performance variable and currently, some authors are also moving from the triangle to the diamond and the star model for main project management constraints identification and characterization. So, a deeper discussion on these recent models and chosen variables that are being used on the exploration project management main constraints models was missed. Notice also, that it should be very clear on the project performance variables considered in place during the study,

 -          In the research plan is indicated that “Due to the lack of factually-based data in this field”, but the article indicates that the study was carried out in the first stage by an exploratory study employing a qualitative approach and later a quantitative research (Survey) development (research design, fig. 1). Wasn’t a literature review also done to check and support the lack of factual data? Please clarify the role of all the literature review done on the study and update according,

 -          As although the research design presented in Fig.1 indicates that qualitative research of the study and the literature review are related to the survey, the research findings are presented and discussed in the inverse order. So, please check and clarify the study design on the quantitative and qualitative connections as well as the triangulation design used. It is suggested that the authors add new information detailing the quantitative research as well as the qualitative research design details both in the text and in the diagram format. This will provide a more detailed snapshot of the research steps followed and will help the reader in a clearer understanding of the paper’s findings and results in discussion,

 -          Details on how Cronbach’s test was set to clarify the reported achieved results and associated conclusions between the entire questionnaire and the table 1 questions results were missed.  Also, on data analysis section indicates that the “findings of this study were principally analyzed quantitatively, using statistical software”. The indication of the statistical analysis design associated with data categorization and test assumptions check, as well as the software used in the study and the research questions in place was also much missed,

 -          It is strongly suggested to clarify and present all the relevant qualitative and quantitative design details used before the research findings are presented and discussed and separate research design from the study results and discussion,

 -          On the Findings Regarding the Research Questions section the research questions H1 to H4 (see p. 12), the H1, and H2 were studied as one-tailed (related both to budget) but H3 (relating with schedule association) and H4 (relating with performance association) were studied as a two-tailed. Also, notice that in table 1 the schedule questions (H3 associated) are first exposed, and all have the same data gathering structure. Please align the order of the reported results and findings with the table or adjust the table order and clarify why the statistical analyses that were performed on the budget were not also performed for the schedule and performance. Consider including a two-tailed test for the Budget variable as done for Schedule and the Performance,

 -          Please also include in the Study Limitations and Recommendations for Future Research section the limitations of the study in terms of the data used, proposed methods and chosen approaches, and developed analysis,

 -          As usual final thorough proofreading is recommended.

Author Response

Answers to the reviewers

Reviewer 1- Comments	Answer
The title research question should be rechecked and reviewed to be more convincing.	The title of the paper was revised to: “Offset obligation in defense project: schedule, budget, and performance implications”
The potential implication of the chosen “Iron Triangle,” should be also deeper discussed in the article to be more convincing. As referred to in the article the quality and other variables have been used instead of the performance variable and currently, some authors are also moving from the triangle to the diamond and the star model for main project management constraints identification and characterization. So, a deeper discussion on these recent models and chosen variables that are being used on the exploration project management main constraints models was missed. Notice also, that it should be very clear on the project performance variables considered in place during the study	As explained in the Methodology section, overall project success is a much wider concept than the traditional “Iron Triangle.” However, the concept of project success for defense projects remains very conservative and remains based on the Iron Triangle’s basic criteria of schedule, budget and performance. We discuss this issue both in the Methodology and Discussion sections.
In the research plan is indicated that “Due to the lack of factually-based data in this field”, but the article indicates that the study was carried out in the first stage by an exploratory study employing a qualitative approach and later a quantitative research (Survey) development (research design, fig. 1). Wasn’t a literature review also done to check and support the lack of factual data? Please clarify the role of all the literature review done on the study and update according,	We made some modifications in the Methodology section, and clarified the role of the literature review, presented the stages in the correct order , in in both the Research Design and Findings sections. We clarified figure 1 accordingly.
As although the research design presented in Fig.1 indicates that qualitative research of the study and the literature review are related to the survey, the research findings are presented and discussed in the inverse order. So, please check and clarify the study design on the quantitative and qualitative connections as well as the triangulation design used. It is suggested that the authors add new information detailing the quantitative research as well as the qualitative research design details both in the text and in the diagram format. This will provide a more detailed snapshot of the research steps followed and will help the reader in a clearer understanding of the paper’s findings and results in discussion,	We clarified the study design and the quantitative and qualitative connections, and why triangulation is used. We added more explanations regarding the study design and research steps to the Methodology section.
Details on how Cronbach’s test was set to clarify the reported achieved results and associated conclusions between the entire questionnaire and the table 1 questions results were missed.  Also, on data analysis section indicates that the “findings of this study were principally analyzed quantitatively, using statistical software”. The indication of the statistical analysis design associated with data categorization and test assumptions check, as well as the software used in the study and the research questions in place was also much missed,	The study examined each one of the three variables (budget, performance, schedule) from two perspectives. The first relates to a specific project in which the respondents were involved. The second concerns the respondents’ perception of the practices implemented by their organizations. That is, each one of the three variables (budget, performance, schedule) was the subject of two statements: one related to a specific project and one concerning the respondents’ perception of the practices implemented by their organizations. Cronbach’s alpha was calculated for each pair of statements: Cronbach’s alpha for budget equals 0.70;for performance, 0.82; and for schedule, 0.71. This information has been added to the body of the article before Table 1, and Table 1 was rewritten to include a clearer indication of variables. The name of the SPSS 27 statistical software, which was used for all of statistical analyses , is now specified.
-          It is strongly suggested to clarify and present all the relevant qualitative and quantitative design details used before the research findings are presented and discussed and separate research design from the study results and discussion,	We clarified the qualitative and quantitative design details in the Methodology and Findings sections.
On the Findings Regarding the Research Questions section the research questions H1 to H4 (see p. 12), the H1, and H2 were studied as one-tailed (related both to budget) but H3 (relating with schedule association) and H4 (relating with performance association) were studied as a two-tailed. Also, notice that in table 1 the schedule questions (H3 associated) are first exposed, and all have the same data gathering structure. Please align the order of the reported results and findings with the table or adjust the table order and clarify why the statistical analyses that were performed on the budget were not also performed for the schedule and performance. Consider including a two-tailed test for the Budget variable as done for Schedule and the Performance,	To improve the research findings, the number of hypotheses was reduced to 3 (instead of 4) and the hypotheses themselves were arranged in the order of the three variables: budget, performance and schedule. The reworded hypotheses are two-tailed, to improve the research findings
Please also include in the Study Limitations and Recommendations for Future Research section the limitations of the study in terms of the data used, proposed methods and chosen approaches, and developed analysis,	Another recommendation for future research relates to the statistical analyses: to improve and expand the statistical power of the analyses in this field , we recommend combining different types of classification such as TwoStep cluster analysis, k-means cluster analysis, hierarchical cluster analysis, silhouette statistics for cluster analysis, Naïve Bayes, decision-tree analysis, discriminant analysis, ntile analysis and nearest neighbor analysis
As usual final thorough proofreading is recommended	Final proofreading was done by a professional language editor.

 

Reviewer 2 Report

Offset and defense projects: Are they related?

 

The study combines qualitative tools ‒ ten semi-structured interviews conducted with academics and professionals ‒ with quantitative tools ‒ surveys sent to project managers and other people employed in leading Israeli defense manufacturing companies. In-depth analysis of the responses and findings of the qualitative study reveals that Offset risk must be managed like any other project risk. Its influence is evident in the failure to meet project budgets. The present study examined four hypotheses, using Chi-square tests of independence, goodness-of-fit tests, loglinear Poisson regression analyses and fluctuation charts. We found that low Offset Percentage levels are associated with low levels of Project Budget Exception and with low levels of Perception of Project Budget Exception. No association was found between Offset Percentage and Schedule, and between Offset Percentage and Performance.

 

1.         This is an interesting piece of work. The developed model is quite well described, and appears quite impressive. The findings and results of the case study are also very impressive. The results are clearly analyzed and well argued. The discussion and conclusions are clear and persuasive. Please underscore the scientific value added/contributions of your paper in your abstract and introduction and address your debate shortly in the abstract.

2.         The paper is very well structured. The material is well presented. I would suggest the author to discuss these references in your context and references.

3.         I would suggest you to include those assumptions in your literature review.  for your references, Yann Jaegler, Anicia Jaegler, Fatima Zahra Mhada, Damien Trentesaux & Patrick Burlat (2021) A new methodological support for control and optimization of manufacturing systems in the context of product customization, Journal of Industrial and Production Engineering, 38:5, 341-355; and Omar Elfarouk, Kuan Yew Wong & Wai Peng Wong (2022) Multi-objective optimization for multi-echelon, multi-product, stochastic sustainable closed-loop supply chain, Journal of Industrial and Production Engineering, 39:2, 109-127

4.         Page 17- I would ask to re-numbered those interview findings. he developed model is quite well described, and appears quite impressive. The findings and results of the case study are also very impressive. The results are clearly analyzed and well argued. The discussion and conclusions are clear and persuasive.

Author Response

Answers to the reviewers

Comments	Answer
Reviewer 2
Please underscore the scientific value added/contributions of your paper in your abstract and introduction and address your debate shortly in the abstract.	We added the contributions of the paper to the Abstract and the Introduction.
I would suggest you to include those assumptions in your literature review.  for your references, Yann Jaegler, Anicia Jaegler, Fatima Zahra Mhada, Damien Trentesaux & Patrick Burlat (2021) A new methodological support for control and optimization of manufacturing systems in the context of product customization, Journal of Industrial and Production Engineering, 38:5, 341-355; and Omar Elfarouk, Kuan Yew Wong & Wai Peng Wong (2022) Multi-objective optimization for multi-echelon, multi-product, stochastic sustainable closed-loop supply chain, Journal of Industrial and Production Engineering, 39:2, 109-127	We added the recommended references in the Conclusions and Recommendations section, regarding in theme3.
.         Page 17- I would ask to re-numbered those interview findings. he developed model is quite well described, and appears quite impressive. The findings and results of the case study are also very impressive. The results are clearly analyzed and well argued. The discussion and conclusions are clear and persuasive.	We presented the finding of the qualitative study in the beginning of the Findings section and re-numbered the interview findings.

 

Round 2

Reviewer 1 Report

The author's revision followed and clarified the study design, improved the proposed text by updating it, and reordered it as well as added new suggested supporting text and references to better support the proposed contribution as suggested, although some discussions were not presented to the suggested extent.  Nevertheless, the improvements, please consider the following final comments to increase the improvement of the reviewed version:  

 -          Please notice that in the study it is reported that all experts interviewed had the same methods (the semi-structured interviews and content analysis) and triangulation refers to the use of multiple methods or different types of data sources. So, please considered revise the paragraph in lines 397-402, highlighting that for triangulation purposes the results were retrieved from 3 different types of experts: 1) Project managers;2) Senior managers, and 3) Academics, changing the current argumentation to avoid any argument confusion. Also check and add any other relevant triangulation results retrieved from each of the sources that can support the triangulation claim.

-          Please verify and revised the text of section Findings on the interview findings as it should highlight (line 475+) clarifying the common results retrieved from each of the 3 different types of experts and the treatment done to any specific findings retrieve to better support the triangulation claim (the use of a table format considering the 3 expert types can help on text readability).

-          At the editorial level, please considered:

-           Avoid the use of adjectives as much as possible for example please consider reviewing the newly added text in line 16 as “…very a valuable factor to be considered when preparing for and managing a project….”.

-          Also please check and delete the text 0.70 in the performance in table 1 (first column, 3rd line)

-          At the beginning, of line 562 please removed the highlighted bold of the first letter (A) of that paragraph

Author Response

Response to round 2 comments on the paper:

Offset Obligation in Defense Projects:

Schedule, Budget, and Performance Implications

 

As before, we want again to thank the reviewers for their comments, helping us to improve the paper considerably. In this document, we detail how the comments were addressed in the revised paper.

As required, changes in the manuscript body are highlighted using the MS-Word Track Changes mechanism. There follows a description of the major changes in this version:

 

Reviewer #1

Clarify the triangulation process:

We added clarification on lines 412-415 regarding the triangulation process.

We added table 2 in the findings section - Findings of the Qualitative Study

 

Table 2: The categories discerned within the interviewees’ responses

Category	Projects Managers	Senior Managers	Academics
Offset impact on budget, performance and schedule of defense projects
budget compliance as the criterion most affected by offset obligation
Offset is defined and managed as a risk
Implementation of offset by drawing up a methodical purchasing plan
The vendors must comply with the offset requirement

 

 Avoid the use of adjectives as much as possible for example please consider reviewing the newly added text in line 16 as “…very a valuable factor to be considered when preparing for and managing a project….”

We rewrote the sentence as follows: “This finding points to a factor to be considered when preparing for and managing a project, and even more during the early stages of examining a transaction, prior to acceptance”.

 

Also, please check and delete the text 0.70 in the performance in table 1 (first column, 3rd line)

We corrected the text.

At the beginning, of line 562 please remove the highlighted bold of the first letter (A) of that paragraph.

We removed the highlighted bold.

 

Author Response File: Author Response.docx