Analysis of GitHub Advanced Security: Security Integration in GitHub and Azure DevOps

Round 1

Reviewer 1 Report (Previous Reviewer 2)

Comments and Suggestions for Authors

This paper is an evaluation-oriented study that analyzes and synthesizes the practical limitations and trade-offs of using GitHub Advanced Security (GHAS) as a unified security layer across GitHub and Azure DevOps environments from architectural, configuration, and operational perspectives.

1. Figure 1 does not provide additional information beyond what is already described in Section 3.2 regarding the GHAS components. As such, the figure appears to be redundant and should be removed.

2. While the paper presents a DevSecOps architecture and operational model leveraging GitHub Advanced Security (GHAS), it is not sufficiently clear whether the proposed architecture (Section 4) and configuration design (Section 5) reflect design decisions derived from the unique characteristics of GHAS. The current architectural description follows a generic DevSecOps reference model that separates concerns into source code management, security analysis, and governance layers. However, it is not evident whether this structure is a consequence of GHAS’s execution model, policy enforcement mechanisms, or configuration differences between GitHub and Azure DevOps. This aspect requires further clarification.

3. Similarly, from a configuration design perspective, the paper proposes a modular approach that combines centralized policies with repository-level customization. However, it remains unclear how this design reflects GHAS-specific characteristics such as organization-level policy enforcement, the CodeQL configuration model, or platform-specific configuration management differences. The current discussion largely reiterates the general benefits of security-as-code, without clearly demonstrating why the proposed configuration model is particularly necessary or advantageous in a GHAS-based environment compared to existing approaches. The authors are encouraged to strengthen the explanation of the configuration design with these considerations in mind.

4. The paper describes CI/CD pipeline integration in Section 6 and discusses challenges in Section 7. Section 7, however, would benefit from further refinement in the following respects:

1. • The paper positively identifies and discusses a broad range of operational trade-offs, including cost, scalability, platform asymmetry, and the interpretability burden of automated analysis. However, these trade-offs appear to stem from general characteristics and inherent limitations of GHAS and automated security scanning tools, rather than being newly introduced or mitigated by the proposed architectural or configuration design choices. The current presentation primarily enumerates these trade-offs at an observational level, without clearly articulating causal relationships between specific design or configuration decisions and the observed trade-offs.
   • Therefore, the authors are encouraged to more clearly distinguish these trade-offs from the architectural and configuration contributions, and to present a more systematic and structured comparative analysis. Such an analysis could be organized around dimensions such as cost versus security coverage, centralization versus flexibility, and automation scope versus interpretability burden. This would enable readers to better understand whether the identified trade-offs arise from inherent tool limitations, specific architectural or configuration choices, or organizational decision-making contexts.

4. Overall, the readability of the paper is low, and the presentation would benefit from clearer organization and exposition to help readers better understand the paper’s objectives and contributions.

Author Response

We would like to sincerely thank the Editor and the Reviewers for their careful evaluation of our manuscript and for the constructive and insightful comments provided. We greatly appreciate the time and effort invested in reviewing our work. The suggestions have helped us significantly improve the clarity, rigor, and overall scientific quality of the paper.

Below, we address all comments point by point. All changes have been incorporated into the revised manuscript, and the text has been carefully restructured to ensure that each concern is explicitly and transparently addressed.

Reviewer 1

Comment 1:

Figure 1 does not provide additional information and appears redundant.

Response:
Thank you for this valuable observation. We fully agree with the reviewer that the original Figure 1 did not sufficiently add analytical value. In response, the figure has been completely redesigned and replaced with a new GHAS-specific execution and governance model (new Figure 1).

The revised figure explicitly illustrates how GitHub Advanced Security decouples security analysis from CI/CD orchestration while enforcing centralized governance across GitHub and Azure DevOps. The figure is now tightly integrated with the architectural discussion in Section 4 and is explicitly referenced and explained in the surrounding text. This change transforms Figure 1 from a descriptive illustration into an analytical artifact that directly supports the paper’s core argument.

Comment 2:

The architecture appears generic and not clearly derived from GHAS-specific characteristics.

Response:
We thank the reviewer for highlighting this important issue. To address this concern, we substantially revised Section 4 (Solution Architecture) to explicitly ground the architectural design in GHAS-specific execution and policy-enforcement mechanisms.

The revised text now clearly explains how the separation into source control, security analysis, and governance layers emerges directly from:

• GHAS’s shared analysis engines (CodeQL, secret scanning, dependency analysis),

• organization-level policy enforcement,

• and platform-specific execution differences between GitHub and Azure DevOps.

This causal relationship is reinforced visually through the redesigned Figure 1 and conceptually through the abstraction presented in Figure 2.

Comment 3:

The configuration design appears generic and does not clearly reflect GHAS-specific constraints.

Response:
We appreciate this comment and have revised Section 5 (Configuration Design) accordingly. The revised text now explicitly explains how the proposed modular configuration model is motivated by GHAS-specific characteristics, including:

• organization-level policy enforcement in GitHub,

• CodeQL query-pack management,

• and the need to reconcile configuration governance across GitHub and Azure DevOps pipelines.

Additionally, Section 7.3 further contextualizes these design decisions by analyzing the centralization–flexibility trade-off as a direct consequence of GHAS deployment in heterogeneous environments.

Comment 4:

The discussion of trade-offs is descriptive rather than analytical.

Response:
We thank the reviewer for this insightful observation. To address it, Section 7 has been completely restructured and expanded into a research-question-driven analytical discussion.

The revised section:

• introduces explicit research questions,

• structures the analysis along clearly defined trade-off dimensions (cost vs. coverage, centralization vs. flexibility, automation vs. interpretability, platform asymmetry),

• and systematically distinguishes between tool limitations, architectural design choices, and organizational decisions.

This restructuring transforms Section 7 from an observational discussion into a causal and analytical synthesis of the study’s findings.

Comment 5:

The readability of the paper is low.

Response:
We appreciate this feedback and have undertaken a comprehensive editorial revision of the manuscript. The revised version includes:

• clearer section introductions and transitions,

• improved signposting throughout the paper,

• tighter integration between figures and text,

• and reduced redundancy across sections.

As a result, the overall narrative flow and readability have been substantially improved.

Once again, we would like to thank the Editor and the Reviewers for their constructive feedback. We believe that the revisions have substantially improved the manuscript in terms of clarity, analytical depth, and scientific rigor. We hope that the revised version now meets the expectations for publication and we remain at the reviewers’ disposal for any further clarification.

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

The topic is relevant and the empirical work performed by the authors has merit and high quality. However, the scientific level of this paper can be significantly improved. Furthermore, some figures are very basic and could offer better quality and insights.

Main improvements:

1. Introduction section is written without the use of any reference. This is a very relevant issue that must be corrected. There are several claims not properly justified.
2. Research gap should be better explored and based on scientific evidence.
3. The impact of DevSecOps in software industry should be better explored in the literature.
4. Figure 1 looks to be very informal and lacks quality rigor. It should also reflect the technical components that are used to perform the three reported functions and address how these elements are interconnected.
5. The criteria to have chosen some tools in Table 1 should be clarified.
6. Organize the tools of Table 1 by alphabet order.
7. I recommend the authors to increase the quality and design of Figure 4. Honestly, like it is shown, it looks like a more piece of text than a figure. Same issue applies to Figure 8.
8. Figures 5, 6, and 7 are relevant but they should be better contextualized and explained by the authors.
9. It is important to better organize the information of Table 2 (alphabetically order).
10. Section 7 is relevant and the authors perform an interesting discussion regarding several domains. However, this discussion arises without any presentation of the research questions that could guide the interpretation of the results.
11. Conclusions section should also be improved and organized to explore the theoretical and practical contributions of this study.
12. Authors talk about the relevance of agile practices in Tables 1 and 2. However, the impact of agile practices in DevSecOps is not properly explored.

Author Response

We would like to sincerely thank Reviewer 2 for the careful reading of our manuscript and for the constructive and detailed feedback provided. We particularly appreciate the positive assessment regarding the relevance of the topic and the quality of the empirical work. The reviewer’s comments were instrumental in helping us substantially improve the scientific rigor, structure, and presentation quality of the manuscript.

In response, the paper has undergone a comprehensive revision, including strengthening the theoretical grounding, clarifying the research gap, restructuring the analytical discussion, and significantly improving the quality and contextualization of figures and tables. Below, we address each comment in detail.

Comment 1:

The Introduction section is written without the use of any reference. Several claims are not properly justified.

Response:
We fully agree with the reviewer. The Introduction has been substantially revised to incorporate relevant and up-to-date references from the DevSecOps literature. Unsupported claims were either removed or explicitly justified through citations. The revised Introduction now grounds the motivation of the study in established research on CI/CD, DevSecOps, and software supply chain security, thereby improving both its scientific rigor and contextual clarity.

Comment 2:

The research gap should be better explored and based on scientific evidence.

Response:
Thank you for highlighting this important issue. The research gap is now explicitly articulated in the revised manuscript. We clarify that, while existing studies examine DevSecOps practices and individual security tools, there is limited design-oriented research analyzing GitHub Advanced Security as a cross-platform security layer spanning both GitHub and Azure DevOps. This gap is now supported by references and clearly motivates the contribution of the present study.

Comment 3:

The impact of DevSecOps in the software industry should be better explored in the literature.

Response:
We appreciate this suggestion and have expanded the Background and Related Work section to better reflect the industrial impact of DevSecOps adoption. Additional literature discussing DevSecOps in large-scale, enterprise, and cloud-native environments has been incorporated. This contextualization strengthens the practical relevance of the study and situates GHAS adoption within broader industry trends.

Comment 4:

Figure 1 looks very informal and lacks quality rigor.

Response:
We agree with the reviewer and have completely redesigned Figure 1. The revised figure now presents a GHAS-specific execution and governance model that explicitly shows the technical components involved in code scanning, secret detection, and dependency analysis, as well as their interconnections across GitHub and Azure DevOps. The figure is now analytically integrated into Section 4 and directly supports the architectural argument of the paper.

Comment 5:

The criteria for choosing the tools in Table 1 should be clarified.

Response:
Thank you for this comment. A clarifying paragraph has been added before Table 1 explaining the selection criteria. Tools were chosen based on their frequent appearance in the DevSecOps literature, their representative coverage of major security domains (SAST, SCA, cloud-native security), and their relevance as comparative baselines to GHAS. This clarification improves transparency and methodological rigor.

Comment 6:

Organize the tools in Table 1 in alphabetical order.

Response:
This suggestion has been fully implemented. Table 1 has been reorganized in alphabetical order to improve readability and consistency.

Comment 7:

Figure 4 and Figure 8 look more like text than figures.

Response:
We appreciate this observation. Figures 4 and 8 have been redesigned and explicitly contextualized in the manuscript. In addition, dedicated introductory sentences were added to explain their analytical purpose. The figures now summarize and visually contrast GitHub-native and pipeline-driven GHAS integration flows, rather than duplicating textual descriptions.

Comment 8:

Figures 5, 6, and 7 should be better contextualized.

Response:
Thank you for this feedback. The surrounding text for Figures 5–7 has been expanded to explicitly explain their relevance and role in illustrating GHAS enablement and operational interfaces. These figures are now clearly integrated into the narrative and support the discussion rather than serving as isolated screenshots.

Comment 9:

Table 2 should be better organized (alphabetically).

Response:
Table 2 has been reorganized and streamlined to improve clarity and consistency. The information is now presented in a clearer and more systematic manner, facilitating easier comparison between the listed elements.

Comment 10:

Section 7 lacks research questions guiding the discussion.

Response:
We fully agree with the reviewer. Section 7 has been restructured to introduce explicit research questions at its beginning. These questions now guide the analytical discussion and provide a clear framework for interpreting the identified trade-offs and design implications.

Comment 11:

The Conclusions section should be improved to explore theoretical and practical contributions.

Response:
Thank you for this valuable suggestion. The Conclusions section has been fully rewritten to clearly distinguish between the theoretical contributions to the DevSecOps literature and the practical implications for organizations adopting GHAS. The revised section also explicitly discusses limitations and directions for future research.

Comment 12:

The impact of agile practices in DevSecOps is not properly explored.

Response:
We appreciate this observation. The revised manuscript now more clearly situates GHAS and DevSecOps practices within agile and CI-driven development contexts. While the paper does not aim to provide a full empirical study of agile methodologies, the role of agile practices in shaping security automation and developer workflows is now explicitly discussed where relevant.

Concluding Remarks to Reviewer 2

We would like to once again thank Reviewer 2 for the detailed and constructive feedback. The manuscript has been substantially strengthened in response to these comments, particularly with respect to scientific grounding, analytical clarity, and presentation quality. We believe that the revised version now provides a clearer and more rigorous contribution to the DevSecOps literature and more effectively communicates both the theoretical and practical significance of the study.

Round 2

Reviewer 1 Report (Previous Reviewer 2)

Comments and Suggestions for Authors

I have no further comments.

Author Response

We thank the reviewer for their review and positive assessment.

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

Thanks for the opportunity to revise again this manuscript. Authors claim some improvements that were not properly addressed. Considering the importance of the first issue, my recommendation is major revisions.

Main issues:

1. Section 1 (Introduction) is still written without any reference.
2. It is stated that “he tools summarized in Table 1”. It should be “the tools…”
3. Font size of lines 1065 and below is inconsistent.
4. The format of references is also not consistent. For journals, the tile of journal and volume should be in italic. The year should be in bold.

Author Response

Response to the Reviewer
We sincerely thank the reviewer for the additional round of review and for the detailed and constructive comments, which helped us further improve the quality and clarity of the manuscript. We have carefully addressed all raised issues as outlined below.
1. References in Section 1 (Introduction)
We agree with the reviewer’s observation. The Introduction section has been revised to include appropriate references supporting the presented concepts and background. Relevant citations have now been added to ensure proper contextualization and alignment with the related literature.
2. Typographical error in Table 1 reference
The typographical error (“he tools”) has been corrected to “the tools”.
3. Font size inconsistency
The font size inconsistency identified in the lower part of the manuscript has been corrected. All text elements have been reformatted to comply consistently with the journal’s template.
4. Reference formatting consistency
The reference list has been thoroughly revised. Journal titles and volume numbers are now formatted in italics, and publication years are shown in bold, fully conforming to the journal’s reference style requirements.
We believe that these revisions significantly improve the manuscript and address all concerns raised by the reviewer. We appreciate the reviewer’s careful evaluation and valuable feedback.

Round 3

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

The paper was revised and significantly improved through the revision process. In accordance, my recommendation is acceptance.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Summary comment:

I think this is fine as a sort of literature review, but I feel there is depth and critical reflection missing to this paper.  If you're doing a literature review, be more rigorous and less biased.  If you're arguing for this approach, be more empirical and quantitative.  I feel like I don't know what the paper is trying to do.  It mostly seems to be repeating a bunch of content that is closer to marketing speak.

 

Detailed notes:

Why do you focus on transitive dependencies in 1.2?  Surely all software has potential issues.  Scanning also isn't a panacea and tends to cause teams to overemphasize unreachable flaws.

 

> The modern software delivery pipeline has evolved into a complex system of inter- connected tools, repositories, and dependencies. This interconnectivity has expanded the potential attack surface, exposing organizations to threats at every stage of development, from source code management to deployment.

Has this increased the attack surface?  Isn't other software using mostly the same tools and isn't it just as vulnerable?  I think the speed and ease of CI/CD deployments is the major shift here.

 

> Unlike external or bolt-on security tools, GHAS operates natively within GitHub Enterprise, enabling continuous, automated security assessment as part of everyday development.

This feels like marketing speak.  What are some capabilities it provides you cannot or do not get otherwise.  Or at least, why is this important (is it due to a high rate of adoption because it is more integrated?).

> GHAS operationalizes DevSecOps principles by making security both visible and 288

actionable at every stage of development. Its combination of semantic analysis through 289

CodeQL, automated credential protection through secret scanning, and proactive 290

dependency monitoring through SCA forms a comprehensive and adaptive security layer 291

within GitHub’s ecosystem. By aligning security automation with developer experience, 292

GHAS lowers the barrier to adoption and transforms secure development from a 293

compliance requirement into an integral part of software engineering culture

 

More marketing speak...

 

The background is unnecessarily long without saying much.

 

> The most mature DevSecOps programs typically adopt 468

a multi-tool strategy, using GHAS as the foundational security layer while extending 469

capabilities through specialized solutions aligned with risk tolerance, compliance 470

requirements, and technology stacks.

citation / supporting data?

 

I think your background would be better suited to list categories of tooling and to be more superficial.  You are quickly crossing the boundary between fact and marketing speak in a lot of places.  If you want to cite what others say, that is fine, but try to make the statements about the systems specific and falsifiable.   

How well do other systems handle the goals in 3.2?

 

I'm confused about what section 4-6 is actually adding to the paper.  Is this telling people how to use these technologies?  It feels like this is more background than anything else.

 

The conclusions in section 7 are nice, but where did they come from?  For example, where is the data to show that CodeQL is hard to use?  A few citations of blog posts, etc. would at least help your argument, but of course a user study would be best.

 

Reviewer 2 Report

Comments and Suggestions for Authors

The most critical flaw of this paper is that its academic contribution is not clearly articulated. The proposed pipeline based on GHAS merely integrates functions such as code scanning, secret scanning, and dependency scanning, all of which are already widely available in existing open-source and commercial tools; thus, it does not introduce fundamentally new concepts. Nevertheless, embedding these functionalities directly into a CI/CD pipeline may provide some degree of differentiation from other platforms, and the “automated detection with immediate feedback” enabled by the pipeline-oriented workflow can function as a practical mechanism for reducing security misconfigurations in real-world development environments. Overall, the work appears to emphasise applicability over scholarly originality.

 

The second major limitation of this paper lies in its overall structure and presentation. The manuscript does not clearly convey the architecture and implementation of the proposed pipeline, GHAS, making it difficult for readers to understand the core contribution. In addition, the paper repeatedly describes the tools and functions used, often duplicating both abbreviations and their full forms. Redundant explanations frequently reappear, which obscures the key messages the paper intends to convey and weakens the overall readability and coherence.

 

The third major limitation of this paper is the lack of empirical validation for the proposed pipeline.

In Section 3, the authors state: “The main goal is to validate GHAS as a cross-platform security solution that delivers consistent, automated, and developer-centric protection across GitHub and Azure DevOps.” This claim implies specific evaluation dimensions, validation goals, and expected outcomes. However, Sections 4 and 5 merely describe the architecture and configuration design of GHAS, while Section 6 details the integration process into a CI/CD pipeline. Section 6.5 provides implementation details. Nowhere in these sections is there a demonstration or evidence-based evaluation to confirm that GHAS actually functions as a valid cross-platform security solution. Rather than presenting experimental results or case studies, the manuscript primarily reiterates which security principles, auditing mechanisms, and tools are integrated into the pipeline. As a result, the paper appears to assume that the mere presence of automation and auditing mechanisms inherently achieves its stated goals, without actually demonstrating or validating that this is the case.

To elevate this work from a technical report that applies existing tools and practices to a publishable academic paper, the authors must address the three major limitations discussed above. In addition, for better reader comprehension, the manuscript requires significant refinement to eliminate redundant content and should be restructured to present the core ideas with greater clarity and density.