Practical Comparison Between the CI/CD Platforms Azure DevOps and GitHub

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper presents a comprehensive and well-structured comparison between Azure DevOps and GitHub, focusing on their CI/CD capabilities. The authors provided a clear and practical evaluation across multiple dimensions such as scalability, automation workflows, integration, security, pricing, and usability. The comparison and analysis are timely and relevant, especially as many organisations are working on selecting  standardised CI/CD platforms and their possible implementation in the cloud environment.

The paper uses quite efficient methodical feature-by-feature analysis, which is suitable for both academic and industry readers. The inclusion of real-world use cases, performance metrics, and cost considerations makes the findings actionable for software teams, IT decision-makers, and DevOps practitioners.

The paper also compares enterprise needs and developer-centric workflows. Its analysis of the security and compliance frameworks is particularly valuable for industry applications.

While the paper is not highly novel in its scope, it is practical and contributes sufficiently to the understanding of tool selection applications DevOps workflows. The clear presentation and practical insights make it suitable for publication in a journal like Future Internet.

Author Response

First, I would like to thank you for your thorough review of our paper “Practical Comparison Between CI/CD Platforms Azure DevOps and GitHub” (futureinternet-3544488) and helpful comments to improve it.

 

Reviewer 1

Comments to the Authors
This paper presents a comprehensive and well-structured comparison between Azure DevOps and GitHub, focusing on their CI/CD capabilities. The authors provided a clear and practical evaluation across multiple dimensions such as scalability, automation workflows, integration, security, pricing, and usability. The comparison and analysis are timely and relevant, especially as many organisations are working on selecting  standardised CI/CD platforms and their possible implementation in the cloud environment.

The paper uses quite efficient methodical feature-by-feature analysis, which is suitable for both academic and industry readers. The inclusion of real-world use cases, performance metrics, and cost considerations makes the findings actionable for software teams, IT decision-makers, and DevOps practitioners.

The paper also compares enterprise needs and developer-centric workflows. Its analysis of the security and compliance frameworks is particularly valuable for industry applications.

While the paper is not highly novel in its scope, it is practical and contributes sufficiently to the understanding of tool selection applications DevOps workflows. The clear presentation and practical insights make it suitable for publication in a journal like Future Internet.

 

 

 

To Reviewer 1:

            Thank you very much for your review and valuable remarks.

 

1. This paper presents a comprehensive and well-structured comparison between Azure DevOps and GitHub, focusing on their CI/CD capabilities. The authors provided a clear and practical evaluation across multiple dimensions such as scalability, automation workflows, integration, security, pricing, and usability. The comparison and analysis are timely and relevant, especially as many organisations are working on selecting standardised CI/CD platforms and their possible implementation in the cloud environment.

The paper uses quite efficient methodical feature-by-feature analysis, which is suitable for both academic and industry readers. The inclusion of real-world use cases, performance metrics, and cost considerations makes the findings actionable for software teams, IT decision-makers, and DevOps practitioners.

The paper also compares enterprise needs and developer-centric workflows. Its analysis of the security and compliance frameworks is particularly valuable for industry applications.

While the paper is not highly novel in its scope, it is practical and contributes sufficiently to the understanding of tool selection applications DevOps workflows. The clear presentation and practical insights make it suitable for publication in a journal like Future Internet.

Response 1:  Thank you for your constructive feedback. I appreciate your recognition of the comprehensive comparison and its forward-looking perspective on DevOps trends.

Based on your comments, the following major changes have been made to the manuscript:

- A new subsection has been added to the methodology to outline the size of the data and sources. This addition improves transparency, allowing readers to better understand the analysis.

- A new subsection has been added with recommendations for addressing migration complexity, such as: technical hurdles, learning curves, and strategies to mitigate these issues. This provides a more balanced view of the practical considerations associated with platform transitions. 

 

 

Thank you very much for your remarks and comments. They were very useful for me to emphasize the main tasks and contributions of the manuscript, and also to focus the readers attention on the new and unique elements.

Reviewer 2 Report

Comments and Suggestions for Authors

The paper provides a comprehensive and well-structured comparison between Azure DevOps and GitHub, two leading CI/CD platforms under Microsoft. The authors thoroughly evaluate various dimensions, including CI/CD capabilities, scalability, security, pricing, and usability, which are crucial for organizations when selecting a DevOps platform. The paper analyzed real-world case studies, industry best practices, and adoption trends to offer valuable insights to software engineers. The strengths and limitations between Azure DevOps and GitHub have been discussed in detail. Additionally, the paper's exploration of future directions, such as AI-driven automation and enhanced security practices, demonstrates a forward-looking perspective that aligns with emerging trends in the DevOps landscape.

However, one limitation of the paper is the lack of clarity regarding the data size and sources used for comparison. More details about the scale of data collected, such as the number of companies, projects, or pipelines analyzed, shall be provided. Additionally, the paper lacks a critical examination of the potential challenges organizations might face when transitioning from one platform to the other, such as migration complexities or the learning curve associated with adopting new tools.

Comments on the Quality of English Language

The paper is well organized and well written with clear English language.

Author Response

First, I would like to thank you for your thorough review of our paper “Practical Comparison Between CI/CD Platforms Azure DevOps and GitHub” (futureinternet-3544488) and helpful comments to improve it.

 

Reviewer 2

Comments to the Authors

The paper provides a comprehensive and well-structured comparison between Azure DevOps and GitHub, two leading CI/CD platforms under Microsoft. The authors thoroughly evaluate various dimensions, including CI/CD capabilities, scalability, security, pricing, and usability, which are crucial for organizations when selecting a DevOps platform. The paper analyzed real-world case studies, industry best practices, and adoption trends to offer valuable insights to software engineers. The strengths and limitations between Azure DevOps and GitHub have been discussed in detail. Additionally, the paper's exploration of future directions, such as AI-driven automation and enhanced security practices, demonstrates a forward-looking perspective that aligns with emerging trends in the DevOps landscape.

However, one limitation of the paper is the lack of clarity regarding the data size and sources used for comparison. More details about the scale of data collected, such as the number of companies, projects, or pipelines analyzed, shall be provided. Additionally, the paper lacks a critical examination of the potential challenges organizations might face when transitioning from one platform to the other, such as migration complexities or the learning curve associated with adopting new tools.

 

 

To Reviewer 2:

           

Thank you for your review and valuable remarks.

 

1. However, one limitation of the paper is the lack of clarity regarding the data size and sources used for comparison. More details about the scale of data collected, such as the number of companies, projects, or pipelines analyzed, shall be provided. Additionally, the paper lacks a critical examination of the potential challenges organizations might face when transitioning from one platform to the other, such as migration complexities or the learning curve associated with adopting new tools.

Response 1:  Thank you for your productive feedback. We appreciate your comments and suggestions, which helped us identify areas for improvement in the manuscript.

We agree that the choice to focus on Azure DevOps and GitHub requires clarification. An explanation for this decision has been added: Both platforms are widely adopted, share a common ecosystem under Microsoft, and are often compared in practice due to overlapping user bases. This focus allows for a deeper analysis of their similarities and differences, which is valuable for organizations. We recognize the importance of including other platforms, and in this regard, information has been added about additional tools such as AWS CodePipeline, GitLab CI/CD, and Jenkins.

A new subsection has been added that evaluates the automated testing capabilities of both platforms. This addition enhances the relevance of the document to practitioners and is consistent with the critical role of testing. A new subsection has been added with recommendations for addressing migration complexities, such as: technical hurdles, learning curves, and strategies to mitigate these issues. This provides a more balanced view of the practical considerations associated with platform transitions.

 

Thank you very much for your remarks and comments. I greatly appreciate your efforts to get very thorough and detailed with my manuscript! They were very useful for me to emphasize the main tasks and contributions of the manuscript, and also to focus the readers attention on the new and unique elements.

Reviewer 3 Report

Comments and Suggestions for Authors

The paper presents a comparison between two CI/CD platforms, namely Azure DevOps and GitHub. 
It analyzes the different features and capabilities of these platforms, including performance, security, costs and usability.

The main issue I found in the paper is related to the fact that only two platforms are considered, and in addition both are from the same company.
Why was this choice made? Why have other platforms/tools not been considered?
For example, some other CI/CD platforms that could be included in the comparison:
- Atlassian suite (Bitbucket, etc)
- AWS CodePipeline
- Jetbrains TeamCity
- Jenkins
- GitLab CI/CD

One further comment is related to capabilities analyzed: in my opinion, another dimension to include in the comparison is the "automated testing" capabilities of the platform, which is a crucial issue in any CI/CD pipeline.

Author Response

First, I would like to thank you for your thorough review of our paper “Practical Comparison Between CI/CD Platforms Azure DevOps and GitHub” (futureinternet-3544488) and helpful comments to improve it.

 

Reviewer 3

Comments to the Authors
The paper presents a comparison between two CI/CD platforms, namely Azure DevOps and GitHub.

It analyzes the different features and capabilities of these platforms, including performance, security, costs and usability.

The main issue I found in the paper is related to the fact that only two platforms are considered, and in addition both are from the same company.

Why was this choice made? Why have other platforms/tools not been considered?

For example, some other CI/CD platforms that could be included in the comparison:

- Atlassian suite (Bitbucket, etc)

- AWS CodePipeline

- Jetbrains TeamCity

- Jenkins

- GitLab CI/CD

One further comment is related to capabilities analyzed: in my opinion, another dimension to include in the comparison is the "automated testing" capabilities of the platform, which is a crucial issue in any CI/CD pipeline.

 

 

 

To Reviewer 3:

            Thank you very much for your review and valuable remarks.

 

1. The main issue I found in the paper is related to the fact that only two platforms are considered, and in addition both are from the same company.

Why was this choice made? Why have other platforms/tools not been considered?

For example, some other CI/CD platforms that could be included in the comparison:

- Atlassian suite (Bitbucket, etc)

- AWS CodePipeline

- Jetbrains TeamCity

- Jenkins

- GitLab CI/CD.

Response 1:  Thank you for your productive feedback. We appreciate your comments and suggestions, which helped us identify areas for improvement in the manuscript.

We agree that the choice to focus on Azure DevOps and GitHub requires clarification. An explanation for this decision has been added: Both platforms are widely adopted, share a common ecosystem under Microsoft, and are often compared in practice due to overlapping user bases. This focus allows for a deeper analysis of their similarities and differences, which is valuable for organizations. We recognize the importance of including other platforms, and in this regard, information has been added about additional tools such as AWS CodePipeline, GitLab CI/CD, and Jenkins.

2. One further comment is related to capabilities analyzed: in my opinion, another dimension to include in the comparison is the "automated testing" capabilities of the platform, which is a crucial issue in any CI/CD pipeline.

 Response 2:  Thank you very much for your comment. A new subsection has been added that assesses the capabilities of automated testing on both platforms. This addition enhances the relevance of the document to practitioners and is consistent with the critical role of testing. In addition, another new subsection has been added with recommendations for addressing the complexities of migration, such as: technical hurdles, learning curves, and strategies to mitigate these issues. This provides a more balanced view of the practical considerations associated with platform transitions.

 

 

Thank you very much for your remarks and comments. I greatly appreciate your efforts to get very thorough and detailed with my manuscript! They were very useful for me to emphasize the main tasks and contributions of the manuscript, and also to focus the readers attention on the new and unique elements.

 

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

In their revision, the authors answered all of the questions and issues raised in my review.
Regarding the two platforms considered, a paragraph has been added explaining that both platforms considered share more than 50% of the market.
Regarding the second comment, specific considerations related to automated testing have been added to the paper.

Thus, the paper has been improved as a result of their revision, and in my opinion, it is worth being accepted.