A Practical Approach to Defining a Framework for Developing an Agentic AIOps System

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article addresses the evolution of automation in IT operations management, highlighting the shift from traditional approaches toward more autonomous models. In particular, it proposes an Agentic AIOps approach, which integrates intelligent agents capable of autonomously detecting anomalies, classifying incidents, and executing corrective actions in complex enterprise environments. Through a conceptual framework and a methodological proposal, the study aims to contribute to the design of more adaptive, scalable, and efficient AIOps systems that can reduce human intervention and improve operational performance.

I present the following observations in a constructive spirit, with the aim of strengthening the quality of the manuscript.

 

Although the article introduces the evolution of IT management towards more automated environments, it does not sufficiently develop the specific challenges organizations face when implementing advanced automation. A more precise characterization of the current limitations of traditional approaches (e.g., ITIL or Dev(Sec)Ops) would be helpful, particularly regarding the human workload required for incident resolution.

It is recommended that the gaps faced by current approaches (e.g., response times, operational workload, scalability) be specified in greater detail. Including empirical data, case studies, or recent references could reinforce the urgency and relevance of the proposed approach.

 

Although the article presents research questions that align with the study's general objective, these are not adequately substantiated or contextualized within the problem framework. There is a lack of a clear explanation of their genesis—how they emerge from a concrete problem, what specific gaps in the literature or practice motivate them, and what the consequences are of not addressing them. Furthermore, no explicit connection is established between the limitations identified in traditional approaches (such as ITIL, PRM-IT, or Dev(Sec)Ops) and the need to explore new alternatives like Agentic AIOps. This disconnect weakens the perceived relevance and urgency of the questions.

To strengthen the argumentative coherence of the article, it is essential to more robustly justify why these questions are important to address and how answering them would help resolve current challenges in advanced IT operations automation. What would be achieved by answering each one? How would this contribute to advancing the state of the art or solving critical problems in IT operations management?

 

While a framework for Agentic AIOps systems is presented, the results focus on observed benefits without directly linking them to the initial issues stated. This disconnect may hinder the reader's understanding of the proposed solution's actual impact on the problem it intends to solve.

To achieve greater argumentative cohesion, it is advisable to clearly articulate how the Agentic AIOps system addresses explicitly the weaknesses identified. This could be achieved by structuring the results according to the explicitly stated objectives or research questions.

There is a lack of clear linkage between the systematic mapping results and the research carried out. It is necessary to explain which general gaps were identified rather than merely referring to isolated articles as justification. How do the findings from the mapping relate to the research questions proposed in the Introduction? Although the article states that trends, gaps, and emerging challenges were identified, these gaps are not detailed, nor is it explained how they are connected to the need for an approach like Agentic AIOps. No studies are cited to evidence current shortcomings, nor are data or concrete examples provided regarding the limitations of existing automation or the degree of human intervention that remains necessary.

Be more precise in the conclusions to emphasize the significance and achievements of the research.

In line 332, there is red-coloured text. Please update it.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

I appreciate undertaking such a research subject, novel and with a high potential impact within literature.

However, you need to properly identify within literature the concepts you intend to study and debate in this regard, present a general situation at least.

Further,  you need to refer to Data collection, considering setting databases, research parameters, keywords and design queries, refining results, data analysis by publication date, year, scientific categories and so on.

You may even add here a data visualization and citation analysis.

In regard to your methodological aspects, you need to provide more detail as to clarify the used steps. 

Results are interesting, but a source for your graphs should be added, as well as the software used for getting it, if the design is yours. 

Moreover, results need to be discussed in the light of previous literature.

A final paragraph with limitations should complete the design of your manuscript.

 

Best regards,

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The paper discusses Agentic AIOps, a major IT operations subject. Integration of AI-driven agents for autonomous IT operations fits automation and AI trends. The authors establish clear research questions for the paper. The proposed framework for Agentic AIOps is well-structured, covering process alignment (ITIL, PRM-IT), service taxonomy, and automation phases. The authors position their work well in AIOps and MLOps landskape. The authors thoroughly reviewed academic and industry literature (IEEE, ACM, Gartner, IBM, etc.). The paper provides am well-defined historical context of AIOps evolution, from observability to predictive analytics and closed-loop automation. The growing automation-based autonomy levels paradigm is judicious. With a practical application and relevance to industry, the paper aligns its solution with established IT service management frameworks like PRM-IT and ITIL.

I have some comments/suggestions in order to improve quality of the study:

1. In Introduction section, it would be good that the authors include a direct comparison with current solutions (such as IBM Watson AIOps and Splunk AIOps), even though they have addressed the evolution of AIOps;
2. In the Results section:
   • Several parts appear to be dense and could be broken down by the authors into smaller and more readable segments;
   • The authors should detail AIOps evolution presented in Figure 1;
3. In the Discussion section:
   • Because several parts of the text of this section seem too dense, the authors should divide it into smaller and easier-to-read segments;
   • The authors should add an analysis on risk mitigation because IT operation may bring security vulnerabilities;
   • The authors should improve the framework overview in Figure 4 with a more structured and detailed breakdown;
   • Incident Management mapping presented in Figure 5 should be detailed by the authors;
4. In the Conclusion section, the authors should discuss the Limitations of their approach (e.g. lack of experimental results or implementation examples);
5. The authors should enhance the transitions between sections/subsections for improving the readability of the paper;
6. As stated in Instructions for Authors of Electronics journal (https://www.mdpi.com/journal/electronics/instructions#preparation), “Acronyms/Abbreviations/Initialisms should be defined the first time they appear in each of three sections: the abstract; the main text; the first figure or table. When defined for the first time, the acronym/abbreviation/initialism should be added in parentheses after the written-out form”. In the paper, there are included:

a. Several definitions of the same acronym:

• • AIOps is defined in lines 41 and 123;
  • ML is defined in lines 43 and 133;

b. Instances of unabbreviated material are used following the abbreviation:

• • RAG is defined in line 387 and used in line 354;

c. The majority of the acronyms of the Abstract are defined only in the main text;

d. The acronym GenAi is defined in the Abstract (line 16), but undefined in the main text (line 176);

e. There are undefined acronyms: Dec(Sec)Ops, CD/CI, KPI;

f. A lot of the acronyms defined in the paper are not included in the Abbreviations table;

7. The references should be described, depending on the type of work, in the format specified in Instructions for Authors of Electronics journal (https://www.mdpi.com/journal/electronics/instructions#preparation).

I hope that my feedback is useful to the authors in improving their paper and I wish them all the best in pursuing this important area of research.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

After performing a second review on the improved version of the manuscript, I feel that it has it s merit but does not meet the Journal Standards and criteria for publication.

Specifically, you did not considerably improved the manuscript after the previous feedback.

Moreover, you cannot add text pretending you have used methodological aspects that were not considered and were not specified within the initial version of the paperwork. You state that "To further refine our understanding of the field, we employed a mixed methods re-133 
search approach, combining quantitative analysis of literature and industry reports with 134 
qualitative insights gathered from expert discussions and focus groups."This is an important methodological aspect that one could not oignore reporting within the manuscript' s methodology.

Further, citation such as "... explore ([9], [10], [11], [12], [13], [14], 114 
[15], [16], [17], [18]), ACM Digital Library ([13], [18], [19],[20]), and arXiv ([4], [5], [21], [22], 115 
[23], [24], [25]). In parallel, we reviewed industry white.. " is not recommended.

The references list is very scarce and insufficient. 

The Results section is quite shallow and rudimentary.

I am sorry I cannot be more positive on the situation.

Best regards,

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

I appreciate the authors for thoroughly addressing all of my comments and suggestions to improve the quality of the study.

I hope my feedback has been helpful in refining the paper and wish the authors all the best in advancing this significant area of research.

Author Response

Comments 1: 

I appreciate the authors for thoroughly addressing all of my comments and suggestions to improve the quality of the study.

I hope my feedback has been helpful in refining the paper and wish the authors all the best in advancing this significant area of research.

Response 1: Thank you very much for the review and for helping us to improve the quality of our research study! Best regards!

Round 3

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

The manuscript has sufficiently been improved as for being published.

Best regards,