Submit to this Journal Review for this Journal Propose a Special Issue

Article Menu

Share Help Cite Discuss in SciProfiles

Open AccessReview

Peer-Review Record

Algorithms in Low-Code-No-Code for Research Applications: A Practical Review

Algorithms 2023, 16(2), 108; https://doi.org/10.3390/a16020108

by Fahim Sufi

Reviewer 1:

Ibrahim Khalil

Reviewer 2: Anonymous

Reviewer 3:

Shahriar Badsha

Reviewer 4:

Alberto Rodrigues Da Silva

Reviewer 5: Anonymous

Algorithms 2023, 16(2), 108; https://doi.org/10.3390/a16020108

Submission received: 14 November 2022 / Revised: 6 February 2023 / Accepted: 10 February 2023 / Published: 13 February 2023

(This article belongs to the Collection Featured Reviews of Algorithms)

Round 1

Reviewer 1 Report

In recent times, many apps are being developed using LCNC platforms. Even researchers are also performing their data analysis with LCNC platforms. Hence, a shift from traditional high level coding to no-code platform is noticeable. This paper highlights this shift in trend on writing algorithm in an impressive manner (with the help of Figure 1). This review comprehensively covers the advantage and disadvantages of LCNC platforms and then focuses into usage of LCNC platforms in research domain. Since application of LCNC based algorithms is quite new within research community, I believe this review is one first on this topic.

However, the author can think about adding one more benefit of using LCNC, which is “Easy to Learn”. Since modern LCNC platforms provide an interactive visual interface, non-programmer scientists can learn it very quickly. They don’t have to memorize any complex coding structures or syntaxes (as it is only drag and drop).

Also, another core contribution of this paper is demonstrating how easy it is to come up with a comprehensive solution on Cyber-Attack dashboard with the help of LCNC (as shown in Fig. 4, Fig. 5, Fig. 6, Fig. 7 and Table 3). Since significant content of this paper reviews LCNC platform highlighting on Cyber-Attack, the author can consider adding “Cyber-attack” as an additional keyword.

This quality of the paper is good and it was an interesting read for me. I am in favor of accepting this manuscript.

Author Response

It is encouraging to know that three reviewers found this review to be interesting and recommended this paper to be accepted. I am pleased to inform that I have considered all the valuable comments and suggestions of the honorable reviewer. Accordingly, the manuscript has been thoroughly updated.

I should mention that according to the valuable suggestion, I have added the following new sub-section:

“2.11. Easy to Learn: While traditional low and high level programming languages have a steep learning curve, anyone can easily learn any of the modern LCNC platforms within days. This is because modern LCNC platforms provides an interactive visual interface supporting drag and drop actions. A citizen developer can obtain visual cues from the highly visual interface and can easily develop a complex application. This “easy to learn” feature of LCNC platforms is encouraging for non-programmer research scientists for their research data analysis.”

Reviewer 2 Report

· In Line 9, instead of saying "all" study, refer "several".

· In Subsection 3.1. and 3.2, change LNCN to LCNC. I think it was typo.

· In line 238, change "low level" to "low-level" for the sake of consistency (since other places the term "low-level" was used).

· Within the abstract section, you can use lower case for "Global News Analysis, Social Media Analysis, Landslide, Tornado, Digitization of Process, Manufacturing, Logistic, and software / App development".

· In line 181, the word "Named Entity Recognition" can be removed, since it was previously defined in line 59.

Author Response

Reviewer 3 Report

This paper covers an interesting review on “Low Code No Code” way of writing algorithms. “Low Code No Code” being a very recent trend, there are not many review papers available on this topic. Hence, I found this paper uniquely contributing to the body of knowledge. This paper highlighted the wide ranges benefits of low-code platforms. Moreover, this review provides a succinct review of how existing scientific studies have utilized the benefits brought forward by low-code platforms. Thus, scientists and researchers can now have a comprehensive understanding of how to adopt these low-code platforms within their area of research. Overall, the paper is mostly well written.

I have the following recommendations:

1) A flow chart or block diagram showing the conceptual outline of the review would be helpful for the readers to get understanding of the process easily.

2) In the Conclusion, author need to discuss the following:

a. Can Low-Code platforms completely eliminate hand-coding?

b. Which limitations of modern Low-code platforms are the main bottlenecks of widespread adoption of Low-Code platforms?

There were also few minor issues:

3) In Line 9, instead of saying "all" study, refer "several".

4) In Subsection 3.1. and 3.2, change LNCN to LCNC. I think it was typo.

5) In line 238, change "low level" to "low-level" for the sake of consistency (since other places the term "low-level" was used).

6) Within the abstract section, you can use lower case for "Global News Analysis, Social Media Analysis, Landslide, Tornado, Digitization of Process, Manufacturing, Logistic, and software / App development".

7) In line 181, the word "Named Entity Recognition" can be removed, since it was previously defined in line 59.

Author Response

1) A flow chart or block diagram showing the conceptual outline of the review would be helpful for the readers to get understanding of the process easily.

Many thanks for this suggestion. Accordingly, I have added a new section called “2. Research Method”. This section demonstrated how we performed review of previous paper in the area of low code no code development (including the inclusion/exclusion criteria, keywords, literature source etc.). Moreover, Figure 2 have been added to provide conceptual outline of the review in the form of a block diagram.

2) In the Conclusion, author need to discuss the following:

Can Low-Code platforms completely eliminate hand-coding?
Which limitations of modern Low-code platforms are the main bottlenecks of widespread adoption of Low-Code platforms?

Within conclusion, following paragraph has been added to address these concerns:

“Modern LCNC Platforms provide competitive advantages in solving critical research questions predominantly through its AI based data analysis and information processing capabilities. Widespread adoption LCNC platforms within the research community will be seen once critical concerns like ongoing cost commitments and vendor lock-ins are addressed. However, it is unlikely that LCNC platform would completely remove the requirement of hand coding in future since it is not a revolutionary technology [73].”

There were also few minor issues:

3) In Line 9, instead of saying "all" study, refer "several".

4) In Subsection 3.1. and 3.2, change LNCN to LCNC. I think it was typo.

5) In line 238, change "low level" to "low-level" for the sake of consistency (since other places the term "low-level" was used).

7) In line 181, the word "Named Entity Recognition" can be removed, since it was previously defined in line 59.

I am pleased to inform that I have considered all the valuable comments and suggestions of the honorable reviewer. Accordingly, the manuscript has been thoroughly updated.

Reviewer 4 Report

This paper intends to provide a general overview of its author's "practical review" of some "algorithms" supported by Low-Code-No-Code (LCNC) platforms for solving "Research Questions".

The subject addressed in this paper is relevant and interesting. However, this is not a scientific paper nor it is organized and presented as a paper to be published in a journal like the "MDPI Algorithms".

I present below some comments and suggestions that the author can consider to improve his work in this respect:

1) This is not a "scientific paper" per si because it does not provide an analysis of the related work or any validation or empirical evidence of the discussed review. Even for a literature review, the author can present and follow a more systematic process, for instance, the "systematic literature review" methodology.

2) Even the title " Algorithms in Low-Code-No-Code for Solving Research Questions: A Practical Review" raises several issues: the term "Algorithms" suggests the internal algorithms supported (ou provided) by LCNC platforms, and it is not clear what they are about. The same issue for "Research Questions" may suggest not "research applications" or "research application domains" but more "research questions" that are nowadays considered in the design of LCNC platforms.

3) The authors shall explain and discuss in what aspects is this paper different from their previous work/papers, i.e. discuss the related work.

3) The authors shall better introduce and define the core concepts used throughout the paper, namely what they mean by "algorithms", "features", "studies", "research problems", and "research areas"… Maybe a conceptual model would help to understand and use these concepts.

4) Regarding the questions stated in the end of section 1, why are they relevant to be considered in a scientific paper?

5) Need to better organize and focus the paper. For example, the demonstration of a practical example (that implemented a cyber-attack monitoring application) was a little confusing and distracting from the reader's perspective. For instance, the inclusion of a long table of data (Table 3), or the Figures 5 to 7 in the section Conclusion was strange and irrelevant from the reader's perspective.

Author Response

This paper intends to provide a general overview of its author's "practical review" of some "algorithms" supported by Low-Code-No-Code (LCNC) platforms for solving "Research Questions".

It is encouraging to know that four reviewers found this review to be interesting and three reviewers recommended this paper to be accepted. I am pleased to inform that I have considered all the valuable comments and suggestions of the honorable reviewer. Accordingly, the manuscript has been thoroughly updated. I should highlight that the updated manuscript has been restructured now in a more cohesive and logical manner. Following the processes of systematic literature review (e.g., providing inclusion criteria, exclusion criteria, providing peer-reviewed database sources, applying research questions etc.), the updated paper now addresses clearly defined 6 research questions.

I present below some comments and suggestions that the author can consider to improve his work in this respect:

I would like to thank the honorable reviewer for this suggestions on improving this paper. To make the paper more scientific following systematic literature review, I have done the following:

Three new research questions have been added in an effort to restructure the entire manuscript in a more logical and cohesive manner
Two New Sections (One for addressing research method, database search, search term used, inclusion/exclusion criteria and another one for demonstrating the use of LCNC) have been added
2 New table (i.e., Table 1, and Table 2) have been added to depict conceptual use of common terminologies as well as detailed inclusion and exclusion criteria for literature search
1 New figure (i.e., Figure 2) have been added to demonstrate the overall conceptual diagram of the research methodology

Many thanks for this helpful suggestion. The term “Algorithm” does not suggest internal algorithms of LCNC platforms. Existing LCNC platforms being developed by industry giants like Microsoft, Google, Outsystems and others, are like black box to researchers and programmers. Rather, programmers, scientists and researchers uses functionalities or features of LCNC platform to implement and deploy their research algorithms or pseudocodes in a non-programmatic (i.e., visual) way. To clearly demonstrated this concept, the updated manuscript now contains a newly created section 6.

Within section 6, a new algorithm (i.e., Alg1: Cyber Attack Intelligence Algorithm) is presented in line 514 of the updated manuscript. Then, this section describes (with Fig. 3, Fig. 4, Fig. 5, Fig. 6., Fig. 7, Fig. 8, Table 4, and Table 5) how LCNC platform is used to implement and deploy the presented algorithm in line 514.

Moreover, the title of the paper is updated. In accordance to the suggestion of the honorable reviewer the updated title omitted “Solving Research Question”. It is now updated as “Algorithms in Low-Code-No-Code for Research Application: A Practical Review”. Furthermore, to clarify the terminology “Algorithms” within the scope of this study, I have added a new table (i.e., Table 1. Consistent use of common terminologies) within a newly added section called “Research Methods”.

3) The authors shall explain and discuss in what aspects is this paper different from their previous work/papers, i.e. discuss the related work.

Many thanks for this suggestion. Accordingly, I have added a new section called “2. Research Method”. This section demonstrated how we performed review of previous paper in the area of low code no code development (including the inclusion/exclusion criteria, keywords, literature source etc.). Moreover, with the help of newly added Fig. 2, it is also shown previous work/papers have been utilized to generate new areas of study (addressing the 6 research question of this study).

I concur with this great suggestion from the honorable reviewer. Accordingly, I have added a new table (Table 1) to introduce the readers with the conceptual use of “algorithm”, “feature”, “studies”, “research problems”, “research area”, “research question” etc. Table 1 is strategically placed within the newly added section called “Research Methods” that demonstrated systematic literature review.

4) Regarding the questions stated in the end of section 1, why are they relevant to be considered in a scientific paper?

The concept of LCNC is new. At present, major LCNC platforms are being used for solving industry problems like automation, software development, and developing model driven applications. Apart from heavy use within industry, only a handful of researchers have adopted LCNC platforms on research domains. This paper serves as a guidance to researchers to adopt the benefits of LCNC platforms to solve their research problems. Hence, the questions at the end of section 1, provides practical details to non-programmer researchers into following:

Which feature of LCNC platforms could be used for solving research question
Which popular LCNC platforms could be used by the researcher to solve their research problems
Which areas of research (i.e., research domain) could be solved by modern LCNC platforms
How can a researcher practically adopt LCNC for solving their research problems (i.e., demonstration in chapter 6)

As per the suggestion comments by this reviewer as well as other reviewers, the manuscript have undergone significant restructuring. Instead of three main aim of this review, there are now six main aims. 2 new sections have been created and hence, within the updated manuscript, these 6 main aims (i.e., RQ1, RQ2, …RQ6) have been discussed in 4 different sections as per the following arrangements:

RQ1: What are the benefits of using LCNC platforms in general? – In Section 3
RQ2: What are the limitations of using LCNC platforms in general? - In Section 4
RQ3: Which features of modern LCNC platforms were used in existing studies? - In Section 5
RQ4: Which LCNC platforms were mainly used in solving research problems? - In Section 5
RQ5: What research problems or which area of research adopted LCNC platforms? - In Section 5
RQ6: How can a researcher adopt modern LCNC platform in solving critical research questions? - In Section 6

Most importantly, the newly added Figure 2 now clearly shows how these 6 research questions have been addressed within the framework of systematic literature review.

Reviewer 5 Report

Summary

This article reviewed the use of Low-Code-No-Code algorithms across various studies and domains. The key aims of the review were to identify the features of modern LCNC platforms used in the included studies, which LCNC platforms were used, and what areas of research or research problems LCNC algorithms were applied to. In addition, the author provided a practical example using LCNC to analyse and visualise Cyber-attack data. The provides a comprehensive summary for the benefits and limitations of LCNC platforms. The article identifies the key features of LCNC used across the included studies and areas of application. Most of the included studies are however authored by the author of this review. The databases searched, the search terms used, and methods used to identify studies, are not mentioned which impacts reproducibility.

General Concept Comments

The use of Low-Code-No-Code platforms is an important area and allows researchers from any discipline to use artificial intelligence and machine learning technology. The author has highlighted the many benefits of LCNC platforms.

The manuscript would benefit from significant restructuring to improve the clarity and communication of the main findings. I suggest structuring the manuscript in sections by the three main aims of the review article, followed by a discussion of the benefits and limitations of LCNC, then the Cyber-Attack example in a separate section.

The manuscript would benefit from a more in-depth discussion of the included articles to highlight the practical uses of such platforms.

Major Points

· Line 41 and 42: Suggest providing a definition of LCDP and LCAP.

· The discussion of the benefits and limitations of LCNC is detailed and comprehensive.

· The methods used to identify the articles for inclusion in this review are missing. In addition, the inclusions and exclusion criteria of the studies is missing.

· Section 4.1: A detailed discussion of the mathematics of the algorithms used is not relevant. Consider moving these to the appendix or cite a more detailed explanation for reference. A discussion of salient or exemplary uses of the AI/ML algorithms is probably more relevant to the review overall.

· Please place the example of Cyber Attack data analysis and visualisation in a separate section rather than integrated throughout the manuscript. This will improve clarity and flow. In addition, please further describe the methods used so this example is reproducible.

Minor Points

· Figures 6 and 7: Suggest using a screenshot of the application interface rather than a photo of the device to improve clarity of the image.

Author Response

I am pleased to inform that I have considered all the valuable comments and suggestions of the honorable reviewer. Accordingly, the manuscript has been thoroughly updated. Also, I am also grateful for the detailed and in-depth suggestions provided by the honorable reviewer. All these comments and suggestions are rational and I believe addressing all these comments would improve the overall quality of this manuscript.

I am pleased to inform that I have considered all the valuable comments and suggestions of the honorable reviewer. Accordingly, the manuscript has been thoroughly updated with following:

Three new research questions have been added in an effort to restructure the entire manuscript in a more logical and cohesive manner
Two New Sections (One for addressing research method, database search, search term used, inclusion/exclusion criteria and another one for demonstrating the use of LCNC) have been added
2 New table (i.e., Table 1, and Table 2) have been added to depict conceptual use of common terminologies as well as detailed inclusion and exclusion criteria for literature search
1 New figure (i.e., Figure 2) have been added to demonstrate the overall conceptual diagram of the research methodology
1 New subsection (3.11 Easy to Learn) have been added for providing more comprehensive benefits of LCNC
In regards to the comment “Most of the included studies are however authored by the author of this review”, I have removed 4 of my own reference and added few other highly relevant references. Now within the updated manuscript, on top of my 11 highly relevant LCNC platform centric publications (i.e., Journal publications with reputed publishers like IEEE Transactions, Elsevier, and MDPI), there are 62 additional references to existing literatures/articles/contents not published by me.

General Concept Comments

RQ1: What are the benefits of using LCNC platforms in general? – In Section 3
RQ2: What are the limitations of using LCNC platforms in general? - In Section 4
RQ3: Which features of modern LCNC platforms were used in existing studies? - In Section 5
RQ4: Which LCNC platforms were mainly used in solving research problems? - In Section 5
RQ5: What research problems or which area of research adopted LCNC platforms? - In Section 5
RQ6: How can a researcher adopt modern LCNC platform in solving critical research questions? - In Section 6

The manuscript would benefit from a more in-depth discussion of the included articles to highlight the practical uses of such platforms.

Practical in-depth discussions in terms of how to use LCNC platforms like Microsoft Power Platform is focused within the newly created section 6. This section takes the reader through step-by-step practical use of LCNC platform to develop a cyber threat assessment solution. Moreover, Table 3 added a new LCNC platform called “Primary AI” along with corresponding reference showing its use towards “AI Education for students”.

Major Points

Line 41 and 42: Suggest providing a definition of LCDP and LCAP.

Both these terminologies have already been defined in the second paragraph of Introduction section with the following statement:

“After Forrester defined the terminology “Low-code development platform (LCDP)” in 2014, Gartner coined the term low-code application platform (LCAP) in 2016 [7].”

The discussion of the benefits and limitations of LCNC is detailed and comprehensive.

The fact that the honorable found the benefits and limitations of LCNC comprehensive within this review is highly encouraging and motivating. I am pleased to inform that within the updated manuscript, a new benefit (i.e., subsection 2.11. Easy to Learn) of LCNC added and discussed in details.

The methods used to identify the articles for inclusion in this review are missing. In addition, the inclusions and exclusion criteria of the studies is missing.

Methods used to identify the articles for inclusion in this review is now described in details within the newly added section 2. Inclusion and exclusion criteria are clearly portrayed within newly added Table 2. Moreover, Figure 2 have been added to provide schematic diagram of overall literature review process used within this study.

Section 4.1: A detailed discussion of the mathematics of the algorithms used is not relevant. Consider moving these to the appendix or cite a more detailed explanation for reference. A discussion of salient or exemplary uses of the AI/ML algorithms is probably more relevant to the review overall.

I agree with this valuable suggestion. Equation 1 to 27 are not overly relevant from the perspective of a review study on LCNC algorithm. Hence, as suggested by the honorable reviewer, all these 27 equations representing the mathematics behind the algorithms have been removed. Now, in the updated manuscript sub-sub sections like 4.1.1, 4.1.2, 4.1.3, and 4.1.4 only contains exemplary uses of AI/ ML algorithms without referring the equations behind them.

Please place the example of Cyber Attack data analysis and visualisation in a separate section rather than integrated throughout the manuscript. This will improve clarity and flow. In addition, please further describe the methods used so this example is reproducible.

I concur with this valuable suggestion and as a result, I have created a new section (i.e., section 6. Demonstration of LCNC Adoption in Modern Research). As per the valuable suggestion of the honorable reviewer all examples of Cyber Attack data analysis and visualization have been added to this section. Now, this section provides an end-to-end understanding of using LCNC platforms to solve critical research questions like producing cyber intelligence for strategic decision makers. Moreover, the entire solution has been made available to public codebase for the sake of reproducibility.

Minor Points

Figures 6 and 7: Suggest using a screenshot of the application interface rather than a photo of the device to improve clarity of the image.

Figure 5 and 6 already provided screen shots of the program. Moreover, Figure 7 and Figure 8 have been recaptured with higher resolution for the sake of clarity. Both Figure 7 and 8 portrays deployments of mobile apps (both Android and iOS) by LCNC platforms.

Round 2

Reviewer 4 Report

This paper provides a literature review of Low-Code-No-Code (LCNC) platforms for solving "Research Questions" and discusses several features and algorithms supported by these platforms.

The subject addressed in this paper is relevant and interesting. The structure of the paper has been improved in this 2nd version. Thus, I consider that the paper has some appropriate contributions to the community.

I have a few comments and suggestions to improve the final version of the paper, mainly presentation-related aspects:

1) Tables and Figures shall be better “framed” in the text area. See, e.g. Tables 1, 3, 4 and Figures 4, 5, 6, 7, …

2) Remove Table 5 from the paper because it is not relevant to the paper.

3) Figures 6 to 8 (and respective explanations) shall be moved from section 7 to section 6.

Author Response

This paper provides a literature review of Low-Code-No-Code (LCNC) platforms for solving "Research Questions" and discusses several features and algorithms supported by these platforms.

Firstly, I would like to thank the reviewer for having interest this work. I am pleased to know that the honourable reviewer believes that this paper has appropriate contribution to the community. I appreciated all the suggestions and recommendations of the honourable reviewer. These suggestions have significantly enhanced the overall quality of this revised manuscript.

I have a few comments and suggestions to improve the final version of the paper, mainly presentation-related aspects:

1) Tables and Figures shall be better “framed” in the text area. See, e.g. Tables 1, 3, 4 and Figures 4, 5, 6, 7, …

Many thanks for this observation. I can confirm that these tables and figures have been now framed properly in the updated manuscript.

2) Remove Table 5 from the paper because it is not relevant to the paper.

Many thanks for this observation. Accordingly, I have removed the table and placed it within the appendix section. Moreover, I have highlighted the significance of this table within the revised manuscript.

3) Figures 6 to 8 (and respective explanations) shall be moved from section 7 to section 6.

I agree with this sensible suggestion. Accordingly, I have moved both Fig. 6 and Fig. 7 from section 7 to section 6.

Reviewer 5 Report

The article reviews the use of Low-Code-No-Code algorithms across a variety of applications.

Table 1 should be removed. This is not useful to the readership.

Similarly figure 2 is not relevant. This can be briefly and adequately described in the text.

The inclusions and exclusion criteria should be clear and not use terms like 'etc'. Why are tutorial and short papers excluded?

The discussion of the AI/ML Algorithms is far too detailed and detracts from the main point of the review. Assume the readership has an understanding of these algorithms. Cite relevant sources for further detail if required. The focus should be on the application of the algorithms rather than the theory which is widely understood and discussed in further detail in dedicated sources.

In section 6 it is not clear what sources the cyber attack data is from. The example should be reproducible to readers if it is to be useful. Similarly detail is missing on how tweets were translated.

Table 5 is extremely detailed and of little demonstrative use. A graphic would convey this information more suitably.

Figure 7 and Figure 8 are not suitable for publication - proper screenshots should be obtained.

It is ok to include author references if these are relevant. Removing these simply because they belong to the author may miss important applications. The references should reflect a comprehensive search for the literature and discussion of articles that are relevant to the main aims of the review.

Overall the aim of the article has significant merit and would be of interest to the readership. The article is limited significantly by incomprehensible english and superfluous detail not related to the main aims of the article. In addition, while this revision expands on the discussion on the applications of such LCNC algorithms, further discussion of these applications to highlight their usefulness is warranted. Further, the Cyber Attack analysis lacks the detail required to reproduce this.

Professional editing and senior input is recommended.

Author Response

I would like to thank you for taking the time in reviewing my paper. Several of your valuable suggestions contributed towards enhancing the overall quality of the updated manuscript. However, I found few of your comments to be superfluous, vague, and contradictory. Hence, I would like to raise following 3 observations before delving into the details of review responses:

Observation 1: You (i.e., Reviewer 5) are the only reviewer (out of the 5 reviewers) that highlighted “self-citation” being an issue (i.e., inappropriate self-citation). In your earlier review (i.e., 1st round of review), you mentioned “Most of the included studies are however authored by the author of this review.”.

That is why in the 1st revision, I had removed some of my highly relevant publications. Now, in this 2nd review, you are saying “It is ok to include author references if these are relevant. Removing these simply because they belong to the author may miss important applications.”. I find these comments completely contradictory to each other. I wonder if I add my highly relevant publications (that I had removed from the previous version of the manuscript based on your previous suggestion) back again, you would again mention “Inappropriate self-citation detected as most of the included studies are authored by the author”. So, please be consistent on your review comments on your future review reports. Contradictory comments from the honorable reviewer like yourself waste valuable time of the author.

Observation 2: Moreover, in your future reviews, please read the paper properly, before commenting and highlighting an issue that is not there in the first place. For example, in your 1st review, you highlighted “Line 41 and 42: Suggest providing a definition of LCDP and LCAP.”. It is more worrying that your highlighted this point as a “Major Points”. However, if you had read the manuscript diligently, then you would have found that both these terminologies (“LCDP” and “LCAP”) had been previously defined within introduction section. So, without properly going through the manuscript if you ask the author to define “LCDP” and “LCAP”, while the term have already been defined, you are wasting the valuable time of the author. I pointed this issue in my earlier responses to your 1st review report. I am assured that you realized your mistake and didn’t raise this issue again in your 2nd review.

Observation 3: Furthermore, in this review, you mentioned “Figure 7 and Figure 8 are not suitable for publication - proper screenshots should be obtained.” First of all, when you say “proper”, do you mean screenshots taken at a higher resolution. If that is the case, then please note that both Figure 7 and Figure 8 are captured at a higher resolution at 330 ppi (i.e., HD Quality). Hence it is suitable for publication as evidenced by my following publications:

https://www.sciencedirect.com/science/article/pii/S221501612200334X (Publisher: Elsevier)
https://www.sciencedirect.com/science/article/pii/S2772662222000613 (Publisher: Elsevier)
https://www.sciencedirect.com/science/article/pii/S2667096822000179 (Publisher: Elsevier)
https://link.springer.com/article/10.1007/s13369-022-07250-1 (Publisher: Springer)
https://www.mdpi.com/2071-1050/14/16/9830 (Publisher: MDPI)
https://www.mdpi.com/2071-1050/14/10/6303 (Publisher: MDPI)
https://ieeexplore.ieee.org/abstract/document/9834043 (Publisher: IEEE)
https://ieeexplore.ieee.org/abstract/document/9737676 (Publisher: IEEE)
https://ieeexplore.ieee.org/abstract/document/9612169 (Publisher: IEEE)
https://ieeexplore.ieee.org/document/9546772 (Publisher: IEEE)
https://ieeexplore.ieee.org/document/5643150 (Publisher: IEEE)
https://ieeexplore.ieee.org/document/4909289 (Publisher: IEEE)
https://www.techscience.com/cmc/v72n2/47229/html (Publisher: Techscience Press)

I should highlight the fact that all the above journal publications have one or more screenshots of devices (i.e., Samsung phone, iPad, similar to Figure 7 and Figure 8) that were deemed to be suitable for publications. If all the renowned publishers like Elsevier, Springer Nature, MDPI, IEEE have already successfully published my screenshots similar to Figure 7 and Figure 8, then why do you think “Figure 7 and Figure 8 not suitable publication”. Please note again, all these highly cited publications (for example, https://ieeexplore.ieee.org/document/5643150 have 90 citations) within highly revered journals (for example, https://ieeexplore.ieee.org/document/4909289 in IEEE JSAC with an impact factor of 13) were authored by me. I was either the only author or the main author (i.e., first author). I highlighted the selected journal publications to prove my point why I confidently believe Figure 7 and Figure 8 are suitable for publication. I can provide more examples, where I have successfully published device screenshots with publishers like IEEE, MDPI, Springer Nature, MDPI and others). As long as the screenshots over 300 ppi in resolution, it is considered to be publishable.

Blatantly saying “Figure 7 and Figure 8 are not suitable for publication - proper screenshots should be obtained.” without any justification is simply carelessness as the word “proper” could be interpreted in different ways. I understand that you might have a busy schedule and you do not have time to explain what you mean by “proper” (i.e., higher resolution, image capture from another angle etc.). If you are too busy to explain your point, simply do not accept the request to review. Otherwise, you are wasting valuable time of the author.

The article reviews the use of Low-Code-No-Code algorithms across a variety of applications.

Table 1 should be removed. This is not useful to the readership.

Table 1 was not there in the original version of the manuscript. However, it was added, since review 4 had specifically requested for it (for the sake of clarity towards the readers). He mentioned in the first round of review “The authors shall better introduce and define the core concepts used throughout the paper, namely what they mean by "algorithms", "features", "studies", "research problems", and "research areas"…”.

Hence, following this valuable suggestion, I had updated the manuscript with Table 1 and as result, reviewer 4 has now suggested the paper be accepted for publication with some very minor presentation change.

Similarly figure 2 is not relevant. This can be briefly and adequately described in the text.

In your first review suggestion, you had mentioned “The databases searched, the search terms used, and methods used to identify studies, are not mentioned which impacts reproducibility.” Moreover, you had also highlighted “The methods used to identify the articles for inclusion in this review are missing.”. Figure 2 clearly describes the databased used and systematic literature review methods used to identify the articles for inclusion within the scope of this study. It is very common to represent the search and selection process of review articles using schematics like Figure 2, as evident from the following publications:

Figure 2 of Springer Publication, https://link.springer.com/article/10.1007/s10270-021-00964-0
Figure 1 of MDPI Publication, https://www.mdpi.com/2071-1050/10/10/3821
Figure 1 of MDPI Publication, https://www.mdpi.com/2076-3417/11/4/1842
Figure 1 of IEEE Publication, https://ieeexplore.ieee.org/document/8637760
Figure 1 of IEEE Publication, https://ieeexplore.ieee.org/document/9706446
Figure 4 of IEEE Publication, https://ieeexplore.ieee.org/document/9771452
Figure 1 of Pubmed Central Publication, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9112080/

All the journal papers above used schematics to visually summarize the process of database search and systematics literature review with the help of schematic diagrams like Figure 2, while describing the process within the text.

I should highlight the fact that review 4 in the 1st review suggested “the author can present and follow a more systematic process, for instance, the "systematic literature review methodology”. Since this is a standard practice to diagrammatically represent the process of systematic literature review, Figure 2 was introduced in updated manuscript following reviewer 4’s direction. As a result, review 4 has now suggested the paper be accepted for publication with minor presentation change.

The inclusions and exclusion criteria should be clear and not use terms like 'etc'. Why are tutorial and short papers excluded?

I agree with your valuable comment and as result I have updated Table 2. I can confirm that term like ‘etc’ were removed from Table 2. Moreover, to address the sensible comment, “Why are tutorial and short papers excluded?”, I have added the following new paragraph within Section 2 (Research Methods):

“General tutorial papers, tutorial videos, online discussions were omitted (as seen from Table 2), since the focus of this study was the usage of LCNC platforms on solving research problems. Since generic tutorial papers, videos, and discussion did not focus on solving research problems, literatures of these types were omitted. Short papers less than 4 pages were also excluded since they did not delve into the details of using LCNC platform.”

Many thanks for highlighting this issue. To reduce distraction and maintain cohesive flow towards the readership, I have now omitted the unnecessary details from your suggested AI/ML Algorithms sections. For example, in this updated manuscript, I have removed lines 344 ~ 350, 352 ~353, 358 ~ 365, 389 ~ 405, 438 ~ 440, 446 and others. Now, each of the AI/ML algorithms have only one paragraph allocated for discussing how other researchers have utilized that specific algorithm (with all the necessary references).

In section 6 it is not clear what sources the cyber attack data is from.

Many thanks for highlighting this issue. Accordingly, I have added the following new paragraph within section 6:

“There were following two different types of data obtained for this demonstration:

Realtime Cyber-Attack Data collected from Anti-Virus Vendors (i.e.., https://statistics.securelist.com/)
Realtime Cyber related Twitter Feeds obtained using Twitter API (i.e., https://developer.twitter.com/en/portal/dashboard)

Fig. 3 essentially implements line 1 of Alg1 and detailed process of obtaining data from social media, websites, and other online avenues have been demonstrated in [13] [16] [17] [18] [19] [20] [21] [22].”

The example should be reproducible to readers if it is to be useful.

Many thanks for this valuable comment. As a result, I have made all the input data (i.e., Tweets) as well as the solution files in publicly accessible repository. I have also added the following paragraph within the updated manuscript so that this study could be reproduced:

“As mentioned earlier, the cyber intelligence solution was developed using Microsoft Power Platform. All the source files (including the .pbix Microsoft Power BI solution, Cyber related Tweets etc.) are publicly available at https://github.com/DrSufi/COVID_Index_Anomaly for the sake of research reproducibility.”

Similarly, detail is missing on how tweets were translated.

Please read the manuscript properly before commenting. Section 5.2.4. clearly mentions how Tweets are translated. It says the following:

“Microsoft Cognitive Services Text Analytics API [71] enables social media analysis and researchers to implement language detection and translation using the visual interface of Microsoft Power Automate [53]. A non-programmer scientist can easily drag and drop appropriate language detection and translation components and perform dynamic translations as demonstrated in [13] [16] [17] [18] [19] [20] [21] [22]. These research studies demonstrated that live tweets in 110 different languages can easily be comprehended and analyzed without writing a single line of code (i.e., harnessing the power of modern LCNC platforms).”

Honouring your comment, I have also added the following line again within section 6:

“As previously mentioned in section 5, Microsoft Cognitive Services Text Analytics API [71] was used for translation, sentiment analysis, and NER of these Tweets.”

Table 5 is extremely detailed and of little demonstrative use. A graphic would convey this information more suitably.

Many thanks for this suggestion. Table 5 shows the results of applying AI/ML algorithms using LCNC platform (e.g., translation, sentiment analysis etc.). I agree that table 5 is extremely detailed. Hence, I have now placed this within the appendix section.

However, I disagree with the comment “A graphic would convey this information more suitably.”. Table 5 have 13 columns representing 13 dimensions of the presented information. A 3D scatter chart with color-codes and size attribute could only represent up-to 5 dimensions. Hence, A single graphic showing 13 dimensions needs to go through several steps of data transformations for reducing dimensions as shown in https://ieeexplore.ieee.org/ielaam/2945/7835767/7784854-aam.pdf.

Figure 7 and Figure 8 are not suitable for publication - proper screenshots should be obtained.

Both Figure 7 and Figure 8 have been captured at a higher resolution of 330 ppi (HD quality). I have previously captured device screenshot even in lower resolution than Fig. 7 and Fig. 8, and they have been successfully published by renowned journal (with impact factor of up to 13) and well-established publisher (e.g., IEEE, Elsevier, Springer Nature, MDPI etc.). I have already provided links to these publications with device screenshots (similar to Fig. 7 and Fig. 8) within Observation 3 as evidence. I am not sure, why you think that these are not suitable for publication.

I have already provided my response on this. Please refer to Observation 1.

Overall the aim of the article has significant merit and would be of interest to the readership.

Many thanks for comment. I also believe that it would attract significant interest towards the readers.

The article is limited significantly by incomprehensible english and superfluous detail not related to the main aims of the article.

While all other reviewers (i.e., 4 reviewers) confidently selected “English language and style are fine/minor spell check required”, I am doubtful about your comment on “incomprehensible English”. Especially, when you did not even provide a single example of a sentence that is incomprehensible from the manuscript. Rather than throwing random and incongruous comments, please provide some examples for your future reviews, so that the author can take precise actions on the manuscript. Being the head of department for Federal Government agencies like Defence for the last 12 years and being directly associated one of leading research institute (within Top 50 in global ranking), I always provide actionable feedback (i.e., not vague comments).

In addition, while this revision expands on the discussion on the applications of such LCNC algorithms, further discussion of these applications to highlight their usefulness is warranted.

Please note that the application of LCNC based algorithms and usefulness in solving research problems are already highlighted in paragraph 2, 3, and 4 of Section 5. Moreover, Table 3 and Table 4 demonstrates the usefulness on LCNC platform. Please refer to the paragraphs below:

“Similarly, researchers analyzing human behavior or perception of society might want to execute NLP algorithms like sentiment analysis, NER, category classification, language detection & translations. Since sociologists, behavioral scientists, and political scientists are often not comfortable in hand-coding NLP Algorithms, LCNC platforms allow them a quick and easy approach to analyze their data. For example, using the LCNC platform, a social scientist may analyze anti-vaccine sentiments without hand-coding NLP algorithms [21]. Likewise, a political scientist or researcher may obtain critical insights on multidimensional geopolitical impact arising from COVID-19 as shown in [13] [22].

Since data-driven multidisciplinary research is gaining popularity in recent times, studies in [18] [19] [20] automatically obtained data from several thousand sources (e.g., Twitter, CNN, BBC, New York Times, etc.) using AI-based data acquisition features of LCNC platforms.

Social scientists and researchers working with strategic decision-makers are often unable to create mobile Apps for obtaining AI-based insights. As demonstrated in [23] [24] [25], social scientists easily created mobile apps running in iOS, Android, and Windows for evidence-based policy making using the LCNC platform. Without LCNC platforms, the researchers would need to hire expert coders for developing mobile apps or AI-based algorithms. Table 3 summarizes different features of LCNC platforms currently used by re-searchers explaining RQ 3. In terms of answering RQ 4 (i.e., LCNC platforms used in re-search), Microsoft Power Platform is the most popular, followed by Mendix, SetXRM, vf-OS Platform, Aurea BPM, CRISP_DM, Primary AI and others as shown in Table 4. According to Table 4, LCNC platforms supported multidisciplinary research in the area of manufacturing, supply chain management, software development, business process automation, Education, global news analysis, COVID-19, social media analysis, and even disaster management (e.g., Landslide, Tornado etc.). This answers RQ 5 or various areas of research supported by existing LCNC platforms.”.

Moreover, this suggestion constructively, I have added the following new paragraph as the end of section 6 to highlight the usefulness of LCNC platform:

“As seen in this section, using the LCNC platform a researcher can easily obtain data from multiple sources (e.g., social media, online NEWS sites, online databases) and apply AI algorithms to deploy not only cyber intelligence solution, but also political threat intelligence, COVID-19 intelligence, social cohesion intelligence, military intelligence and many other innovative solutions.”.

Further, the Cyber Attack analysis lacks the detail required to reproduce this.

Many thanks for this valuable comment. Taking this valuable input, I have made the source code (as well as the data) used within this demonstration publicly available at https://github.com/DrSufi/COVID_Index_Anomaly. Anyone can download this solution and reproduce this research outcome using the details presented in the study.

Moreover, within section 6, I have now added the following details:

“There were following two different types of data obtained for this demonstration:

Realtime Cyber-Attack Data collected from Anti-Virus Vendors (i.e.., https://statistics.securelist.com/)
Realtime Cyber related Twitter Feeds obtained using Twitter API (i.e., https://developer.twitter.com/en/portal/dashboard)

…

Research reproducibility is without any doubt a critical factor for open research. Hence, in the last 1 year almost all my research associated data and software solutions are hosted by peer reviewed journals like MethodsX (Elsevier), International Journal of Information Management Data Insights (Elsevier) and Software Impact (Elsevier). Following are some selected examples:

https://methods-x.com/article/S2215-0161(22)00334-X/fulltext
https://doi.org/10.1016/j.jjimei.2022.100074
https://doi.org/10.1016/j.simpa.2021.100177
https://doi.org/10.1016/j.simpa.2022.100218
https://doi.org/10.1016/j.simpa.2022.100357
https://doi.org/10.1016/j.simpa.2022.100319

Professional editing and senior input is recommended.

Generally, when I review papers, I provide several examples from the reviewed manuscript to back-up my point. It gives the author, a clear picture of where the manuscript requires updating. However, blatantly commenting “Professional editing and senior input is recommended.” without any references or examples is misguiding and creates suspicion.

English being my first language, my 350-page long PhD thesis (that I wrote) received the best PhD thesis award from RMIT University in 2011 (https://fahimsuficom.globat.com/img013.jpg). Moreover, I have authored about 17 Journals papers within last 12 months (as evident from https://scholar.google.com/citations?hl=en&user=NtC8BsgAAAAJ&view_op=list_works&sortby=pubdate). Almost all these publications, I was either the only author or the main contributing author (i.e., first author and corresponding author). None of these papers (published by reputed publishers like IEEE, Springer, Elsevier, MDPI) had a history of being recommended for professional editing.

Moreover, all other reviewers for this manuscript deemed that English language and style for this paper is fine recommending minor spell check (as seen below):

Reviewer 1: (x) English language and style are fine/minor spell check required
Reviewer 2: (x) English language and style are fine/minor spell check required
Reviewer 3: (x) English language and style are fine/minor spell check required
Reviewer 4: (x) English language and style are fine/minor spell check required

Considering all the valuable comments of the reviewers, I have updated this manuscript thoroughly and believe that it no longer requires professional editing service.

Article Menu

Algorithms in Low-Code-No-Code for Research Applications: A Practical Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI