CrowDSL: Platform for Incidents Management in a Smart City Context

Round 1

Reviewer 1 Report

1. Line 16, 'the last years, the Internet of Things', missing words in the beginning?
2. Line 296, could you add a table or a description to detail out the proportion of users in different skill levels?
3. Can you switch the order between Table 2 and Table 3?
4. Can you align the table, or figure captions?
5. How scalable is the CrowDSL tool, what's the data scale it can handle?
6. Suggest the author add some evaluation for the scalability, performance for the tool and method, along with the user surveys.

Author Response

1. x) Moderate English changes required

• We have read, checked, reviewed, and corrected the article many times.

Does the introduction provide sufficient background and include all relevant references?

( )           (x)          ( )           ( )

• We have changed the introduction including more information about the problematic and the contribution.

Is the research design appropriate?

( )           (x)          ( )           ( )

• We have added more explanations in the introduction, the methodology, and the conclusions to improve it.

Are the methods adequately described?

( )           (x)          ( )           ( )

• We have added more explanations in the methodology to improve it.

Are the results clearly presented?

( )           ( )           (x)          ( )

• We have improved the results, incorporating more information and more comments in them.

Are the conclusions supported by the results?

( )           (x)          ( )           ( r)

• We have improved the conclusions, incorporating more information and more comments in them, and linking better with the results.

Line 16, 'the last years, the Internet of Things', missing words in the beginning?

• Thank you so much for seeing this typo. We have corrected it.

Line 296, could you add a table or a description to detail out the proportion of users in different skill levels?

• Thank you so much, we think that this can improve the article. We have added the age range of the users, and a table with the number of users according to their level in IT and the IoT.

Can you switch the order between Table 2 and Table 3?

• We switched the order between Table 2 and Table 3 as suggested by the reviewer. Thank you for your suggestion.

Can you align the table, or figure captions?

• All the tables and all the figure captions are now centered. Thank you for the suggestion.

How scalable is the CrowDSL tool, what's the data scale it can handle?

• Thank you for your question. CrowDSL relies on standard technologies like XML, Python or JSON and well-established design patters and frameworks. So, the data scale it can handle depends mainly on the specific features of each computer that uses it. As future work we will perform load tests and other types of tests to obtain concrete values.

Suggest the author add some evaluation for the scalability, performance for the tool and method, along with the user surveys.

• Thank you. We think that evaluation is a very interesting work that, since it is not the focus of the work we have done and is more related to the specific features of the computers that use the language (mainly depending on the available memory and other computer’s resources) we left it as a future work.

Reviewer 2 Report

1. Some abbreviations used before defined such as DSL.
2. Correct the order of figures and tables.
3. Abstract and Summary should be revised.
4. Add some latest references from 2021/2020.

Author Response

(x) English language and style are fine/minor spell check required

• We have read, checked, reviewed, and corrected the article many times.

Does the introduction provide sufficient background and include all relevant references?

( )           (x)          ( )           ( )

• We have changed the introduction including more information about the problematic and the contribution.

Are the conclusions supported by the results?

( )           (x)          ( )           ( )

• We have improved the conclusions, incorporating more information and more comments in them, and linking it better with the results.

Some abbreviations used before defined such as DSL.

Thank you so much. We probably forgot to change someone to the beginning of the document. Now, we have re-checked the article and we have:

• Corrected the XMI acronym in its first appearance (line 78) and XML in 76.
• Changed app for application (line 121, 122, and 125.
• Added abbreviations for FixMyStreet (FMS) in line 129.
• Added JavaScript Object Notation (JSON) in line 194.
• Added JavaScript Object Notation for Linked Data (JSON-LD) in line 240.

Correct the order of figures and tables.

• Thank you for the suggestion. The order of figures and tables is now corrected.

Abstract and Summary should be revised.

• Thanks for the comment. We have reviewed the whole article and improved it. We hope that now everything is clearer.

Add some latest references from 2021/2020.

• Thank you, we have now included some recent and relevant references.

Reviewer 3 Report

This work describes a Model-driven approach to the design of an incident management system. The authors propose a Domain-specific language and implemented an editor where users can report incident using the language concepts. The editor was evaluated with 20 users who generally expressed satisfaction with the interface.

The contribution of the work is not impressive and there seems to a lack of clarity about the nature of contribution itself. For example, the introduction begins with a long discussion about smart cities, IoT, smart homes, but basically everything before line 60 seems to be pretty much unrelated with the essence of the work. Perhaps this initial part could be more useful, if it presented the problem being addressed by this work. Also, the name of the submission refers to a platform, but the focus seems to be on the editor and its evaluation or maybe in the proposed language. Finally, the authors never explicitly mention what their contribution is, leaving to the readers to take their own conclusions. The only place where there is a mention to a contribution is the very last sentence and it just indicates that the contribution is the use of MDE, suggesting without any support from the results that this has allowed the authors to achieve reusability, interoperability and portability.

As for the user study, it seems reasonably structured, but the evaluation goal does not seem very relevant for everything else being described in this work. It seems to focus on the interface of editor, which could itself be well-designed or badly-designed, regardless of the merits of the MDE approach. It is also puzzling what it means and what could be the relevance for the study that users have different levels of knowledge about IoT, or why should they prefer a text interface rather a graphical one.

Overall, there seems to some work regarding incident modelling, but this submission does not seem to offer a relevant research contribution

Author Response

(x) English language and style are fine/minor spell check required

• We have read, checked, reviewed, and corrected the article many times.

Does the introduction provide sufficient background and include all relevant references?

( )           (x)          ( )           ( )

• We have changed the introduction including more information about the problematic and the contribution.

Is the research design appropriate?

( )           ( )           (x)          ( )

• We have added more explanations in the introduction, the methodology, and the conclusions to improve it.

Are the methods adequately described?

( )           (x)          ( )           ( )

• We have added more explanations in the methodology to improve it.

Are the results clearly presented?

( )           (x)          ( )           ( )

• We have improved the results, incorporating more information and more comments in them.

Are the conclusions supported by the results?

( )           ( )           (x)          ( )

• We have improved the conclusions, incorporating more information and more comments in them, and linking better with the results.

The contribution of the work is not impressive and there seems to a lack of clarity about the nature of contribution itself. For example, the introduction begins with a long discussion about smart cities, IoT, smart homes, but basically everything before line 60 seems to be pretty much unrelated with the essence of the work. Perhaps this initial part could be more useful, if it presented the problem being addressed by this work.

Thank you so much for your valuable suggestions. We have made the next changes.

• We have introduced the problematic around the line 27 and we have tried to create better links between the introduction to the different concepts and the proposal.
• We have added more differences with the related works and the lack in the literature. Basically, we have evaluated the platform with users. Then, we have obtained results to know the opinion of the people who would have to use the DSL which has been developed to them. We have added this information in different parts of the paper.

Also, the name of the submission refers to a platform, but the focus seems to be on the editor and its evaluation or maybe in the proposed language.

• Thank you so much for the appraisal. This is a platform in which we use a DSL. Here, the important thing is the combination of both. We have improved the whole article and we hope that now it is clearer.

Finally, the authors never explicitly mention what their contribution is, leaving to the readers to take their own conclusions. The only place where there is a mention to a contribution is the very last sentence and it just indicates that the contribution is the use of MDE, suggesting without any support from the results that this has allowed the authors to achieve reusability, interoperability and portability.

Thanks a lot for this opinion because it is very valuable to know it. Sometimes authors think that everything is clear and as we can see now, we have to improve it. We have tried to solve in the next ways:

• We have included more information in the introduction about our contribution.
• We have added more differences with the related works and the lack in the literature. Basically, we have evaluated the platform with users. Then, we have obtained results to know the opinion of the people who would have to use the DSL which has been. We have added this information in different parts of the paper.
• We have improved the conclusions too, adding more explanations about the contributions and the differences.

As for the user study, it seems reasonably structured, but the evaluation goal does not seem very relevant for everything else being described in this work. It seems to focus on the interface of editor, which could itself be well-designed or badly-designed, regardless of the merits of the MDE approach.

• Thank you very much for your question. Please, note that the focus is to validate the concrete syntax of the DSL, and which will also indirectly validate its abstract syntax and therefore its semantics, what together make up the main parts of the DSL from the point of view of the MDE approach.

It is also puzzling what it means and what could be the relevance for the study that users have different levels of knowledge about IoT, or why should they prefer a text interface rather a graphical one.

• Thank you for this important note. Now, we have added an explanation in the methodology and the conclusions to improve it.

Overall, there seems to some work regarding incident modelling, but this submission does not seem to offer a relevant research contribution.

• Thank you very much for your suggestions and recommendations. We hope that with the changes made based on the suggestions of all the reviewers, the contribution of the work is now better and clearer.

Round 2

Reviewer 1 Report

1. Improve the figure 5, figure 6 resolution
2. Figure alignment for Figure 3, make it centered?
3. Line 485, the list of terminologies, the format is off.

Author Response

English language and style

(x) English language and style are fine/minor spell check required

• We have read, checked, reviewed, and corrected the article many times by us and other 2 colleagues.

Improve the figure 5, figure 6 resolution

• After seeing it, we noticed that they were a little blurry. We have improved the quality of these figures. In figure 6, we have change a lot the style (colors, patterns, letter, size, etc.) to improve it.

Figure alignment for Figure 3, make it centered?

• Thanks so much for noticing it to us. We have checked all figures and their source code, and centered them.

Line 485, the list of terminologies, the format is off.

• Thank you so much for noticing it and so sorry for the mistake. We forgot or lost a ‘}’ in the Latex template.

Reviewer 3 Report

The authors appeared to have made an effort to improve the submission, but these changes do not seem enough to overcome the serious flaws of the original submission. There were multiple incoherences in the original submission and they seemed to remain in this revised version. My original decision was precisely based on the idea that this was not about introducing some changes (even major ones), it was about revising the whole story of the submission. It makes many claims about IoT, but the study itself does not really seem to be about IoT. It makes many claims about how their usage of Model-Driven Engineering and a specific Domain Language could support reusability, interoperability and portability, but their data is about the usage of an editor. It provides a broad view of related work, but the end it does not really identifies what major limitations of previous work are actually being addressed (not using MDE or not having a DSL are not necessarily limitations) and finally it does not define concrete research objectives that translate in the end into a set of contributions supported by the study data.

It remain very hard to understand what is the take-way of this work, other than the idea that it is possible to use MDE to create an incidents management platform.

Author Response

(x) Moderate English changes required

• We have read, checked, reviewed, and corrected the article many times by us and other 2 colleagues.

The authors appeared to have made an effort to improve the submission, but these changes do not seem enough to overcome the serious flaws of the original submission. There were multiple incoherences in the original submission and they seemed to remain in this revised version. My original decision was precisely based on the idea that this was not about introducing some changes (even major ones), it was about revising the whole story of the submission.

• Thank you so much. We already did a great effort to improve the whole article according to the reviews, which in general were very useful, and our experience in the field. We tried to do our best. However, sometimes, maybe because people in this field (researching) are very occupied and busy they can forget to describe the flaws and problems very well; we are talking about everyone: writers (our case), reviewers, and editors. In this case, if the reviewers do not give us all the feedback that they believe we need to improve the article, we cannot do much more to improve it in the way that that reviewer believes is better (we think that the reviewer proposes changes to improve the article in a good way). We mean, we read the reviews, we change the things in which we agree with the reviewers, and other things that we notice (maybe because we read the whole article several times or maybe because after reading some review and then we realize something). However, if you do not explain in the review letter your opinion or your thoughts very well, we cannot do more. However, we thank you because we think it was a valuable first review and we improved the article so much thanks to you. Nevertheless, if you do not give to us all the incoherencies and flaws that you believe the article has, we can just change your suggestions, trying to interpret others, and other things that we notice when we are correcting and reading the article. This last example is like now, in which you say incoherencies and flaws, or the whole story, but you have not explained which ones. Furthermore, we answer all your petitions of the first review, and we believe that we have corrected them in a right way because you do not say anything here about that. Besides, the other two reviewers seem to agree with our changes and this article. That is why maybe the editor gave us the opportunity to change and improve it.

It makes many claims about IoT, but the study itself does not really seem to be about IoT.

• So sorry but here we strongly disagree. As stated in the document, smart Cities are a subfield inside the IoT. You can find an loT literature about this. Besides, here you have one thesis (in Spanish) in which they explain the different subfields and you can see the different references to that: C. González García, “MIDGAR: Interoperabilidad de objetos en el marco de Internet de las Cosas mediante el uso de Ingeniería Dirigida por Modelos,” University of Oviedo, 2017. https://www.researchgate.net/publication/314188769_MIDGAR_interoperabilidad_de_objetos_en_el_marco_de_Internet_de_las_Cosas_mediante_el_uso_de_Ingenieria_Dirigida_por_Modelos

It makes many claims about how their usage of Model-Driven Engineering and a specific Domain Language could support reusability, interoperability and portability, but their data is about the usage of an editor.

• Our DSL follows the MDE principles. In addition, DSLs can be (among others) textual or graphic. Here, our DSL is graphical. Thus, the ‘editor’ includes different artifacts such as the graphical DSL and other complementary tools to improve the user experience like the drag & drop infrastructure, forms, text boxes, etc. Then, we think that it may be fair to use the data of the ‘editor’ to evaluate the DSL. The same approach have been used in many other research works (some of them are cited in this article in the Methodology subsection).

It provides a broad view of related work, but the end it does not really identifies what major limitations of previous work are actually being addressed (not using MDE or not having a DSL are not necessarily limitations)

• Note that not using MDE or a DSL are not intrinsic limitations, but they are usually the preferred option in the software engineering community to create an adequate ‘language’ to allow users to work in a specific domain.
• In addition, with the use of MDE, it is possible to solve or minimize some of the main problems of software development (low quality software, incorrect estimations, maintenance, interoperability, etc.). Then, this is a way of how to design and create software. Besides, applying MDE, we can do a very deep study of the necessities for solving a problem and the stages that can be automated. After that, it is possible to create a very good DSL because you have all the information of the problem.
• With the use of models, we can create a tailored language which can allow the reusability in the source code, can automate different phases, can abstract more the problem and different elements, and can be very oriented to the users, facilitating the use of that application.
• Then, the created DSL represents the problem that was studied with MDE and allow users to use that exact part. The problem here is if the DSL represents well the specific domain.

Then, how can we measure the quality of this research and how can we know if the DSL has been created correctly and represent the intended problem? Using the data of the ‘editor’ to know if this work is correct according to the use of it and the opinion of the participants.

• According to the related work, they did not test the systems with users, so, they may create an application which users could like or dislike. In our case, we have created a DSL to solve the problem of reporting incidents in Smart Cities and we have tested with users to whether this is a good way or not to solve this problem.

and finally it does not define concrete research objectives that translate in the end into a set of contributions supported by the study data. It remain very hard to understand what is the take-way of this work, other than the idea that it is possible to use MDE to create an incidents management platform.

• In summary, we suggest that the use of MDE to create a DSL is a good idea to solve the problem of the communication of incidents in a Smart City. Until the moment of this article, some applications appeared but they did not test with real users, and they did not apply some specific strategy to create them

Round 3

Reviewer 3 Report

no comments