Digital Service Quality Measurement Model Proposal and Prototype Development

Round 1

Reviewer 1 Report (Previous Reviewer 4)

Comments and Suggestions for Authors

I this manuscript authors presented Digital Service Quality Measurement Model and implement it on  463,886 posts sent to the X account of the chosen municipality were analysed using 18 the text mining method. Manuscript has its merit but organization and presentation of the material should be improved.

1. Introduction should be shorter. The authors need to present problem, research goal, research questions or hypothesis and to give possible novelty. Some of the material should be moved to the conceptual background.

2. Section Conceptual background need to provide state of the art contribution and analysis of existing model and trends and less just to present theory.

3. Findings, Discussion and Conclusion should be rewritten. Authors need to present novelty of their modela and to compare and contrast their model with existing ones. The advantages and disadvantages of the model should be presented. In addition it could be useful to provide different suggestions for possible stakeholders (using the results of the research). Authors also need to give direct answer on the research questions thy raised.

In conclusion the manuscript needs improvement in the structure, better presentation of the methodology, model and improved discussion and conclusion.

Comments on the Quality of English Language

proof reading 

Author Response

Dear Reviewer,

Thank you for your thorough review and constructive feedback on our manuscript. We greatly appreciate your suggestions for improving the organization and presentation of our material. We have carefully considered your comments and made significant revisions to address your concerns. Below is a summary of the changes we have made:

1. Introduction:

   • We have shortened the Introduction section, focusing on presenting the problem, research goal, research questions, and the novelty of our study. We have moved some of the detailed background information to the Conceptual Background section to streamline the Introduction.

2. Conceptual Background:

   • We have enhanced the Conceptual Background section to provide a more comprehensive state-of-the-art review. This includes a thorough analysis of existing models and trends, highlighting their strengths and weaknesses, and positioning our contribution within the current research landscape.

3. Findings, Discussion, and Conclusion:

   • We have rewritten the Findings, Discussion, and Conclusion sections to emphasize the novelty of our model. We have provided a detailed comparison and contrast with existing models, discussing both the advantages and disadvantages of our approach. Additionally, we have included specific suggestions for stakeholders based on our research results.
   • We have also ensured that the research questions raised in the Introduction are directly answered in the Discussion and Conclusion sections, providing clear and concise responses to each question.

4. Structure and Presentation:

   • Throughout the manuscript, we have improved the overall structure and presentation of the methodology, model, and discussion. These revisions aim to enhance the clarity and coherence of our work, making it more accessible and informative for readers.

We believe that these revisions significantly strengthen our manuscript and address the concerns you raised. We are confident that the improved structure, clearer presentation of the methodology, and enhanced discussion and conclusion sections will meet the high standards expected for publication.

Thank you again for your valuable feedback and guidance. We look forward to your positive response.

Sincerely,

Reviewer 2 Report (Previous Reviewer 3)

Comments and Suggestions for Authors

Dear Editor and Authors,

 

I would like to express my gratitude for the opportunity to review the manuscript once again. The authors have taken my comments on board and made improvements to the article following my suggestions. The structure of the paper is now logical and predictable, which contributes to its readability. The theoretical part has been supplemented, with the authors referring to their previous work on developing a model service quality measurement using sentiment analysis and text mining techniques. The conclusion section has also been improved, with the authors referring to the presented literature and providing a more in-depth discussion.

Furthermore, it would be beneficial to include a discussion of the limitations of the study and suggestions for future research.

Author Response

Dear Reviewer,

Thank you very much for the opportunity to review our manuscript again and for your constructive feedback. We are pleased to hear that the revisions we made have improved the structure and readability of the paper. We have taken your suggestions seriously and have made further enhancements to address your comments.

1. Improved Structure and Readability:

   • We appreciate your positive feedback on the logical and predictable structure of the paper. We aimed to enhance the flow and coherence of the manuscript, making it easier for readers to follow our arguments and findings.

2. Theoretical Supplementation:

   • As per your suggestion, we have supplemented the theoretical part by referring to our previous work on developing a model service quality measurement using sentiment analysis and text mining techniques. This provides a stronger foundation for our current study and clarifies the continuity of our research.

3. Enhanced Conclusion:

   • We have improved the conclusion section by providing a more in-depth discussion that ties our findings to the existing literature. This helps to contextualize our contributions and highlights the relevance of our work within the broader academic discourse.

4. Discussion of Limitations and Future Research:

   • In response to your recommendation, we have included a discussion of the limitations of our study. We have addressed potential biases, the constraints of using social media data, and the limitations of the tools employed. Additionally, we have outlined suggestions for future research to build upon our findings and address these limitations.

We believe these revisions further strengthen our manuscript and we are grateful for your guidance in improving the quality of our work. Thank you again for your valuable feedback and support.

Sincerely,

Reviewer 3 Report (New Reviewer)

Comments and Suggestions for Authors

The manuscript has potential but needs some improvements to achieve its purpose. The advantage of the paper is a good review of the literature and an innovative research approach. On the other hand, the work seems unfinished. In this context, authors should consider the following:

1) In the "Introduction" section, the authors list several questions. In the "Discussion" section, authors should clearly address the answers to the above questions. In the "Conslusion" section, they should add a text that will summarize the answers in a concise way.

2) In the literature review, the authors refer to various models such as SERVQUAL, SERVPERF, and others. The discussion did not provide an overview of the connection between the mentioned models and the obtained results.

3) What is the proposed model? The aforementioned models have a clear structure and constructs. Authors should clearly present their model as this is the key objective of their manuscript.

4) Does the research have limitations? Authors should clearly state the limitations.

In general, the paper seems unfinished because it lacks a clear presentation of the new model and a comparison with the starting models or those currently present in the literature.

Author Response

Dear Reviewers,

Firstly, we would like to thank you for your thorough review and valuable feedback on our manuscript. Based on your comments, we have made several revisions to improve the paper. Below is a detailed summary of the changes made in response to your suggestions:

1. Introduction and Conclusion Sections:
• We have addressed the questions listed in the Introduction section more clearly in the Discussion section. Additionally, we have added a concise summary of the answers in the Conclusion section.
2. Literature Review and Connection with Results:
• In the literature review, we have discussed the connection between the mentioned models such as SERVQUAL, SERVPERF, and the obtained results. We highlighted the strengths and weaknesses of these models and explained how our findings align with them.
3. Definition of the Proposed Model:
• We have clearly presented the structure and components of the proposed model. We described how our model differs from existing models like SERVQUAL and SERVPERF and what innovative approaches it incorporates. Figures and tables have been added to visualize the steps and functioning of our model.
4. Research Limitations:
• We have explicitly stated the limitations of our research. We discussed the potential biases due to the use of social media data, the impact of political factors on the data, the limitations of the MAXQDA software, and the temporal constraints of the analyzed data.

We believe these revisions enhance the scientific contribution of our manuscript and provide a clearer understanding of our research findings. We hope that these changes have made our paper more comprehensive and readable.

Thank you once again for your constructive feedback.

Sincerely,

Round 2

Reviewer 1 Report (Previous Reviewer 4)

Comments and Suggestions for Authors

Authors corrected manuscript according to remarks.

Comments on the Quality of English Language

proof reading

Reviewer 3 Report (New Reviewer)

Comments and Suggestions for Authors

No comments.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This article is very difficult to classify as a definite result of scientific research. Most likely, it is some report on the implementation of a practical project funded by some municipality or a student essay. This thesis is confirmed by the fact that the authors could not even formulate the purpose of the study, which could be characterized as a kind of scientific project. The authors also failed to identify specific tasks to achieve the goal (there is no adequate goal - there are no adequate tasks for the implementation of the study). Section 3 of the article is titled by the authors as "Methodology", however, the authors are silent about the methodology of the implementation of scientific research, and this, apparently, is due to the fact that the authors do not own such a methodology. The material of the article, which the authors classify as a model, is an arbitrary set of images (probably in the form of graphs, for which neither the vertices nor the edges of this graph displaying the corresponding connections between the vertices are uniquely defined). The authors talk about a certain quality of municipal services, however, they do not define methods, methods and adequate criteria for assessing such quality. Conclusions based on the materials of the article are generally absent. The arguments presented by the authors in the "Conclusion" section only confirm my thesis regarding the lack of a scientific component in this work.

Author Response

Thank you for your valuable comments. We have carefully analysed what you have written to improve the study. First of all, we would like to state that the study does not receive financial support from any institution. Studies similar to the method we have put forward in the study are mentioned in the literature section. In the studies we examined in the literature, it is recommended to use sentiment analysis instead of service. When these studies were examined, it was determined that the text-mining method was not applied. In this study, the keywords that will reveal the service quality were extracted from the tweets sent using the text mining method. We realised that we could not emphasise this enough in the study. The body of the study has been reorganised by considering your opinions on this issue. The matrices and figures used to determine the keywords are the data we obtained in the text mining application. These reveal the relationships of keywords with each other. This explanation has also been added to the study. The methodology section has been improved in line with your comments. Findings on how service quality is evaluated have been added to the study. The conclusion has been evaluated in light of the existing studies in the literature. Thank you for your opinions.

Reviewer 2 Report

Comments and Suggestions for Authors

The article presents a study on municipal service quality measurement using digital tools. It emphasizes the importance of evaluating municipal services to ensure efficient and effective public service delivery. The authors propose a digital approach, leveraging text mining and sentiment analysis, to analyze public opinions and feedback on municipal services. This method aims to enhance the understanding of public needs and improve service quality.

The literature review in the article covers various aspects of digital tools in public service evaluation. It discusses previous studies on text mining and sentiment analysis, highlighting their applications in different fields, including public administration. The authors reference works that have utilized these techniques for analyzing large volumes of text data, extracting insights from unstructured data sources like social media, customer feedback, and online reviews. This background sets the stage for the proposed methodology, demonstrating its relevance and potential effectiveness in the context of municipal service quality assessment.

The core hypothesis of the study is that text mining and sentiment analysis can effectively measure and analyze public opinions on municipal services. To test this, the authors collected data from Twitter, where residents discuss municipal services. Using text mining techniques, they extracted keywords and phrases indicative of public sentiment. Sentiment analysis was then applied to categorize these into positive, negative, or neutral sentiments.

 

Although the subject is interesting and the article relatively well written, I have several questions and comments that lead me to propose some major changes.

Theoretical Positioning

The expectation set by the title suggests a critique of traditional methods for measuring service quality, followed by a proposition for a digital alternative. However, the literature review deviates from this path. The authors begin with a focus on Twitter (now known as X) and attempt to justify data collection from this social media platform. This approach seems tangential to the central thesis of proposing a digital methodology for service quality assessment.

Furthermore, the critique of the SERVQUAL/SERVPERF model is limited and somewhat misdirected. The authors argue that these models lack adaptability to various sectors and that municipalities offer diverse services, necessitating a new model. This critique overlooks the inherent flexibility of the SERVQUAL model, which is designed to be service-neutral, focusing on universal service quality dimensions like accessibility and reliability. The practical aspects, primarily in the tangible dimension, can be adapted to various scenarios, contradicting the authors' justification for a new model.

The article then shifts to discussing text mining and sentiment analysis, which, while methodologically relevant, diverge from the theoretical discussion. This shift indicates a lack of clarity in the article's focus. Are the authors critiquing the limitations of public service measurement methods and proposing a digital solution, or are they presenting a methodological paper on text mining and sentiment analysis with municipal service quality as a case study?

This ambiguity in positioning leads to a disjointed narrative. The theoretical and methodological elements seem disconnected, with the former not adequately setting the stage for the latter. The discussion on text mining and sentiment analysis, while methodologically sound, appears more as an independent topic rather than an integrated part of a cohesive argument about service quality measurement.

In conclusion, the article suffers from a lack of clear positioning. The theoretical critique of existing models is insufficiently developed, and the transition to methodological discussions is abrupt and somewhat disjointed. For a more coherent and impactful paper, the authors need to clarify their stance: either focus on the theoretical limitations of current public service measurement methods and how digital approaches can address these gaps, or present a methodological exploration of text mining and sentiment analysis within the context of municipal services. This clarity will not only strengthen the argument but also provide a more structured and focused narrative.

 

Methodology & Results

 

The methodology employed presents several areas of concern that merit critical examination. These include the lack of comprehensive quality indicators, unclear sections, and the ambiguity surrounding the appendices.

 

Quality Indicators

A significant limitation of the study is the absence of robust quality indicators for the results. While a confusion matrix is used to demonstrate the accuracy of sentiment analysis, this alone is insufficient to establish the reliability and validity of the overall findings. In scientific research, especially when proposing a new methodology as an alternative to traditional approaches, it is crucial to provide comprehensive quality indicators. These indicators could include measures of reliability, validity, and error rates, among others. Without these, the reader is left to take the authors' claims at face value, which undermines the credibility of the research and hinders its acceptance as a convincing alternative to traditional service quality measurement methods.

 

Section 3.4

Section 3.4 of the paper (p.8) lacks clarity, leaving readers uncertain about certain aspects of the methodology. It is unclear whether a service quality questionnaire was also implemented as part of the study. If so, details regarding its design, implementation, and how it integrates with the digital approach are not adequately explained. This lack of clarity hampers the reader's understanding of the full scope and nature of the research methodology.

Furthermore, the relevance of the final paragraph of this section (p.9, "To access Tweet management...") is questionable. Its connection to the overall methodology and how it contributes to the study's objectives is not clearly articulated, leading to confusion about its purpose in the context of the research.

 

Appendices

The co-occurrence matrix, particularly the one related to public transport services, is a point of confusion. The terms used in the matrix are in Turkish, and it is not clear whether they refer to names of municipalities, attributes of service quality, or something else entirely. This lack of clarity impedes the reader's ability to interpret the tables and understand their significance in the context of the study. For non-Turkish speakers, this presents a significant barrier to fully grasping the research findings.

Moreover, the table titled "public transport services" seems to contain terms that do not align with city names (e.g., Tesis, Veteriner, Lokal). This discrepancy raises questions about the accuracy and relevance of the data presented. Without clear explanations or corrections, these tables remain ambiguous and diminish the paper's overall comprehensibility and credibility.

In addition, there is no reference to appendices in the text. As a result, it's not clear when you need to turn to them to understand this research.

 

In conclusion, the methodology of the study is hindered by significant shortcomings. The lack of comprehensive quality indicators, unclear sections, and ambiguous data presentation in the co-occurrence matrix all contribute to a less convincing argument for the proposed methodology as a viable alternative to traditional service quality measurement methods. Addressing these issues is crucial for enhancing the scientific rigor and clarity of the research, thereby strengthening its contribution to the field.

 

 

Author Response

Dear Reviewer, thank you for your detailed review and comments. Thanks to your in-depth comments, our study has come to a better point. 

Service quality models such as SERVQUAL and SERVPERF are referred to as traditional models. In the proposed service quality model, the data collection, processing and display stages are completely digital. for this reason, the title of the study is "A Digital Proposal for Municipal Service Quality Measurement". Considering your and other referees' comments, the new title of the study has been changed to "Development of Service Quality Measurement Model with Sentiment Analysis and Text Mining: A Municipal Case Study From Turkey". 

The titles and flow of the study have been brought in line with the title. thank you for drawing attention to this issue. In addition to traditional service quality models, studies using sentiment analysis technique and municipal studies using text mining have been added to the study. in this way, we think that we better express the main issue we present in the title of the study.  

SERVQUAL/SERVPERF or traditional service quality models are survey-based. In order to collect data, a one-to-one questionnaire needs to be administered to people. In addition, validity and reliability tests need to be carried out again in each sector for the scale items. In addition, factor analyses are required for service quality dimensions. When we look at the studies using traditional models, the samples are between 300-400 people. In the literature, authors mention their difficulties. One of these difficulties is the difficulty of remembering. Because the time when the service is received and the time when the questionnaire is filled out are different. The model we propose overcomes this difficulty. In order to present the proposed model, 463886 tweets that occurred in a long period of 8 years were used. The size of the processed data increases the validity of the research. 

The lack of clarity in the flow of the paper has been addressed. New headings have been added to the article to address the deficiencies you have pointed out. There are 23 studies in the literature on service quality and sentiment analysis. These studies have been added to the article. The difference between our proposed model and these studies is presented. While presenting a new service quality model in the study, a case study was conducted to demonstrate the usability of the proposed model. While presenting the model for the first time, the municipality is taken as an example to enable a good data set to be obtained. 

Thank you for pointing out the discrepancies in the narrative. In the edited version of the paper, the narrative discrepancies have been corrected. The theory of the study is better addressed. The changes allow us to present the arguments in the paper more clearly. 

Agrees with the elements you have stated in the theoretical positioning. For this reason, the outline of the study has been modified. While designing the SERVQUAL model, 10 service quality dimensions were put forward. These service quality dimensions were then reduced to five. The number of items in the scale is stated as 22. For example, one of the service quality dimensions is tangibles. The scale items revealing this dimension should be rewritten for each sector. In the literature, it has been observed that these items are handled differently even for the same sectors. There are statistical tests to be performed in each adaptation. Collecting data in these models is laborious. There is a difference between when the service is received and when the questionnaire is answered. These difficulties are mentioned in the literature. For this reason, in the model we propose, data are taken from Twitter. For example, there are 463886 tweets in an 8-year period for the municipality we are considering. 156004 of these Tweets were found to be related to service quality. The model we propose provides an advantage over classical models in terms of sampling and obtaining data. 

The titles of service quality and sentiment analysis and text mining and municipalism have been added to the article. How the sentiment analysis technique is handled in service quality studies is presented by giving examples from the literature. In addition to this, text mining techniques are applied in our proposed model. With text mining, it is tried to determine the response of citizens to the services provided by the municipality. Because people tweet as a result of the service they receive. For example, Tweets about public transport services are much more than funeral services. This gives us an idea about the importance of the services provided by the municipality. 

The ambiguities in the positioning have been tried to be eliminated with the new titles and the changed flow of the article. As a result of the opinions you have expressed

This ambiguity in positioning leads to a disjointed narrative. The theoretical and methodological elements seem to be disconnected from each other, with the former not sufficiently laying the groundwork for the latter. The discussion of text mining and sentiment analysis, while methodologically sound, appears to be a standalone topic rather than an integrated part of a coherent argument for service quality measurement.

As a result, the paper suffers from a lack of clear positioning. The theoretical critique of existing models is not sufficiently developed and the transition to methodological discussions is abrupt and somewhat disjointed. For a more coherent and effective article, the authors need to clarify their stance: they should either focus on the theoretical limitations of existing public service measurement methods and how digital approaches can address these gaps, or present a methodological review of text mining and sentiment analysis in the context of municipal services. This clarity will not only strengthen the argument but also provide a more structured and focussed narrative.

Methodology and Results We are on the same page with the shortcomings you mentioned in the methodology. Thank you for your views. In methodology, there is the matrix of confusion for the accuracy of emotion analysis and the value F1, which is determined from the values obtained from this matrix. As in machine learning based estimation studies, these values reveal the accuracy of the prediction in emotion analysis studies. The training set developed by Microsoft was used to ensure high model success. In text mining, the accuracy of the keywords we reveal has been revealed with code maps and matrices as the authors mentioned in the literature. The proposed model works in this way. For example, while waiting for the bus, you waited too long at the stop and tweeted about it. It is decided by the text mining method that this Tweet is related to the service provided by the municipality. The sentiment status of your Tweet represents your views on this service. If you want to review it, you can visit our web page where data can be observed.when you enter Erhan.system.io website, you can login to the system with the user name mdpi@mdpi.com and password 12345678. You can list Tweets sent to the municipality by searching on any topic. In this list you can see the sentiment analysis for each Tweet. 106 Keywords have been identified regarding the service quality of the municipality. These 106 keywords are related to ten service factors. For example, if you type "metro" in the Tweet analysis section, you will list the Tweets that citizens sent in 2016-2023. Emotion analysis also gives us how successful this service is. Previously, a study was conducted on the water and sewage services of the municipality. The availability of the model we recommend is laid out there. If you want to examine that work, you can reach it at https://dergipark.org.tr/en/download/article-file/3039624. There are a lot of technical aspects to the topic we have put forward in the article. For this reason, we noticed that we could not fully reveal the flow of the article with your opinions. We tried to eliminate the ambiguities. No questionnaire was applied in the study. If the project you propose to Twitter is accepted, you get an api to draw data from Twitter. This api allows you to withdraw data for the past. The study was posted and accepted on Twitter. Thus, in 2016, the shipments of the two official Twitter accounts of the municipality were obtained. When similar studies with emotion analysis were examined, no other study that examined data over such a wide time period was found. The study also covered 463886 Tweets. In this respect, the data set is quite satisfactory. The validity of the work has been tried to be increased with the size of the data handled.

Appendix

Because the data discussed in the study is in Turkish, the program outputs are in Turkish. The English equivalent of the resulting keywords is given in the study. The large amount of data handled causes confusion in ways. In the technique we have revealed, the association matrices about keywords show the relationship of key keliemes to one another. For example, in relation to the size of a service quality, a researcher might look at the coexistence matrix if he wonders why not any words exist. If the word is not included in the union matrix, it does not belong to the service size. However, it should be known that the method we reveal can be applied to different languages. Disruptions caused by the size of the data have been eliminated. In relation to all service dimensions, the aggregate matrices and code maps are included. Service sizes with only two and one keyword do not have code maps and matrices. There must be at least three keywords for this map technique to be implemented. References from relevant parts of the article to annexes were made. Thank you for your warning on this matter.

We would like to express our thanks once again for the detailed review you have made about the article. In this way, we realized our mistakes and made the work more understandable and more scientific.

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Editor and Authors,

Thank you for the opportunity to review the manuscript entitled “A Digital Proposal for Municipal Service Quality Measurement”

Text mining is a specialised domain that applies data mining techniques over text. Sentiment analysis aims to identify and extract opinions, moods and attitudes of individuals and communities. When text mining and sentiment analysis techniques are combined in a project on social media data, the result is often a powerful descriptive or predictive tool. The authors used it for a digital model for service quality measurement.

The structure of the paper is logical and predictable, which contributes to its readability. However, the article is closer to a practical study than a scientific one, which should rigorously address all the relevant elements: the aim and object of the research, the research questions or hypotheses posed based on the literature, the current scientific sources, the methodology and the results presented with theoretical implications, practical implications and limitations of the research carried out along with recommendations for the future.

The theoretical basis of the article is very limited, especially in evaluating the quality of urban services.

In the methodology section authors should explain why they chose the @ankarabbld and @mavimasa Twitter accounts. The methodology is not clear and lacks essential steps of the procedure.

The service quality measurement model design with sentiment analysis and text mining in  Figure 1 should have reference to previous authors' article (OPUS– Journal of Society Research, 20(54), 472-486) where authors already proposed this model for service quality measurement using sentiment analysis and text mining techniques. This model was used also “…seeks to answer how to develop a service quality measurement model from social media data processed with sentiment analysis and text mining methods” (p.480).

The authors should also update the information (in a footnote, for example) that Tweeter has changed to the X platform.

 

The conclusion section is rather more a result section than a conclusion. Authors should refer results to in the presented literature and deeper discussion is needed. Moreover, there are lack of limitations and future research parts.

Author Response

Dear Reviewer, thank you for your valuable opinions. It has come to a better point to work with your views.   In the Study, a new model for service quality measurement was proposed. An application was developed to reveal the validity of the proposed model. Researchers in the literature have discussed the difficulties encountered in classical service quality models. In the Study, these difficulties are included. The challenges that arise in classic service quality models in the proposed model have been tried to be overcome. In addition, more accurate service quality measurement has been tested to be done. The aim and sub-purpose of the Study were developed in line with your views. The scientific aspect of the Study was reinforced with new titles added to the paper. Service quality studies using sentiment analysis techniques were added to the Study. These works are taken from the Web of Science database. Richness was added to the Study by adding bibliometric information about these studies. In addition, examples of the uses of text mining for municipalities were given by adding municipal and text-mining fish to the Study.   Our aim in the Study is to propose a new purpose for service quality measurement. An application has been developed so that this proposal does not remain only theoretical. Since text mining will be used in the proposed model, obtaining rich data is essential in increasing the Study's validity. For this reason, a field of application that concerns more people has been selected. Municipalities are institutions with a lot of services. If the model produces results in municipalization application, it will also be available in different sectors. Ankara Metropolitan Municipality is the capital and the second-largest city of Turkey. It also received the city of mayor award in 2021 for its service quality. Tweets sent to the official accounts of the municipality between 2016-2023 were downloaded through an API. This API was obtained due to an academic project application to Twitter. Thus, such long-time data were obtained.463886 Tweets were detected in the specified date range. A measure of service quality using sentiment analysis technique has not encountered such rich data in the literature. For this reason, the availability of the model we recommend is also possible for different sectors. Thank you for your criticism of the methodology of the work. It has been beneficial for us to see our deficiencies in this regard. To address the flaws in this regard, the literature has been made more prosperous in the first place. In the methodology section, the technical characteristics of the system developed after the theoretical model is explained. In the appendix section, there are results to support our methodology. A confusion matrix and F1 value reveal the model's success for sentiment analysis. Although 463886 Tweets were sent to the relevant accounts of the municipality, not all are related to the quality of service. Text mining techniques have been used to find which tweets contain which words related to the quality of service. The relationships of the words extracted from the resulting tweets are given in code maps and firsthand matrices. I hope that the methodological deficiencies you mentioned have been eliminated. The study "A Proposed Service Quality Measurement Model using Sentiment Analysis and Text Mining: The Case of Water and Sewerage Services," in which the theoretical foundations of the Study are explained, has taken its place in the bibliography. This Study studied a more limited data set, and only one service branch of the municipality was considered. The availability of the model we recommend is demonstrated. In the new Study you examined, a study was carried out, and its application was designed considering all the services provided by the municipality. In the discussion part of the Study, the difference was explained from the studies where service quality was measured using sentiment analysis. In the introduction part of the Study, the new name is indicated by adding footnotes where Twitter first passed. In the conclusion part of the Study, it was recommended to use the proposed model in other municipalities or different sectors and to compare the results. We hope we have addressed any shortcomings you noted with our changes in the Study. Thank you for taking your time and sharing your thoughts. Best regards.   

Reviewer 4 Report

Comments and Suggestions for Authors

The research topic is interesting. The paper is well organized and has an appropriate structure. The references list is appropriate but I would like to see a couple of references from the field of quality management systems, or ISO 9001:2015.  In addition, there is the suggestion to correct the title of the manuscript such as "Service quality measurement model development with sentiment analysis and text mining".

The research question is very interesting "How to develop a service quality measurement model with sentiment analysis and text mining?" but the authors have not introduced a quality measurement model; or Key Performance Indicators. The authors were more focused on the analysis of the content of tweets. We do not have a conclusion about the service quality measurement, quality dimensions, or customer satisfaction. Authors need to focus on their research question or change it in order to be in line with the findings and presented study. Also, the basic tools for analysis were used. We do not have much data about samples or sources.... In addition, making the connection with ISO 9001:2015 or even of concept of QMS 4.0 could be interesting.

In conclusion, the topic is interesting but the manuscript should be improved in the presentation of the research or the research question should be corrected. 

Comments on the Quality of English Language

proof reading

Author Response

Dear reviewer, thank you for taking the time to review our work. We think that the points you have specified for the study will make our article better, so we have accepted your legitimacy in the arms you have stated and tried to eliminate our deficiencies. In the study, we completed our literature review to address our deficiencies in ISO 9001:2015 and made the necessary corrections to bring a new perspective to the study. 

In the model we propose, after the tweets about an institution are obtained, the keywords expressing the service quality of the institution are extracted from these tweets using the text mining method. Frequencies, co-occurrence of words, and their relationships with each other are considered when removing these words. The keywords obtained are searched in the tweets sent about the organisation. Sentiment analysis is applied to the results. This way, the comments about the institution's services are analysed. For example, if a user tweets the expression "buses are very uncomfortable, I'm tired of travelling standing", according to the proposed model, it will indicate that this post is related to public transport services and the sentiment analysis is negative. This method was applied to 463886 tweets and determined that 156004 tweets were related to service quality. With the text mining method, it was revealed that the municipality has ten service dimensions. The proposed model can be used in a municipality and different sectors. Since the keywords in the proposed model are extracted directly from the content created by the users, the results are very original. For example, the word "mole" has been identified as related to the municipality's road maintenance and repair services. Users often used the phrase "molehill" for broken roads. Some words are often misspelt because the posts are sent from the phone. The difference between the proposed model and similar studies is the use of text mining, sentiment analysis, and the data size. In addition, the proposed model is not only theoretical but also applied. If you want to see how the system works, log in to erhan.sistem.io with mdpi@mdpi.com e-mail address and 12345678 password. 

The conceptual framework has been developed per your and other reviewers' views. The title of the article has been revised. In addition, the necessary programme outcomes have been added to the appendices to make the method more understandable. Service was purchased for proofreading the article. If there are uncorrected parts of the English, we can review it again.

Best regards

 

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Unfortunately, the corrections to the text of the article, which were made by the authors after the implementation of the first stage of its review, did not fundamentally change the essence of the results presented by the authors. Still, the authors of the article believe that the purpose of the scientific article should be to form an answer to the question: "How to develop a model for measuring the quality of service using sentiment analysis and text mining?" As I stated in the previous review, unfortunately, the authors have not mastered the theoretical foundations of the implementation of scientific research, therefore neither the purpose of this article nor the research tasks formulated by the authors correlate with any of the two main components of the implementation of scientific research, namely: system analysis (object, processes or systems, etc.) and the synthesis of some new result based on a previously implemented analysis. This result determines the scientific novelty of the research. Unfortunately, such scientific novelty, in my opinion, is not present in this article. There is a huge amount of text in which the authors of the article describe a set of approaches to determining the quality of services within the framework of the theory of total management, as well as evaluate the advantages and disadvantages of a number of software products intended for practical use by firms providing certain services. Perhaps this article can be considered as a kind of literary review, but even in this case, the level of implementation of such a literary review does not correspond to the level of rating in the international citation databases (SCOPUS, WoS, etc.) of the journal to which this article is submitted. The authors devoted an entire section No. 3 in the text of this article, entitled "Methodologists". But in this part of the article we are not talking at all about the application of one of the components of the classical methodology of scientific research. The authors have not been able to name any method of implementing scientific research that they would apply. Maybe the authors of the article are very competent specialists in the field of processing information contained in databases. Maybe they actually use CAQDAS, MAXQDA, Quirkos, Transana or some other software in practice. But, everything the authors write about in their article is not directly related to science.

Reviewer 2 Report

Comments and Suggestions for Authors

Positioning 

The revised article, although attempting to deepen the literature on service quality, still suffers from a lack of clarity in its positioning and argumentative structure. The initial ambiguity between critiquing traditional methods of measuring service quality and proposing a digital alternative remains a central concern.

 

Firstly, the literature review still appears to oscillate between theoretical discussions and specific elements related to data mining and the use of Twitter (now X) as a data source. This duality creates a narrative tension that distracts the reader from the article's central thesis. The authors seem to waver between critiquing existing models like SERVQUAL/SERVPERF and exploring methodological aspects of text mining and sentiment analysis. This wavering underscores a lack of thematic focus and dilutes the impact of the argument.

 

Furthermore, the transition to methodological discussions on text mining and sentiment analysis is abrupt and seems disjointed. These sections, while methodologically sound, appear disconnected from the theoretical discussion on service quality measurement. This disconnection creates a fragmented narrative, where theoretical and methodological elements coexist without harmonious integration.

 

In conclusion, despite the revisions made, the article still lacks a clear trajectory and cohesive narrative structuring. To strengthen the article, the authors should either focus on a thorough critique of current methods of measuring public service quality and how digital approaches can address these gaps or focus on a methodological exploration of text mining and sentiment analysis, using municipal service quality as a case study. This clarification would help to align theory and methodology in a more structured and focused narrative.

 

Methodology

 

A persistent and significant limitation of the study, as observed in the revised version, is the continued absence of robust quality indicators for the results. The use of a confusion matrix to demonstrate the accuracy of sentiment analysis, while informative, does not suffice to establish the reliability and validity of the overall findings. In scientific research, particularly when proposing a new methodology as a viable alternative to established traditional approaches, it is imperative to provide a comprehensive set of quality indicators.

 

These indicators are not limited to mere accuracy measurements but should extend to include measures of reliability, validity, and error rates, among other relevant metrics. The inclusion of such indicators is not just a matter of rigor but is essential to underpin the scientific credibility of the approach. It lends weight to the claims made and allows for a more critical and informed evaluation by the scientific community.

 

The current iteration of the paper, regrettably, fails to address this crucial aspect. Without these indicators, the reader is left to accept the authors' conclusions without the necessary empirical grounding. This omission significantly undermines the credibility of the research and hinders its potential acceptance as a convincing alternative to traditional methods of service quality measurement.

 

In conclusion, for the paper to effectively contribute to the field and be considered a legitimate scientific alternative to existing methodologies, it is essential that the authors address this gap. Incorporating comprehensive quality indicators will not only bolster the empirical strength of the study but will also enhance its acceptance and application within the scientific community.

Reviewer 4 Report

Comments and Suggestions for Authors

Authors addressed the main questions and provide answers.

Comments on the Quality of English Language

Proof reading.