Submit to this Journal Review for this Journal Propose a Special Issue

Article Menu

Share Help Cite Discuss in SciProfiles

Open AccessArticle

Peer-Review Record

Evaluating a User-Centered Environment-Friendly Mobile Phone App for Tourists and Residents Using Facial-Recognition Software

Sustainability 2023, 15(20), 14689; https://doi.org/10.3390/su152014689

by Lama Sameer Khoshaim¹

, Serhat Yüksel^2,3,*

and Hasan Dinçer²

Reviewer 1: Anonymous

Reviewer 2:

Saad Al-Saad

Reviewer 3:

Seul Ki Lee

Sustainability 2023, 15(20), 14689; https://doi.org/10.3390/su152014689

Submission received: 1 September 2023 / Revised: 8 October 2023 / Accepted: 8 October 2023 / Published: 10 October 2023

Round 1

Reviewer 1 Report (Previous Reviewer 1)

Thank you for the opportunity to review your revised manuscript. The paper is I think much clearer.

Here are a few final suggestions. Thre are no line numbers on the revised manuscript, so I will try to locate these suggestions as best I can:

Abstract

P1 Line 6: Suggest something like 'analysis' in place of 'situation'.

Introduction

P2 Line 26: Ref 4 is meant to be on 'the principles of sustainable tourism', but Ref 4 in the Ref list is at best only tangentially relevant. I'd suggest using the following instead:

Mason P (2021) Tourism Impacts, Planning and Management. Fourth Edn. Routledge: Abingdon, UK

P3 Line 33: Instead of 'figure out', I'd suggest something like ' examine' or 'investigate'.

P3 Line 36: Instead of 'allows', maybe use something like 'enables'.

P 5 Line 16: Maybe add '(described below)' after 'with golden cuts' to reassure the reader further explanation of this very technical term is coming.

Finally - good idea to move some of the more technical results and analysis into an appendix.

Fine - see a few suggestions, above.

Author Response

Response to Reviewer 1

Reviewer Comment 1: Abstract

P1 Line 6: Suggest something like 'analysis' in place of 'situation'.

Author Answer 1: Based on the comments of the reviewer, we have changed the expression in the abstract part as follows.

“With the help of this analysis, the appropriate strategies can be implemented without incurring high costs.”

Reviewer Comment 2: Introduction

P2 Line 26: Ref 4 is meant to be on 'the principles of sustainable tourism', but Ref 4 in the Ref list is at best only tangentially relevant. I'd suggest using the following instead:

Mason P (2021) Tourism Impacts, Planning and Management. Fourth Edn. Routledge: Abingdon, UK

P3 Line 33: Instead of 'figure out', I'd suggest something like ' examine' or 'investigate'.

P3 Line 36: Instead of 'allows', maybe use something like 'enables'.

P 5 Line 16: Maybe add '(described below)' after 'with golden cuts' to reassure the reader further explanation of this very technical term is coming.

Finally - good idea to move some of the more technical results and analysis into an appendix.

Author Answer 2: According to the comments of the reviewer, we updated the manuscript by making following improvements. The details of these improvements are given below.

Ref 4 is changed as follows by considering the suggestions of the reviewer.
- Mason, P. (2020). Tourism impacts, planning and management. Routledge.
The term of “figure out” is changed by “examine” in all manuscript.
The term of “allow” is changed by “enable” in all manuscript.
We added the expression of “described below” after “with golden cuts” in P5 line 16.
Based on the directions of the reviewer, we have put only 3 tables in the main manuscript. On the other side, 5 technical tables are transferred to the appendix.

Author Response File: Author Response.docx

Reviewer 2 Report (New Reviewer)

Dear Authors,

This research is conducted and written well. However, the following comments might improve the quality of your work:

Abstract

Abstract is copy-pasting from a paragraph in the introduction (lines 113-119).

Line 16-17, the authors stated that “With the help of this situation, the appropriate strategies can be implemented without incurring high costs.” What are the appropriate strategies? I did not find them in the text below. Appropriate word is general. You should specify to help other researchers build their future research on the results of this research.

Introduction

The first two paragraphs of the introduction are about the importance of tourism. They are general and not related to the topic of the research (environment-friendly mobile phone app for tourists and residents). Deleting them would not impede the quality of the introduction.

Line 55, the authors stated “these applications”. Which applications? You did not discuss anything about applications above. This impedes the credibility of the research.

The third, fourth and fifth paragraphs of the introduction have no insights. They are not sequenced well. Please rewrite them with harmony.

Lines 119-122, no need for this paragraph. No need to inform the readers that “the following section reviews the relevant literature for this study”. This is something that goes without saying.

Literature review

Please provide an overview of current knowledge, identify relevant theories, and gaps in the existing literature. Overall, literature review suffers from major flaws, please take my aforementioned comments into consideration when you revise and resubmit manuscript.

Methodology

The author/s started the methodology with a paragraph about literature review. This paragraph should be moved and integrated in the literature review section highlighting the contribution of the used methods in collecting and analyzing the data. Authors should understand the difference between methodology and literature review.

The author/s did not explain the type of the methodology they employed. Please explain it at the beginning of the methodology section.

The author/s did not explain how they collect the data? Please explain it.

Lines 232-236, this paragraph is vague. You mentioned “refer to the details of this situation”. What SITUATION? In addition, you mentioned “events”. What do you mean by events?

Results Analysis

Line 296, the title “Analysis Results” should be changed to “Results Analysis”.

Author/s were selective in explaining the data of table 2. You explained some and left others in the paragraph following table 2. Please elaborate on this.

Lines 324-325, the author/s stated “In this process, a team of experts consisting of six different people is formed. Two of these people are users. These people have master’s degrees”.

Discussion and Conclusion

Both the discussion and conclusion sections combined together. Please allocate a discussion section, in which identify relevant researches, and discuss the results of your research with the results of other relevant research.

The conclusion was a recap of the results section with no meaningful insights. You need to elaborate on this part.

Does this research have theoretical and practical implications? If yes, please highlight them. Please separate the theoretical and practical implications in new sections and discuss them thoroughly.

Limitations and future research are very simple. Please separate them in new sections and discuss them thoroughly.

References

Please add the DOIs of the entire references to be more accessible.

Author Response

Response to Reviewer 2

Reviewer Comment 1: Abstract

Abstract is copy-pasting from a paragraph in the introduction (lines 113-119).

Author Answer 1: Based on the comments of the reviewer, we have improved the manuscript by rewriting the abstract part. In this context, we have changed all expressions so that this bar becomes different from the paragraph in the introduction. Additionally, we have also removed the expressions stated by the reviewer. The final form of the abstract part is demonstrated as follows.

“Abstract

This study aims to examine a user-centered environment-friendly mobile phone app for tourists and residents in Saudi Arabia. To achieve this objective, a new fuzzy decision-making model is constructed. In this scope, by considering detailed literature examination, eight criteria are identified. Facial expression-based quantum spherical DEMATEL with golden cuts are taken into consideration to examine the possible impact and directions among these criteria. The main contribution of this study is that priority factors in the development of the environment-friendly tourism mobile application can be identified. Moreover, facial expressions of the decision makers are taken into consideration in the fuzzy decision-making analysis. This situation helps to achieve more appropriate results. The findings demonstrate that user-centeredness is the most significant environment-friendly mobile application performance criterion of the tourism industry. Additionally, functionality is also another important determinant with respect to this situation. It is of great importance to ensure customer satisfaction for the mobile applications used in the development of tourism to be successful. Thus, user feedback should be taken into consideration for these applications to work more effectively. Furthermore, tourism applications should provide users with fast and accurate information. In this context, mobile applications should offer customized offers to users to increase customer satisfaction.”

Reviewer Comment 2: Introduction

Line 55, the authors stated “these applications”. Which applications? You did not discuss anything about applications above. This impedes the credibility of the research.

The third, fourth and fifth paragraphs of the introduction have no insights. They are not sequenced well. Please rewrite them with harmony.

Lines 119-122, no need for this paragraph. No need to inform the readers that “the following section reviews the relevant literature for this study”. This is something that goes without saying.

Author Answer 2: According to the comments of the reviewer, we have improved the manuscript by rewriting the introduction part. The main improvements are demonstrated below.

The first two paragraphs in the introduction part are removed by considering the suggestions of the reviewer.
The expressions of “these applications” are removed from the manuscript. Additionally, more definitions are given related to the mobile applications.
The third, fourth and fifth paragraphs of the introduction are rewritten. In this context, the following issues are taken into consideration.
- Paragraph: The importance of mobile applications in tourism industry.
- Paragraph: The significance of environment-friendly issues to generate mobile applications in tourism industry.
- Paragraph: The main determinants that have an impact on the effectiveness of the sustainable tourism application.
- Paragraph: The purpose of this study.
- Paragraph: The details of the proposed model and main contributions.
The final paragraph of the introduction part is removed from this section based on the comments of the reviewer.

The final version of the introduction paragraph is given below.

“1. Introduction

Mobile applications play an important role in the tourism sector and contribute to the development of tourism. They offer tourists a wide range of information about the region [1]. Tourists will be able to have information on many important issues about places to visit, route suggestions and restaurants on these platforms [2]. This will significantly contribute to the development of tourism as it will facilitate the travel of tourists significantly [3]. Mobile applications also facilitate the transportation and accommodation of tourists [4]. Thanks to them, tourists can more effectively manage their hotel reservations and flight ticket purchasing processes [5]. On the other hand, it will be possible for visitors to access useful information about local public transport facilities and services. Mobile applications also enable tourists to have information about places to visit. Hence, tourists can access news and views about the places [6]. In addition, some mobile applications enable users to access real-time information about these places.

Environment-friendly mobile applications mean that while trying to develop the tourism sector, it is also aimed to minimize the negative environmental impact. Because tourism sector may have negative impacts on environmental issues, the popularity of these environment-friendly applications increases [7]. Within this framework, these apps support eco-friendly activities to the customers. In this framework, guidance is provided to the users of the application to increase the level of awareness regarding environmental issues [8]. Within this context, data management should be provided more effectively. This helps to optimize energy consumption. Environment-friendly mobile applications also support the economic sustainability of tourism businesses. On the other hand, environment-friendly mobile apps help preserve and support local cultures and communities [9]. Owing to the mobile applications, tourists can be informed about the importance of environmental issues in tourism activities. This can include environmentally friendly activities, protecting natural areas, supporting local cultures and communities. Moreover, mobile applications can encourage users to act environmentally responsible. In this context, application users can purchase digital tickets to reduce paper usage [10]. In addition, these apps can provide users with guidance including recycling points and waste management information.

Therefore, it is necessary to address key issues around sustainability. The sustainability of the mobile application for the tourism sector means its ability to sustain its long-term success and impact in terms of environmental, economic, and social aspects. First, it is important that these practices are compatible with country practices. In addition, this application needs to meet the expectations of the users. Otherwise, they will not be preferred, and this will reduce the usability of the application [11]. On the other hand, this app should be functional. In this framework, it should offer many different feelings to the users. For this application to be successful, the technical infrastructure must also be sufficient. On the other hand, it is necessary to ensure the financial profitability of this application. Otherwise, the long-term sustainability of the unprofitable application is not very possible. As can be seen, improvements need to be made to create a sustainable tourism mobile application. However, the costs will increase while making new improvements [12]. Therefore, it may not be financially possible to take many actions. It is necessary to make a priority analysis among these variables.

Accordingly, the purpose of this study is to evaluate a user-centered environment-friendly mobile phone app for tourists and residents in Saudi Arabia. The main reason of selecting this country is that COVID-19 pandemic has also had a huge impact on the country's tourism industry. Due to this pandemic, the country's citizens have turned to local tourism rather than international travel. Within this framework, tourists visit famous landmarks throughout the country. On the other hand, tourism activities in this process are carried out through word-of-mouth communication or advertisements made through social media. This situation is not sufficient for the effective development of tourism in the country. In other words, tourists cannot access enough information to make informed choices about many local tourism facilities and restaurants. This situation creates a significant barrier to the development of tourism activities.

Within this framework, a novel fuzzy decision-making model is constructed to achieve this objective. In this context, eight criteria are defined based on a detailed literature examination. These criteria are analyzed with facial expression-based quantum spherical DEMATEL with golden cuts to examine the possible impact and directions among them. The main contribution of this study is that priority factors in the development of the tourism mobile application will be determined. On the other hand, the proposed model has also significant contributions to the literature. Firstly, by considering facial expressions of the decision makers, the uncertainties in the analysis process can be minimized. Additionally, while using DEMATEL technique to weight the determinants, the causal directions of these items can be taken into consideration. This condition has a powerful contribution to achieve more effective findings.”

Reviewer Comment 3: Literature review

Author Answer 3: According to the directions of the reviewer, we have improved the manuscript by rewriting the literature review part. Within this context, we have divided this part into two sub categories that are literature on determinants of environment-friendly mobile applications and literature review results. In the first subsection, we have mainly given the results of the similar studies in the literature. On the other side, in the second sub section, we have focused on the main findings of the literature review. In this context, the main missing part of the literature is also underlined. Additionally, we have also given details regarding how we fill this missing part in the literature. The final form of this section is indicated below.

“2. Literature Review

This section consists of two different parts that are literature on determinants of environment-friendly mobile applications and literature review results.

2.1. Literature on Determinants of Environment-Friendly Mobile Applications

It is of great importance to comply with the legal regulations for the environment-friendly mobile applications used in the development of tourism to be successful. The data pool of these mobile applications contains the personal information of the users [13]. Therefore, the protection of personal data on this platform is of vital importance. Otherwise, this data may be stolen by third parties. Carvalho et al. [14] identified that this situation undermines the confidence of users in the application. On the other hand, there may be very high financial penalties for not complying with these legal regulations. Thong et al. [15] defined that tourism mobile applications must comply with relevant consumer protection laws to protect the rights of consumers. In this context, it is necessary to inform consumers accurately, especially about prices [16]. If this cannot be achieved, there is a risk that the application owner company may encounter some legal problems. Moreover, Thananchana et al. [17] concluded that mobile applications must comply with the competition rules. In this context, the details of competition law in the country of use should be thoroughly understood. Furthermore, Kotsyuba et al. [18] mentioned that tourism apps must act in accordance with legal and ethical standards. Encouraging or supporting illegal activities can lead to legal problems.

For the mobile applications used in the development of tourism to be successful, it is of great importance to ensure customer satisfaction. In this context, mobile applications should be designed to be easy to use [19]. Users may encounter some glitches while transacting with the application. In this context, Mendes et al. [20] identified that some measures should be taken to solve customers' problems quickly and accurately. Therefore, user feedback needs to be taken into consideration [21]. These feedbacks are guiding on what kind of problems there are in the application. Furthermore, according to Yu et al. [22], to increase customer satisfaction, mobile applications must be able to provide personalized experiences to users. In this context, this application company is required to conduct a comprehensive data analysis of customers. Ortega e al. [23] defined that based on the historical data of the customers, special offers should be made to these people. This ensures that customer expectations can be met to a significant extent. Additionally, Oliveira et al. [24] stated that more effective communication with customers is required. It is necessary to provide an easy and effective communication channel where customers can convey their complaints.

Technical infrastructure plays a crucial role for the effectiveness of the environment-friendly mobile applications used in the development of tourism. One of the biggest expectations of users from environment-friendly mobile applications is fast and uninterrupted performance [25]. In this context, Kim et al. [26] mentioned that the application should be opened easily and the steps in the application should be performed quickly. The fact that the transactions cannot be performed quickly, and the application freezes constantly reduces the satisfaction of the customers. On the other hand, Botilias et al. [27] defined that tourism applications should be able to work on different mobile platforms. Otherwise, some operator users will not be able to benefit from this application. Environment-friendly tourism applications should include the necessary security measures to ensure the security and data protection of users [28]. Users share their credit card information while purchasing tickets on these mobile applications. Therefore, it should be ensured that the risks in this process are minimized by taking the necessary security measures [29]. Additionally, according to Salido et al. [30], it is important to constantly update and support the technical infrastructure. It may be necessary to add new services to environment-friendly tourism applications. In addition, some updates are required to fix the encountered system errors. In this way, it is possible to increase customer satisfaction more easily.

Mobile applications used in the development of tourism should be functional. The success of a environment-friendly tourism application depends on meeting the needs of the users [31]. With this application, users can perform different operations such as travel planning and ticket purchase. Alauddin et al. [32] determined that the biggest expectation of the users in this process is that they can perform these transactions easily and quickly. The fact that users can use the application without difficulty and unnecessary complexity is one of the key elements of success. According to Bessouat and Haller [33], for users from different countries to benefit from the environment-friendly mobile tourism application, it should be possible to operate in different languages. Otherwise, the application will not be able to serve some segments, and this will hinder the development of tourism. On the other hand, Hassan and Avi [34] stated that tourism applications should be able to integrate with other related services and platforms. The application's combination of various services makes travel planning more efficient for users. Manggopa et al. [35] mentioned that it is important to consider user feedback and reviews to ensure functionality. In this context, an interface should be developed where user experiences can be shared [36]. In this way, it is possible to develop the application by making continuous improvements [37].

2.2. Literature Review Results

The main results of the literature review are indicated below. Tourism industry plays a key role for the economic and social development of the countries. For the effectiveness of the tourism industry, necessary importance should be given to the marketing and promotional activities. A mobile application can have a positive contribution for the improvement of the tourism industry. Necessary actions should be taken to increase effectiveness of the environment-friendly mobile applications related to the tourism industry. There are lots of indicators that affect the performance of environment-friendly mobile tourism applications. However, each action has an increasing impact on the costs. Therefore, it may not be financially possible to make lots of improvements at the same time. Thus, it is necessary to make a priority analysis among these variables and to make improvements to the more important issues in the first place.

The results of the literature review indicate that the main missing part in the literature is that there are limited studies in the literature that focused on the indicators of the effectiveness of the environment-friendly mobile applications. While considering these results of literature examination, this study aims to examine key issues of environment-friendly mobile application performance of the tourism industry in Saudi Arabia. By making this evaluation, it is aimed to fill this missing part in the literature. Within this context, eight factors are evaluated by facial expression-based quantum spherical DEMATEL with golden cuts described below to understand the possible impact and directions among them.”

Reviewer Comment 4: Methodology

The author/s did not explain the type of the methodology they employed. Please explain it at the beginning of the methodology section.

The author/s did not explain how they collect the data? Please explain it.

Lines 232-236, this paragraph is vague. You mentioned “refer to the details of this situation”. What SITUATION? In addition, you mentioned “events”. What do you mean by events?

Author Answer 4: Based on the suggestions of the reviewer, we have made some improvements to the methodology part. The details of these improvements are indicated below.

- We have removed the first expressions related to the methodology with a paragraph about literature review.

- We have given a small explanation about the proposed model in the beginning of the third section based on the suggestion of the reviewer. The final version of this expression is given below.

“A novel fuzzy decision-making model is proposed in this study to identify key determinants of the effectiveness of the environment-friendly mobile application for tourism industry. This proposed model has mainly two different parts. Firstly, the criteria are selected based on the literature review results. After that, the weights of these items are computed by using DEMATEL technique with facial expression-based quantum spherical fuzzy sets. The methods considered in the proposed model are explained in the following subtitles. All equations are given in the appendix part of the manuscript.”

- Data collection process is detailed in the first paragraph of the fourth section. The final version of this expression is given below.

- We have removed the expression of “refer to the details of this situation”. Additionally, we have given more details about the “events”. The final version of this paragraph is given below.

“Quantum theory (Q) considers different probabilities. Because of this advantage, in this proposed model, this theory is integrated with the fuzzy decision-making logic [40]. Equations (1)-(3) give details about the quantum theory. In these equations, represents a collection of events while u gives information about the events [41]. In this process, events give information about the elements of the function. In addition to them, , explains amplitude-based result for the probability and indicates phase angle. Spherical fuzzy sets () provides a comprehensive representation of uncertainty in decision-making problems. They consider membership, non-membership, and hesitancy degrees (, , ) in the analysis process [42]. They are detailed in Equations (4) and (5). The integration of these sets with quantum theory is detailed in Equations (6)-(8). In this proposed model, the degrees are computed with golden ratio (G) where b and a refer to the high and low values [43,44]. These issues are detailed in Equations (9)-(17). While calculating the degrees of Spherical fuzzy sets with golden ratio, it can be possible to minimize the uncertainty in the analysis process. Thus, more effective and reliable solutions can be reached.”

Reviewer Comment 5: Results Analysis

Line 296, the title “Analysis Results” should be changed to “Results Analysis”.

Author/s were selective in explaining the data of table 2. You explained some and left others in the paragraph following table 2. Please elaborate on this.

Lines 324-325, the author/s stated “In this process, a team of experts consisting of six different people is formed. Two of these people are users. These people have master’s degrees”.

Discussion and Conclusion

The conclusion was a recap of the results section with no meaningful insights. You need to elaborate on this part.

Does this research have theoretical and practical implications? If yes, please highlight them. Please separate the theoretical and practical implications in new sections and discuss them thoroughly.

Limitations and future research are very simple. Please separate them in new sections and discuss them thoroughly.

Author Answer 5: Based on the comments of the reviewer, we have made mainly following improvements.

- The title of the fourth section is changed as “The Results of the Analysis”.

- The paragraph after Table 2 is improved. The final form of this new paragraph is denoted as following.

“Relevance gives information that to what extent the mobile application aligns with the principles of environment-friendly tourism. Moreover, user-centeredness refers to the fact that how well the mobile application satisfies the needs and preferences of the local users. This situation attracts the attention of the users so that environment friendly tourism can be improved. Additionally, functionality shows how well the mobile application performs its intended functions and provide a satisfactory user experience. This situation has a positive contribution to the usage of environment friendly mobile applications. Furthermore, there should be technical feasibility to increase quality of mobile applications. With the help of this situation, interruptions in the applications can be minimized. Economic viability explains whether the benefits of the mobile application outweigh its costs in terms of both financial and environmental impacts. The applications should also have powerful social and environment impact to have greater performance. Finally, this application should reflect the authentic cultural and historical values of the places.”

- Discussion and conclusion section is divided into two different sub categories.

- The conclusion part is improved based on the comments of the reviewer.

- Limitations and future research directions are given in a more detailed manner.

- Both theoretical and practical implications are provided in a new section.

The final form of the final three sections is given below.

“5. Discussion

For the mobile applications used in the development of tourism to be successful, it is of great importance to ensure customer satisfaction. Therefore, user feedback should be taken into consideration for them to work more effectively. In this context, it is possible to detect the problems that arise regarding the application early. This enables for quick action to solve problems. Thus, it will be possible to increase customer satisfaction, and this increases the usability of mobile applications. Juliana et al. [47], Anand et al. [48] and Preziosi et al. [49] also concluded that customer expectations should be met effectively for the improvement of the tourism services. On the other hand, Camilleri and Filieri [50], Ioannou et al. [51] and Tiganis et al. [52] mentioned that tourism applications should provide users with fast and accurate information. In this context, care should be taken to ensure that the information in the application is up to date. Otherwise, customer satisfaction will decrease significantly. Moreover, to increase customer satisfaction, mobile applications should offer customized offers to users. According to Mainardes et al. [53] and Li et al. [54], this situation also contributes to increasing customer satisfaction. To increase customer satisfaction, mobile applications should consider user feedback and make continuous improvements. Kucukoglu [55] and Windasari et al. [56] also highlighted that the tourism companies should provide quick solutions for the problems of the customers.

Ensuring customer satisfaction is of great importance in improving the performance of mobile applications related to the tourism sector. This also contributes to the support of these mobile applications for environment-friendly tourism. Thanks to meeting customer expectations, these people will be able to use this application more. In this way, it is possible to reach more people to increase environmental awareness in salting activities. Tourism activities can harm the natural environment and ecosystems in various ways. For example, excessive use of water in tourism activities or pollution of water resources can cause significant damage to the environment [57]. To prevent this problem, it is important to increase the environmental awareness of the customers. As a result of customers giving importance to these issues, businesses may be more sensitive in this regard [58]. Thus, thanks to the mobile application, the tourism sector will be able to develop, and environmental problems will be minimized in this process [59]. Xu et al. [60] made a methodological review of sustainable tourism and emphasized that mobile applications can be helpful for this situation. Similarly, Salido et al. [61] focused on the sustainable tourism industry in Boracay Island and underlined the importance of mobile applications in this regard. Kimutai et al. [62] evaluated the sustainable tourism industry in the East Africa region. They demonstrated that effective mobile application is necessary to improve sustainability in tourism industry.

Theoretical and Practical Implications

Being user-friendly is of great importance in the success of environmentally friendly mobile applications. This allows users to use these applications regularly. The complexity of the interface makes these applications not preferred. In this context, a document should be created within the application to support users. Thanks to these documents, it is possible to direct users to certain steps. In addition to the mentioned point, unnecessary screens should not be created for applications to run quickly. Moreover, users should be offered the opportunity to give feedback. In this way, customer satisfaction can be achieved, and problems can be understood more easily. Thus, necessary steps can be taken to continuously improve the application. On the other hand, menus should be designed to be easy to use. Users should be able to easily access the topics they want to access via the menu.

Conclusion

In this study, it is aimed to evaluate the environment-friendly mobile application performance of the tourism industry in Saudi Arabia. Within this scope, eight criteria are defined based on a detailed literature examination. These factors are analyzed with facial expression-based quantum spherical DEMATEL with golden cuts to examine the possible impact and directions among them. It is identified that user-centeredness is the most significant mobile application performance criterion of the tourism industry. Moreover, functionality is also another important determinant with respect to this situation. Additionally, technical feasibility also plays an essential role for this condition. On the other side, environmental impact and economic viability have the lowest weights.

The main contribution of this study is that priority factors in the development of the tourism mobile application can be identified. With the help of this situation, the appropriate strategies can be implemented without incurring high costs. Similarly, the DEMATEL method provides some advantages. The indicators of the environment-friendly tourism mobile application can have an influence on each other. Hence, the causality relationship between these factors should be taken into consideration to perform a more accurate analysis. As a result, the use of the DEMATEL technique provides benefits in many ways. In addition, considering the facial expressions of the experts who make evaluations during the analysis process is beneficial in many ways. In this scope, this issue can be considered in the analysis process, owing to the consideration of the facial expressions of the experts. This situation enables for more effective analysis results to be achieved.

The main limitation of this study is that this evaluation is conducted for Saudi Arabia. However, the effectiveness of the tourism industry also plays a crucial role for other countries. For this purpose, developing or developed country groups can be evaluated in the following studies. The analysis results can help to present appropriate investment strategies for the improvement of the tourism industry. Also, this evaluation is performed only for the tourism industry. For the future research direction, different industries can be examined, such as banking or textile. Similarly, in future studies, the proposed methodology can be improved. While ranking different country groups based on the performance of tourism mobile applications, TOPSIS or VIKOR techniques can be considered. Furthermore, considering expert opinions in the analysis process can be accepted as another limitation. Hence, in the following studies, econometric models can be created based on numerical data.”

Reviewer Comment 6: References

Please add the DOIs of the entire references to be more accessible.

Author Answer 6: We have added the DOIs of the entire references.

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

This submission could be an interesting empirical study that may contribute to the literature by offering a new perspective, but at this point there are a number of issues that must be addressed in order for it to achieve merit for publication.

1. Please give detailed explanation of the role of environment-friendly mobile app in this study. The reviewer does not see a obvious link.

2. There is a general lack of discussion on the facial expression technology.

3. There is a general lack of explanation of mathematical notations. Also, do the authors really need the equations?

4. Please give detailed information about the sample and data.

5. Table 2 does not really seem connected to the previous section.

6. Discussion and conclusions are too trivial. Please strenghthen.

The quality of communication seems fair.

Author Response

Response to Reviewer 3

Reviewer Comment 1: Please give detailed explanation of the role of environment-friendly mobile app in this study. The reviewer does not see a obvious link.

Author Answer 1: Based on the comments of the reviewer, we have improved the paragraph related to the environment-friendly mobile app in the introduction part. Within this context, we have given more explanations related to this issue. The final form of this improved paragraph is indicated as below.

“Environment-friendly mobile applications mean that while trying to develop the tourism sector, it is also aimed to minimize the negative environmental impact. Because tourism sector may have negative impacts on environmental issues, the popularity of these environment-friendly applications increases. Within this framework, these apps support eco-friendly activities to the customers. In this framework, guidance is provided to the users of the application to increase the level of awareness regarding environmental issues. Within this context, data management in these applications should be provided more effectively. This helps to optimize energy consumption. Environment-friendly mobile applications also support the economic sustainability of tourism businesses. On the other hand, environment-friendly mobile apps help preserve and support local cultures and communities [9]. Owing to these applications, tourists can be informed about the importance of environmental issues in tourism activities. This can include environmentally friendly activities, protecting natural areas, supporting local cultures and communities. Moreover, mobile applications can encourage users to act environmentally responsible. In this context, application users can purchase digital tickets to reduce paper usage [10]. In addition, these apps can provide users with guidance including recycling points and waste management information.”

Reviewer Comment 2: There is a general lack of discussion on the facial expression technology.

Author Answer 2: According to the comments of the reviewer, we updated the manuscript by improving the expressions related to the facial expression technology. For this purpose, we have updated the subsection of 3.1. The final form of this part is indicated below.

“3.1. Decision-Making with Facial expressions

Decision making models help to understand more significant items among many different criteria. This situation provides an opportunity to focus on more important factors. With the help of this situation, it can be possible to make decisions for any subject more appropriately. However, to reach more appropriate solutions, these decision-making techniques are considered with fuzzy logic. This condition has a positive contribution to minimize uncertainty in the decision-making process. The performance of the decision-making methodology depends on some critical issues, such as the quality of the decision makers. Hence, the expert team should consist of the people who have the necessary sufficiency about the subject. However, these people may not be sure for the answers of some questions. In this process, facial expressions of these people can have to reach more appropriate solutions [38]. The Facial Action Coding System (FACS) considers the nonverbal behaviors of the people by using action units (AUs) [39]. In other words, AUs refer to a set of fundamental motions of the facial muscles. The coding of each facial expression involves identifying the specific AUs. Considering the facial expressions of the experts who make evaluations during the analysis process is beneficial in many ways. Experts may experience uneasiness when answering some questions. In this context, this issue can be considered in the analysis process, owing to the consideration of the facial expressions of the experts. Within this scope, online meetings are conducted with the decision makers while obtaining the evaluations. In this process, with respect to the facial expression technology, the emotions of the decision makers by making evaluations are taken into consideration. This enables for more effective analysis results to be achieved.”

Reviewer Comment 3: There is a general lack of explanation of mathematical notations. Also, do the authors really need the equations?

Author Answer 3: According to the comments of the reviewer, we transferred all equations to the appendix part.

Reviewer Comment 4: Please give detailed information about the sample and data.

Author Answer 4: Based on the directions of the reviewer, we have given more details about the sample and data set. For this purpose, we have improved related two paragraphs. The final version of these paragraphs is denoted below.

“Pair combinations give information about the combinations of different AUs together. Additionally, Table 1 demonstrates that experts use five different scales to answer the questions created by the criteria set. Moreover, five degrees are also used based on these scales in the analysis process. For this purpose, eight criteria are defined based on a detailed literature examination. They are detailed in Table 2.”

Reviewer Comment 5: Table 2 does not really seem connected to the previous section.

Author Answer 5: Based on the comments of the reviewer, we have improved the manuscript by giving more details related to Table 2. In the second section, we have evaluated similar studies in the literature. Based on these analysis results, we have identified 8 different criteria regarding the mobile application performance of the tourism industry. These defined criteria are explained in Table 2. This situation is detailed in a more effective manner in the paragraph before Table 2. The final version of this updated paragraph is shown below.

Reviewer Comment 6: Discussion and conclusions are too trivial. Please strenghthen.

Author Answer 6: According to the comments of the reviewer, we have improved the discussion and conclusions section. For this purpose, we have made more discussions by comparing the results of 5 new studies. The details of these new studies are shown below. These studies are selected form the journals indexed in SCI/SSCI in 2023.

Camilleri, M. A., & Filieri, R. (2023). Customer satisfaction and loyalty with online consumer reviews: Factors affecting revisit intentions. International Journal of Hospitality Management, 114, 103575.

Ioannou, I., Kassinis, G., & Papagiannakis, G. (2023). The impact of perceived greenwashing on customer satisfaction and the contingent role of capability reputation. Journal of Business Ethics, 185(2), 333-347.

Tiganis, A., Grigoroudis, E., & Chrysochou, P. (2023). Customer satisfaction in short food supply chains: A multiple criteria decision analysis approach. Food Quality and Preference, 104, 104750.

Mainardes, E. W., Coutinho, A. R. S., & Alves, H. M. B. (2023). The influence of the ethics of E-retailers on online customer experience and customer satisfaction. Journal of Retailing and Consumer Services, 70, 103171.

Li, Z., Zhao, X., & Ou, G. (2023). Understanding customer satisfaction in curated subscription services: Moderating roles of subscription time and information disclosure. Information & Management, 60(6), 103834.

The final version of this updated paragraph is given below.

“For the mobile applications used in the development of tourism to be successful, it is of great importance to ensure customer satisfaction. Therefore, user feedback should be taken into consideration for these applications to work more effectively. In this context, it is possible to detect the problems that arise regarding the application early. This enables for quick action to solve problems. Thus, it will be possible to increase customer satisfaction, and this increases the usability of mobile applications. Juliana et al. [47], Anand et al. [48] and Preziosi et al. [49] also concluded that customer expectations should be met effectively for the improvement of the tourism services. On the other hand, Camilleri and Filieri [50], Ioannou et al. [51] and Tiganis et al. [52] mentioned that tourism applications should provide users with fast and accurate information. In this context, care should be taken to ensure that the information in the application is up to date. Otherwise, customer satisfaction will decrease significantly. Moreover, to increase customer satisfaction, mobile applications should offer customized offers to users. According to Mainardes et al. [53] and Li et al. [54], this situation also contributes to increasing customer satisfaction. To increase customer satisfaction, mobile applications should consider user feedback and make continuous improvements. Kucukoglu [55] and Windasari et al. [56] also highlighted that the tourism companies should provide quick solutions for the problems of the customers.”

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report (New Reviewer)

Dear authors,

Good job. Significant improvement have been made. Although, literature review still needs more improvement. You ignored my comment about identifying the relevant theories. Please work on it.

Greetings

Author Response

Response to Reviewer 2

Reviewer Comment 1: Dear authors,

Good job. Significant improvement have been made. Although, literature review still needs more improvement. You ignored my comment about identifying the relevant theories. Please work on it.

Greetings

Author Answer 1: First of all, we would like to thank the reviewer for her/his valuable comments. Based on the comments of the reviewer, we have improved the manuscript by giving more details about the literature review. In this context, we have created a new sub section named as “2.2. Literature on Fuzzy Decision-Making Models in Mobile Applications”. In this new sub section, we have discussed the fuzzy decision-making models in the mobile applications. The details of this new sub section is denoted below.

“2.2. Literature on Fuzzy Decision-Making Models in Mobile Applications

The important points that affect the effectiveness of the mobile applications are evaluated by fuzzy decision-making models in many different studies. Arif et al. [38] focused on android mobile malware detection. In this process, important indicators are weighted by using fuzzy AHP methodology. Oztaysi et al. [39] created a fuzzy pricing model regarding the mobile advertisements. For this purpose, AHP approach is taken into consideration with Spherical fuzzy sets. Pandey et al. [40] prioritized different issues in wearable apps by using fuzzy DEMATEL methodology. With the help of this issue, the causal directions between these factors can be considered. Peng et al. [41] made an analysis for mobile edge caching scheme preferences by q‐Rung orthopair fuzzy decision‐making framework. On the other side, Madasi et al. [42] used n-cubic q-rung orthopair fuzzy sets to examine the use of mobile app in the education sector. Similarly, Tang et al. [43] makes an evaluation regarding the mobile medical applications with q-rung orthopair fuzzy sets. Anitha and Padma [44] examined the resources of mobile devices with a neuro-fuzzy hybrid framework.”

On the other side, we have also created a literature review summary table in the appendix part. With the help of this new table, it becomes much easier to follow the literature part for the readers. The final form of this new table is shown below.

Appendix – A: Literature Review Results

Table A1: Literature Review Results

Studies	Results
Carvalho et al. [14]	Legal regulations play a crucial role for the environment-friendly mobile applications used in the development of tourism to be successful
Thong et al. [15]
Thananchana et al. [17]
Kotsyuba et al. [18]
Mendes et al. [20]	It is of great importance to ensure customer satisfaction for the mobile applications used in the development of tourism to be successful
Yu et al. [22]
Ortega e al. [23]
Oliveira et al. [24]
Kim et al. [26]	Technical infrastructure plays a crucial role for the effectiveness of the environment-friendly mobile applications used in the development of tourism
Botilias et al. [27]
Salido et al. [30]
Alauddin et al. [32]	Mobile applications used in the development of tourism should be functional.
Bessouat and Haller [33]
Hassan and Avi [34]
Manggopa et al. [35]

Author Response File: Author Response.docx

Reviewer 3 Report (New Reviewer)

The reviewer would like to commend the efforts made by the authors in revision of the manuscript. However, there is still a serious lack of explanation on the data used. The authors have added some explanation - but this just raises more questions that it answers.

The quality of communication is fair.

Author Response

Response to Reviewer 3

Reviewer Comment 1: The reviewer would like to commend the efforts made by the authors in revision of the manuscript. However, there is still a serious lack of explanation on the data used. The authors have added some explanation - but this just raises more questions that it answers.

Author Answer 1: Based on the comments of the reviewer, we have improved the paragraphs related to the data collection process. Within this context, we have used Delphi technique while obtaining the evaluations from the experts. In this context, first, the problem must be clearly defined. Following this, an expert group on the subject is formed. In the third stage, experts share their opinions on these determined questions. These evaluations are then analyzed, and the results sent back to the experts. In this process, experts are advised to reconsider their opinions. These second results are taken into account to make an evaluation. Thus, the hesitation problem faced by experts can be minimized. The final form of these improved paragraphs is indicated as below.

With respect to the data collection, Delphi technique is taken into consideration. In this context, expert opinions are systematically provided. Thus, the hesitation problem faced by experts can be minimized. In this context, first, the problem must be clearly defined. Following this, an expert group on the subject is formed. In the third stage, experts share their opinions on these determined questions. These evaluations are then analyzed, and the results sent back to the experts. In this process, experts are advised to reconsider their opinions. These second results are taken into consideration to make an evaluation. In this framework, the values in Table 1 are taken into consideration.”

Author Response File: Author Response.docx

Round 3

Reviewer 3 Report (New Reviewer)

The reviewer appreciates the authors' efforts in responding, but would like to point out that the outstanding issue is still not resolved.

What was the composition of the Delphi panel, when and where was it done? What are the responses from the Delphi panel members? These would be the information the reviewer is looking for.

The quality of communication is fair.

Author Response

Response to Reviewer 3

Reviewer Comment 1: The reviewer appreciates the authors' efforts in responding, but would like to point out that the outstanding issue is still not resolved.

What was the composition of the Delphi panel, when and where was it done? What are the responses from the Delphi panel members? These would be the information the reviewer is looking for.

Author Answer 1: Based on the comments of the reviewer, we have improved the paragraphs related to the data collection process. For this purpose, wee have given more details about the Delphi panel and responses from the experts. In this context, an expert team is constructed with six different people. These people have necessary qualifications related to the tourism industry, sustainability, mobile applications, and environmental issues. Two of these people are users. These people have master’s degrees. Opinions of two different state officials are also taken into consideration. They work as the top managers in their departments. Finally, the other two are service providers. They have a minimum of 23 years of working experience. While making online meetings, the evaluations are obtained from the decision makers. With respect to the data collection, Delphi technique is taken into consideration. In this context, expert opinions are systematically provided. Thus, the hesitation problem faced by experts can be minimized. In this context, first, the problem must be clearly defined. Following this, an expert group on the subject is formed. In the third stage, experts share their opinions on these determined questions. These evaluations are then analyzed, and the results sent back to the experts. In this process, experts are advised to reconsider their opinions. These second results are taken into consideration to make an evaluation.

For this purpose, in this study, the meetings with the experts are conducted online. First, individual interviews are held with experts and their opinions on the questions are obtained. After this, all different opinions are shared with experts. Later, a group meeting is held with experts. In this meeting, experts discussed these issues together. Following this issue, a specific interview is conducted with the experts again and new expert evaluations are obtained. It is understood that during this process, some changes are made in the evaluations of each expert. Moreover, these new evaluations are shared with the experts again and a group meeting is held with the experts for the second time. In this meeting, experts are given the opportunity to discuss the issue together again. Moreover, experts have been interviewed for the third time and their final opinions are provided. During this process, it is determined that there have been some changes in the opinions of experts 1, 2, 3 and 5. On the other hand, there is no change in the opinions of experts 4 and 6. In this framework, the values in Table 1 are taken into consideration.

This situation is detailed in the first part of the fourth section. The details of these new updated paragraphs are indicated below.

“This study evaluates a user-centered environment-friendly mobile phone app for tourists and residents in Saudi Arabia. Firstly, the dataset is collected by using the observation of facial expression. For this purpose, an expert team is constructed with six different people. These people have necessary qualifications related to the tourism industry, sustainability, mobile applications, and environmental issues. Two of these people are users. These people have master’s degrees. Opinions of two different state officials are also taken into consideration. They work as the top managers in their departments. Finally, the other two are service providers. They have a minimum of 23 years of working experience. While making online meetings, the evaluations are obtained from the decision makers. With respect to the data collection, Delphi technique is taken into consideration. In this context, expert opinions are systematically provided. Thus, the hesitation problem faced by experts can be minimized. In this context, first, the problem must be clearly defined. Following this, an expert group on the subject is formed. In the third stage, experts share their opinions on these determined questions. These evaluations are then analyzed, and the results sent back to the experts. In this process, experts are advised to reconsider their opinions. These second results are taken into consideration to make an evaluation.

For this purpose, in this study, the meetings with the experts are conducted online. First, individual interviews are held with experts and their opinions on the questions are obtained. Details of these opinions are shared in Table A2. After this, all different opinions are shared with experts. Later, a group meeting is held with experts. In this meeting, experts discussed these issues together. Following this issue, a specific interview is conducted with the experts again and new expert evaluations are obtained. Details of these second-round evaluations are shown in Table A3. It is understood that during this process, some changes are made in the evaluations of each expert. Moreover, these new evaluations are shared with the experts again and a group meeting is held with the experts for the second time. In this meeting, experts are given the opportunity to discuss the issue together again. Moreover, experts have been interviewed for the third time and their final opinions are provided. These final opinions are presented in Table A4. During this process, it is determined that there have been some changes in the opinions of experts 1, 2, 3 and 5. On the other hand, there is no change in the opinions of experts 4 and 6. In this framework, the values in Table 1 are taken into consideration.”

On the other side, the details of the evaluations in different rounds are shown in the appendix of the manuscript. The details of these tables are shown below.

Table A2: The expert evaluations for the first round

Expert 1 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(25, 26)	(12, 25)	(27, 25)	(12, 25)	(12, 25)	(15, 5)	(25, 26)
UCTD	(14, 12)		(14, 2)	(12, 25)	(7, 6)	(12, 25)	(10, 14)	(27, 25)
FTLY	(7, 6)	(12, 25)		(12, 25)	(7, 6)	(14, 12)	(27, 25)	(25, 26)
TFBY	(7, 6)	(25, 26)	(5, 26)		(14, 2)	(7, 10)	(14, 2)	(10, 14)
EVBY	(14, 2)	(14, 2)	(14, 2)	(10, 14)		(15, 5)	(7, 6)	(15, 5)
SMPC	(14, 12)	(25, 26)	(27, 25)	(5, 26)	(27, 25)		(5, 26)	(7, 6)
EVMC	(12, 25)	(12, 25)	(15, 5)	(10, 14)	(7, 10)	(14, 2)		(25, 26)
CAUY	(7, 6)	(7, 6)	(14, 12)	(5, 26)	(15, 5)	(5, 26)	(12, 25)
Expert 2 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(10, 5)	(25, 26)	(27, 25)	(27, 25)	(27, 25)	(14, 12)	(5, 26)
UCTD	(27, 25)		(25, 26)	(27, 25)	(14, 12)	(25, 26)	(15, 2)	(27, 25)
FTLY	(14, 12)	(12, 25)		(27, 25)	(27, 25)	(7, 6)	(27, 25)	(12, 25)
TFBY	(14, 12)	(25, 26)	(5, 26)		(14, 2)	(10, 5)	(10, 5)	(7, 6)
EVBY	(10, 5)	(10, 5)	(10, 5)	(7, 10)		(14, 2)	(14, 12)	(15, 5)
SMPC	(14, 12)	(12, 25)	(5, 26)	(14, 12)	(5, 26)		(5, 26)	(12, 25)
EVMC	(25, 26)	(25, 26)	(7, 6)	(10, 14)	(10, 14)	(10, 5)		(12, 25)
CAUY	(7, 6)	(5, 26)	(5, 26)	(5, 26)	(7, 6)	(14, 2)	(27, 25)
Expert 3 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(7, 6)	(27, 25)	(1, 2)	(27, 25)	(27, 25)	(10, 14)	(27, 25)
UCTD	(15, 5)		(25, 26)	(25, 26)	(14, 12)	(27, 25)	(15, 5)	(25, 26)
FTLY	(27, 25)	(27, 25)		(12, 25)	(14, 12)	(27, 25)	(12, 25)	(14, 12)
TFBY	(27, 25)	(7, 6)	(25, 26)		(14, 12)	(10, 5)	(14, 12)	(10, 5)
EVBY	(15, 5)	(12, 25)	(10, 14)	(15, 5)		(7, 6)	(7, 6)	(10, 5)
SMPC	(15, 5)	(5, 26)	(14, 12)	(5, 26)	(7, 6)		(7, 6)	(7, 6)
EVMC	(27, 25)	(7, 6)	(15, 5)	(12, 25)	(15, 5)	(1, 2)		(15, 5)
CAUY	(10, 5)	(7, 6)	(15, 5)	(7, 6)	(15, 5)	(15, 5)	(15, 5)
Expert 4 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(6, 25)	(5, 26)	(27, 25)	(10, 14)	(7, 6)
UCTD	(7, 6)		(27, 25)	(25, 26)	(5, 26)	(27, 25)	(14, 12)	(25, 26)
FTLY	(5, 26)	(14, 12)		(12, 25)	(14, 12)	(27, 25)	(5, 26)	(7, 6)
TFBY	(12, 25)	(14, 12)	(25, 26)		(14, 12)	(15, 5)	(7, 6)	(10, 5)
EVBY	(7, 10)	(12, 25)	(10, 14)	(10, 5)		(1, 2)	(14, 12)	(7, 6)
SMPC	(10, 5)	(5, 26)	(7, 6)	(5, 26)	(7, 6)		(14, 12)	(5, 26)
EVMC	(5, 26)	(12, 25)	(14, 12)	(6, 25)	(10, 5)	(1, 2)		(10, 5)
CAUY	(27, 25)	(12, 25)	(14, 12)	(1, 2)	(14, 12)	(14, 12)	(1, 2)
Expert 5 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(10, 5)	(12, 25)	(7, 6)	(25, 26)	(5, 26)	(25, 26)	(5, 26)
UCTD	(15, 5)		(12, 25)	(25, 26)	(7, 6)	(25, 26)	(10, 5)	(12, 25)
FTLY	(15, 5)	(15, 5)		(25, 26)	(10, 1)	(27, 25)	(25, 26)	(25, 26)
TFBY	(27, 25)	(7, 6)	(25, 26)		(27, 25)	(27, 25)	(14, 12)	(14, 12)
EVBY	(7, 10)	(15, 5)	(10, 14)	(7, 6)		(7, 6)	(10, 14)	(10, 5)
SMPC	(14, 12)	(25, 26)	(12, 25)	(12, 25)	(7, 6)		(10, 5)	(7, 6)
EVMC	(10, 14)	(27, 25)	(10, 5)	(7, 6)	(15, 5)	(6, 12)		(7, 6)
CAUY	(7, 6)	(27, 25)	(10, 5)	(5, 26)	(7, 10)	(5, 26)	(7, 6)
Expert 6 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(7, 6)
UCTD	(14, 12)		(25, 26)	(7, 6)	(7, 6)	(7, 6)	(10, 1)	(25, 26)
FTLY	(14, 12)	(14, 12)		(12, 25)	(10, 1)	(27, 25)	(12, 25)	(25, 26)
TFBY	(27, 25)	(14, 12)	(12, 25)		(5, 26)	(7, 6)	(14, 12)	(14, 12)
EVBY	(14, 12)	(10, 1)	(15, 5)	(7, 6)		(7, 6)	(14, 12)	(14, 12)
SMPC	(14, 12)	(12, 25)	(7, 6)	(12, 25)	(7, 6)		(10, 1)	(5, 26)
EVMC	(10, 1)	(15, 5)	(10, 1)	(14, 12)	(10, 1)	(25, 26)		(14, 12)
CAUY	(7, 6)	(15, 5)	(15, 5)	(5, 26)	(10, 1)	(5, 26)	(7, 6)

Table A3: The expert evaluations for the second round

Expert 1 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(25, 26)	(12, 25)	(27, 25)	(12, 25)	(12, 25)	(15, 5)	(25, 26)
UCTD	(14, 12)		(14, 2)	(12, 25)	(7, 6)	(12, 25)	(10, 14)	(27, 25)
FTLY	(14, 12)	(12, 25)		(12, 25)	(7, 6)	(14, 12)	(27, 25)	(25, 26)
TFBY	(7, 6)	(25, 26)	(5, 26)		(14, 2)	(7, 10)	(14, 2)	(10, 14)
EVBY	(14, 2)	(14, 2)	(14, 2)	(10, 14)		(15, 5)	(7, 6)	(15, 5)
SMPC	(14, 12)	(25, 26)	(27, 25)	(14, 12)	(27, 25)		(5, 26)	(7, 6)
EVMC	(12, 25)	(12, 25)	(15, 5)	(10, 14)	(7, 10)	(14, 2)		(25, 26)
CAUY	(7, 6)	(5, 26)	(14, 12)	(5, 26)	(15, 5)	(14, 2)	(12, 25)
Expert 2 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(10, 5)	(25, 26)	(27, 25)	(12, 25)	(12, 25)	(14, 12)	(5, 26)
UCTD	(27, 25)		(25, 26)	(12, 25)	(14, 12)	(25, 26)	(15, 2)	(27, 25)
FTLY	(14, 12)	(12, 25)		(12, 25)	(27, 25)	(7, 6)	(27, 25)	(12, 25)
TFBY	(14, 12)	(25, 26)	(5, 26)		(14, 2)	(10, 5)	(10, 5)	(7, 6)
EVBY	(10, 5)	(10, 5)	(10, 5)	(7, 10)		(14, 2)	(14, 12)	(15, 5)
SMPC	(14, 12)	(12, 25)	(5, 26)	(14, 12)	(5, 26)		(5, 26)	(12, 25)
EVMC	(25, 26)	(25, 26)	(7, 6)	(10, 14)	(10, 14)	(10, 5)		(12, 25)
CAUY	(7, 6)	(5, 26)	(5, 26)	(5, 26)	(7, 6)	(14, 2)	(27, 25)
Expert 3 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(7, 6)	(27, 25)	(1, 2)	(27, 25)	(27, 25)	(10, 14)	(27, 25)
UCTD	(15, 5)		(25, 26)	(25, 26)	(14, 12)	(27, 25)	(15, 5)	(25, 26)
FTLY	(27, 25)	(27, 25)		(12, 25)	(14, 12)	(27, 25)	(12, 25)	(14, 12)
TFBY	(27, 25)	(7, 6)	(25, 26)		(14, 12)	(10, 5)	(14, 12)	(10, 5)
EVBY	(15, 5)	(12, 25)	(10, 14)	(15, 5)		(7, 6)	(7, 6)	(10, 5)
SMPC	(15, 5)	(5, 26)	(14, 12)	(5, 26)	(7, 6)		(7, 6)	(7, 6)
EVMC	(27, 25)	(12, 25)	(14, 12)	(12, 25)	(15, 5)	(1, 2)		(15, 5)
CAUY	(27, 25)	(12, 25)	(15, 5)	(7, 6)	(15, 5)	(15, 5)	(15, 5)
Expert 4 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(6, 25)	(5, 26)	(27, 25)	(10, 14)	(7, 6)
UCTD	(7, 6)		(25, 26)	(25, 26)	(5, 26)	(27, 25)	(14, 12)	(25, 26)
FTLY	(5, 26)	(14, 12)		(12, 25)	(14, 12)	(27, 25)	(5, 26)	(7, 6)
TFBY	(12, 25)	(14, 12)	(25, 26)		(14, 12)	(15, 5)	(14, 12)	(10, 5)
EVBY	(7, 10)	(10, 5)	(10, 14)	(10, 5)		(1, 2)	(14, 12)	(7, 6)
SMPC	(10, 5)	(5, 26)	(7, 6)	(5, 26)	(7, 6)		(14, 12)	(5, 26)
EVMC	(5, 26)	(12, 25)	(14, 12)	(6, 25)	(10, 5)	(1, 2)		(10, 5)
CAUY	(27, 25)	(12, 25)	(14, 12)	(1, 2)	(14, 12)	(14, 12)	(1, 2)
Expert 5 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(10, 5)	(27, 25)	(7, 6)	(25, 26)	(5, 26)	(25, 26)	(5, 26)
UCTD	(15, 5)		(12, 25)	(25, 26)	(7, 6)	(25, 26)	(10, 5)	(12, 25)
FTLY	(15, 5)	(15, 5)		(25, 26)	(10, 1)	(27, 25)	(25, 26)	(25, 26)
TFBY	(27, 25)	(7, 6)	(25, 26)		(27, 25)	(27, 25)	(14, 12)	(14, 12)
EVBY	(7, 10)	(15, 5)	(10, 14)	(7, 6)		(7, 6)	(10, 14)	(10, 5)
SMPC	(14, 12)	(25, 26)	(12, 25)	(12, 25)	(7, 6)		(10, 5)	(7, 6)
EVMC	(10, 14)	(27, 25)	(10, 5)	(7, 6)	(15, 5)	(6, 12)		(7, 6)
CAUY	(7, 6)	(27, 25)	(10, 5)	(5, 26)	(7, 10)	(5, 26)	(7, 6)
Expert 6 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(7, 6)
UCTD	(14, 12)		(25, 26)	(25, 26)	(7, 6)	(25, 26)	(10, 1)	(25, 26)
FTLY	(14, 12)	(14, 12)		(12, 25)	(10, 1)	(27, 25)	(12, 25)	(25, 26)
TFBY	(27, 25)	(14, 12)	(12, 25)		(5, 26)	(27, 25)	(14, 12)	(14, 12)
EVBY	(14, 12)	(10, 1)	(15, 5)	(7, 6)		(7, 6)	(14, 12)	(14, 12)
SMPC	(14, 12)	(12, 25)	(7, 6)	(12, 25)	(7, 6)		(10, 1)	(5, 26)
EVMC	(10, 1)	(27, 25)	(10, 1)	(14, 12)	(10, 1)	(25, 26)		(14, 12)
CAUY	(7, 6)	(27, 25)	(15, 5)	(5, 26)	(10, 1)	(5, 26)	(7, 6)

Table A4: The expert evaluations for the third round

Expert 1 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(25, 26)	(12, 25)	(27, 25)	(12, 25)	(12, 25)	(15, 5)	(25, 26)
UCTD	(14, 12)		(25, 26)	(12, 25)	(7, 6)	(12, 25)	(10, 14)	(27, 25)
FTLY	(14, 12)	(12, 25)		(12, 25)	(27, 25)	(14, 12)	(27, 25)	(25, 26)
TFBY	(7, 6)	(25, 26)	(5, 26)		(14, 2)	(7, 10)	(14, 2)	(10, 14)
EVBY	(14, 2)	(14, 2)	(14, 2)	(10, 14)		(15, 5)	(7, 6)	(15, 5)
SMPC	(14, 12)	(25, 26)	(27, 25)	(14, 12)	(27, 25)		(5, 26)	(12, 25)
EVMC	(12, 25)	(12, 25)	(15, 5)	(10, 14)	(7, 10)	(14, 2)		(25, 26)
CAUY	(7, 6)	(5, 26)	(14, 12)	(5, 26)	(15, 5)	(14, 2)	(12, 25)
Expert 2 (User)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(25, 26)	(27, 25)	(12, 25)	(12, 25)	(14, 12)	(5, 26)
UCTD	(27, 25)		(25, 26)	(12, 25)	(14, 12)	(25, 26)	(15, 2)	(27, 25)
FTLY	(14, 12)	(12, 25)		(12, 25)	(27, 25)	(7, 6)	(27, 25)	(12, 25)
TFBY	(14, 12)	(25, 26)	(5, 26)		(14, 2)	(10, 5)	(10, 5)	(7, 6)
EVBY	(10, 5)	(10, 5)	(10, 5)	(7, 10)		(14, 2)	(14, 12)	(15, 5)
SMPC	(14, 12)	(12, 25)	(5, 26)	(14, 12)	(5, 26)		(5, 26)	(12, 25)
EVMC	(25, 26)	(25, 26)	(7, 6)	(10, 14)	(10, 14)	(10, 5)		(12, 25)
CAUY	(7, 6)	(5, 26)	(5, 26)	(5, 26)	(7, 6)	(14, 2)	(27, 25)
Expert 3 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(1, 2)	(27, 25)	(27, 25)	(10, 14)	(27, 25)
UCTD	(15, 5)		(25, 26)	(25, 26)	(14, 12)	(27, 25)	(15, 5)	(25, 26)
FTLY	(27, 25)	(27, 25)		(12, 25)	(14, 12)	(27, 25)	(12, 25)	(14, 12)
TFBY	(27, 25)	(7, 6)	(25, 26)		(14, 12)	(10, 5)	(14, 12)	(10, 5)
EVBY	(7, 10)	(10, 5)	(10, 14)	(15, 5)		(7, 6)	(14, 12)	(10, 5)
SMPC	(15, 5)	(5, 26)	(14, 12)	(5, 26)	(7, 6)		(7, 6)	(7, 6)
EVMC	(27, 25)	(12, 25)	(14, 12)	(12, 25)	(15, 5)	(1, 2)		(15, 5)
CAUY	(27, 25)	(12, 25)	(15, 5)	(7, 6)	(15, 5)	(15, 5)	(15, 5)
Expert 4 (Government)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(6, 25)	(5, 26)	(27, 25)	(10, 14)	(7, 6)
UCTD	(7, 6)		(25, 26)	(25, 26)	(5, 26)	(27, 25)	(14, 12)	(25, 26)
FTLY	(5, 26)	(14, 12)		(12, 25)	(14, 12)	(27, 25)	(5, 26)	(7, 6)
TFBY	(12, 25)	(14, 12)	(25, 26)		(14, 12)	(15, 5)	(14, 12)	(10, 5)
EVBY	(7, 10)	(10, 5)	(10, 14)	(10, 5)		(1, 2)	(14, 12)	(7, 6)
SMPC	(10, 5)	(5, 26)	(7, 6)	(5, 26)	(7, 6)		(14, 12)	(5, 26)
EVMC	(5, 26)	(12, 25)	(14, 12)	(6, 25)	(10, 5)	(1, 2)		(10, 5)
CAUY	(27, 25)	(12, 25)	(14, 12)	(1, 2)	(14, 12)	(14, 12)	(1, 2)
Expert 5 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(7, 6)	(25, 26)	(5, 26)	(25, 26)	(5, 26)
UCTD	(15, 5)		(12, 25)	(25, 26)	(7, 6)	(25, 26)	(10, 5)	(12, 25)
FTLY	(15, 5)	(15, 5)		(25, 26)	(10, 1)	(27, 25)	(25, 26)	(25, 26)
TFBY	(27, 25)	(7, 6)	(25, 26)		(27, 25)	(27, 25)	(14, 12)	(14, 12)
EVBY	(7, 10)	(15, 5)	(10, 14)	(7, 6)		(7, 6)	(10, 14)	(10, 5)
SMPC	(14, 12)	(25, 26)	(7, 6)	(12, 25)	(7, 6)		(10, 5)	(5, 26)
EVMC	(10, 14)	(27, 25)	(10, 5)	(7, 6)	(15, 5)	(6, 12)		(7, 6)
CAUY	(7, 6)	(27, 25)	(10, 5)	(5, 26)	(7, 10)	(5, 26)	(7, 6)
Expert 6 (Service provider)
	RLVC	UCTD	FTLY	TFBY	EVBY	SMPC	EVMC	CAUY
RLVC		(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(27, 25)	(7, 6)
UCTD	(14, 12)		(25, 26)	(25, 26)	(7, 6)	(25, 26)	(10, 1)	(25, 26)
FTLY	(14, 12)	(14, 12)		(12, 25)	(10, 1)	(27, 25)	(12, 25)	(25, 26)
TFBY	(27, 25)	(14, 12)	(12, 25)		(5, 26)	(27, 25)	(14, 12)	(14, 12)
EVBY	(14, 12)	(10, 1)	(15, 5)	(7, 6)		(7, 6)	(14, 12)	(14, 12)
SMPC	(14, 12)	(12, 25)	(7, 6)	(12, 25)	(7, 6)		(10, 1)	(5, 26)
EVMC	(10, 1)	(27, 25)	(10, 1)	(14, 12)	(10, 1)	(25, 26)		(14, 12)
CAUY	(7, 6)	(27, 25)	(15, 5)	(5, 26)	(10, 1)	(5, 26)	(7, 6)

Author Response File: Author Response.docx

Round 4

Reviewer 3 Report (New Reviewer)

The reviewer would like to commend the efforts of the authors in addressing the issues raised in the previous round. At this point, the manuscript seems ready after a thorough check on the writing.

The quality of communication is fair, but can be improved.

Author Response

Response to Reviewer 3

Reviewer Comment 1: The reviewer would like to commend the efforts of the authors in addressing the issues raised in the previous round. At this point, the manuscript seems ready after a thorough check on the writing

Author Answer 1: According to the comments of the reviewer, we updated the manuscript by improving the level of English with the help of proofreading made by a native speaker. She corrected many different expressions in the study. Additionally, the lengths of some sentences reduced, and the study becomes more readable.

Author Response File: Author Response.docx

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Thank you for the opportunity to review your interesting manuscript.

There is much of value in the material presented, and here I offer some comments which may assist further development of the paper.

Title

The title as stated is misleading. It's a very long distance from the subject of the paper (examining the efficacy of a mobile phone app used to inform tourists) and the a sustainable and circular economy-oriented tourism investments in Saudi Arabia. The existing title may also not attract some of the readers you are presumably trying to interest - developers in tourism-related mobile phone apps.

Perhaps a title that puts the core of the study upfront would be more appropriate? Something like 'Evaluating a user-centred mobile phone app for tourists and residents using facial-recognition software'?

A title something like this would better reflect the aim of the study (Lines 18-19): 'The main contribution of this article is that priority factors in the development of the tourism mobile application can be identified'.

Overly colloquial language and typos

I suggest re-phrasing language in the manuscript that seems overly colloquial. Specifically:

Line 44: 'serious importance' - maybe something like '... tourism is a significant factor in the economic and social development of the country'

Line 45: '... take some measures...' - perhaps something like '... for these reasons, it may be a significant advantage for countries to develop their tourism industries'

Line 67: 'necessary information' - maybe something like '..it will be possible for visitors to access useful information about local public transport facilities and services'

Line 68: 'necessary information' - maybe something like '..also enable tourists to access information about attractive places to visit..'

Line 69: 'read the comments' - maybe something like '.. tourists can access news and views about...'

Line 70: 'allow users to receive live support...' - maybe something like 'some mobile applications enable users to access real-time information about these places'

Line 80: 'sufficient information' - maybe something like '...tourists cannot access enough information to make informed choices about...'

Line 99: 'some issues' - maybe something like ' it is necessary to address key issues around sustainability' As mentioned below under further explanations required, it would be well here to define sustainability; also, i cannot think of a definition of 'sustainability' that starts with '..it is important that these practices are compatible with country practices' (Lines 99 - 100), unless you are thinking of compatible with national cultural practices?

Line 128: 'consists of literature review' - perhaps something like 'reviews the relevant literature for this study and its methodology'

Line 196: 'literature evaluations' - do you mean ' 'literature review'?

Line 243: I suggest you use a lower case 'q' in quantum theory

Line 322: Suggest 'Discussion' rather than 'Discussions'

Further explanation required

In some places it is unclear what you are discussing. Specifically:

Line 55 ff: '... principles of sustainability' - what are these? What is your definition and what source are you using?

Line 87: 'sustainable mobile apps' - do you mean the apps are 'sustainable', or are you referring to apps on sustainable tourism?? If the later, I would provide a suitable to link it to the next part of that sentence '... help minimize environmental impact'

Line 99 - 100: As noted above, you need to provide a definition of 'sustainability' and clarify why it is import to your study objectives

Line 112ff: 'accordingly this study aims to examine the sustainability and circular economy-oriented mob app performance of the tourism industry in ...' - this is different to the study aim provided in the abstract (Lines 14 - 15) and later (Lines 287 ff) and later again 'the main contribution of this study is that priority factors in the development of the tourism mobile application can be identified' (Line 349). It also belies the points on sustainability noted above, and is different to the claims made for the value of the study (Lines 116 - 120). It is also a very big claim to make and one that the manuscript (as it stands) has not well defined or articulated)

More importantly, it is unclear what you are now studying - is it the 'performance' of 'sustainability and circular economy-oriented mob apps (Lines 112 - 113)? If yes, which apps? If no, what are you intending?

Lines 112 - 129: more broadly, this paragraph starts to describe the methods used. I'd separate this materials and present further down as part of 'Methodology' (starting Line 217).

You need to provide more detail on the tool you are using for analysis (DEMATEL - Line 115 ff). What is this and why is it useful and appropriate for this study? I note many of the sources you use are very technical in nature so you may need to reconsider what the readers of Sustainability might be familiar with and provide appropriate detail and clarification for such a broad readership (or consider a more technically-oriented place for publication)

You also need to provide more explanation and clarification on using facial recognition of 'experts' to evaluate the apps: what is your rationale 9supported by appropriate references)? For apps aimed at visitors and residents, the sample size of such target audiences is v small (see below, comments on Lines 297 ff)

Lines 132 ff: ' comply with the legal rules for the sustainable mobile apps used...' - not sure what you mean here. Do you mean somehow local legal requirements, regulation around 'sustainability', or do you mean legal rules around user privacy, etc.???

Lines 140 ff: ' ...especially about wages' - what do you mean? Are you speaking about minimum wage requirements for app developers???

Lines 214: 'a new model is proposed...' - as noted above, I think you need to have provided much more explanation before this point to provide readers with sufficient information to understand what you are saying.

Section 3.1 (Lines 220 ff) - you very rapidly move into a very technical exposition of your methodology and means of analysis. As noted above, you need to provide more preparation (especially for general readers, in which I would include academics and industry in tourism) for what develops as a very detailed and technical means of analysis. The paper at this point reads like a lot of intermediate (= between the objectives and the results) materials has been omitted.

Beyond this general statement, I would highlight the need for explanations of:

What and why fuzzy logic (Line 229)

Golden ratio (Line 228)

Why you need this level of detail in equation form (Lines 234 ff)

What Q is (Line 234), what As is (Line 241), what 'degrees' here are (Line 247)

Table 1: This needs much more explanation; e.g., what are 'pair combinations', 'scales' and 'degrees'? What is QSFNs?

Table 2 - again needs much more explanation; including referring back to earlier points about 'sustainability' and 'circular economy'- your ref (in this Table) is Ref 4, which is a conference paper from an IT conference!?! Much more relevant and useful sources are readily available. Other 'definitions' suffer from the same issue.

Line 297 ff: Here you describe the sample - both the nature of the participants and sample size need further explanation. This is a very small sample to draw such results (and to give such in-depth analysis) to.

Table 3: Same issues occur here, but in this case to the extent of making the table almost incomprehensible.

Table 4, 5, 6 & 7: Same issues with even more incomprehensibility.

Table 8: same issues but also - what is your system of weighting? Not just the values - what are you weighting and why

Section 5: Based on my comments, I can not see enough data and enough explication of the issues, means of analysis and explanation of results to justify the conclusions presented. These conclusions may well be correct, but the density of the technical parts of the paper, the undefined elements (such as 'sustainability'), the very small sample size, the nature of the participants, etc., do not support the findings .

I am happy to review a second draft if you would care to re-consider the manuscript in light of referees comments.

Generally fine - see specific suggestions above.

Author Response

Response to Reviewer 1

Reviewer Comment 1: Title

Author Answer 1: Based on the comments of the reviewer, we have changed the title of the manuscript. With its current form, it is believed that the title better reflects the aim of the study. The new version of the title is given below.

“Evaluating a user-centered sustainable mobile phone app for tourists and residents using facial-recognition software”

Reviewer Comment 2: Overly colloquial language and typos