Feedback to Minimize Household Waste a Field Experiment in The Netherlands

Round 1

Reviewer 1 Report

I would like to see this paper published. The motivation is well laid-out and the research design is solid. I appreciate that the authors did not just study behaviour but also perceptions and tried to get at the processes driving behavioural changes from t0 to t1. If the results are interpreted more accurately (at least in my view), I think it will make a valuable contribution to the literature. I also have a substantial number of other comments on the paper that I would like to see addressed.

One crucial point concerns the authors interpretation of the results. From my reading they find, very simply, that feedback did not do anything more than the control in terms of behaviour (actual waste), perceptions (self-reported waste, residual waste, reusing, or separating waste), or processes (self-efficacy, descriptive norms, self-identity, and procedural knowledge). I think this is the clear message that should emerge from the results, discussion, and abstract. I would not hint at differences that aren’t clearly supported by the study.

Currently the authors take two approaches to suggest some between treatment differences. One is that they point to that interpretation that “group means are considered statistically significantly different if the overlap of the confidence intervals is less than half the distance of one side of the confidence interval” (p. 9). They then use this idea to suggest at differences at multiple places (e.g. pp. 11, 12). This is fine if we care purely about the t0 to t1 differences. But here we care about the difference in change between the treatment and the control (for a clear description see Gelman and Stern’s 2012 article “The Difference Between ‘Significant’ and ‘Not Significant’ is not Itself Statistically Significant”). I suggest that the authors focus on the latter point; as indicated by the interaction terms in the regression tables there is no difference in the t0 to t1 change between the control and the treatments.

The other approach the authors take is to create a new control group (which they call the “true control” in the abstract) by looking at waste from a comparable neighbourhood that was not in the study (p. 10). They use the no change from t0 to t1 in the control neighbour to argue that relative to this new control—who were not in the study and given no information at all—feedback did reduce waste behaviour. I’m very sceptical about this approach: what makes us think that the new control neighbourhood is similar to the study neighbourhoods? Moreover, since only ≈12% of eligible participants signed up in the study neighbourhoods why not use the study neighbourhood (minus the waste from the included participants) to estimate this control? Or why not use a weighted combination of comparable nearby neighbourhoods? I would suggest that the authors either develop this approach substantially or that they remove it.

A few further comments, questions, or suggestions:

• To what extent can the authors separate between (i) frequency of depositing waste and (ii) amount of waste actually deposited. Can their measure distinguish between, for instance, going twice a week with two bags of waste or going once a week with four bags of waste? If not, then this could be important for their results.
• I was a bit confused about the procedural knowledge variable. I couldn’t find it in the Introduction but it is then included in Methods and Results.
• To what extent are the people participating in t0 and t1 surveys comparable? Table 1 does not contain values on income and education for t1 participants.
• I would like to see two limitations of the study addressed in the discussion. First is that few people participated from the neighbourhood (12%) in the study so there’s large selection at that level. Second there was also large selection from the behavioural side of the study to survey participation; from about n≈700 installing the app to n≈100 filling out the t1 questionnaire. What consequences does this selection have? I can see external validity consequences for the former and both internal and external validity consequences to the latter.
• There are a lot of separate tables and figures. I would suggest trying to combine tables and figures to make more efficient use of space and help the reader. For instance, Table 4 and 5 could be combined into one. Figures 4, 5, 6, and 7 could be combined.
• Incidentally, I couldn’t find figure for results of self-reported waste (p. 10). For all other results there was a figure.
• If Section 3.4 was exploratory please mention that this is the case. (I do think this section is useful!)
• 3. It wasn’t clear to me why a social comparison feedback would lead to weak self-efficacy increase. If a recipient of the information is doing well relative to others, couldn’t this increase self-efficacy?
• 8. I was a bit surprised to see the low Cronbach’s alpha for the separating waste items. Could you mention briefly, even in the supplementary is fine, which items covaried and which ones did not?
• 9. Please use “descriptive norms” label not “social norms” for this measure.
• 19 practical implications heading is not formatted correctly and the final paragraph of the paper “Authors should discuss the results…” is left in from, I presume, the submission instructions.

Author Response

I would like to see this paper published. The motivation is well laid-out and the research design is solid. I appreciate that the authors did not just study behaviour but also perceptions and tried to get at the processes driving behavioural changes from t0 to t1. If the results are interpreted more accurately (at least in my view), I think it will make a valuable contribution to the literature. I also have a substantial number of other comments on the paper that I would like to see addressed.

 Thank you for your positive feedback and the suggestions. We have improved the paper based on your suggestions and think the paper has improved; we hope you agree.

One crucial point concerns the authors interpretation of the results. From my reading they find, very simply, that feedback did not do anything more than the control in terms of behaviour (actual waste), perceptions (self-reported waste, residual waste, reusing, or separating waste), or processes (self-efficacy, descriptive norms, self-identity, and procedural knowledge). I think this is the clear message that should emerge from the results, discussion, and abstract. I would not hint at differences that aren’t clearly supported by the study.

 We agree with the reviewer that our results mainly suggest that we do not find an effect of feedback compared to the control group. We now more clearly emphasize this throughout the paper. However, we do think the comparison with the control neighbourhood is valuable. We selected this neighbourhood in collaboration with the waste collection company because it is rather similar to the neighborhood where we conducted our experiment. Reviewer also indicated that this comparions is valuable. However, we now more clearly emphasize that it is a control neighbourhood. Since it is a different neighbourhood other factors could explain why this neighbourhood did not redcue waste. Therefore, we no longer label it a ‘true control group’, but instead a ‘control neighbourhood’ and clarify this throughout the paper.

Currently the authors take two approaches to suggest some between treatment differences. One is that they point to that interpretation that “group means are considered statistically significantly different if the overlap of the confidence intervals is less than half the distance of one side of the confidence interval” (p. 9). They then use this idea to suggest at differences at multiple places (e.g. pp. 11, 12). This is fine if we care purely about the t0 to t1 differences. But here we care about the difference in change between the treatment and the control (for a clear description see Gelman and Stern’s 2012 article “The Difference Between ‘Significant’ and ‘Not Significant’ is not Itself Statistically Significant”). I suggest that the authors focus on the latter point; as indicated by the interaction terms in the regression tables there is no difference in the t0 to t1 change between the control and the treatments.

Indeed, the results from our multilevel analyses show that the decrease in waste is not larger in the feedback conditions compared to the control condition. We removed the statements on the confidence intervals not overlapping in line with the suggestions from the reviewer.

The other approach the authors take is to create a new control group (which they call the “true control” in the abstract) by looking at waste from a comparable neighbourhood that was not in the study (p. 10). They use the no change from t0 to t1 in the control neighbour to argue that relative to this new control—who were not in the study and given no information at all—feedback did reduce waste behaviour. I’m very sceptical about this approach: what makes us think that the new control neighbourhood is similar to the study neighbourhoods? Moreover, since only ≈12% of eligible participants signed up in the study neighbourhoods why not use the study neighbourhood (minus the waste from the included participants) to estimate this control? Or why not use a weighted combination of comparable nearby neighbourhoods? I would suggest that the authors either develop this approach substantially or that they remove it.

We agree with the reviewer that we cannot call this group a ‘true control’. We selected a different neighbourhood and differences between the neighbourhood may explain the findings, not the feedback intervention. However, we do think it is interesting to include this comparison. In collaboration with the waste collection company we selected this neighbourhood because it is rather comparable to the intervention neighbourhood. See also the comment from reviewer 2. The comparison neighbourhood does not show a change in waste. The groups that received feedback do show a significant reduction in waste. Yet, those groups indeed only represent 12% of the neighbourhood. Namely, those who installed the mobile application. Perhaps this group was already more willing to reduce their waste and therefore installed the app. We explain that future research is needed to test feedback on waste compared to a true control group that does not receive any information and is randomly assigned to a feedback or control condition.

A few further comments, questions, or suggestions:

• To what extent can the authors separate between (i) frequency of depositing waste and (ii) amount of waste actually deposited. Can their measure distinguish between, for instance, going twice a week with two bags of waste or going once a week with four bags of waste? If not, then this could be important for their results.

The card measures how many times households open the underground container. Yet, the container only has room for one full waste bag at a time. Therefore, if a household member throws away four full waste bags at a time, they still need to open and close the container four times. Therefore, the card will still monitor that the container has been opened four times. We clarified this now in the method section. However, it may be that case that households (e.g. older inhabitants who do not want to carry a heavy waste bag) do not dispose full waste bags, but only waste bags that are half full. We could not measure this. Therefore, we also included the self-report measures.

• I was a bit confused about the procedural knowledge variable. I couldn’t find it in the Introduction but it is then included in Methods and Results.

We now added to the present study section that we also measure procedural knowledge to test if information on how to reduce waste increases the extent to which households think they can reduce waste.

• To what extent are the people participating in t0 and t1 surveys comparable? Table 1 does not contain values on income and education for t1 participants.

To increase the response rate at t1 we tried to keep the questionnaire as short as possible. Therefore, we did not measure income and educational level at t1. Therefore, we unfortunately cannot compare the samples on these variables. However, the scores on the other demographic variables suggest that the samples are comparable on these variables.

• I would like to see two limitations of the study addressed in the discussion. First is that few people participated from the neighbourhood (12%) in the study so there’s large selection at that level. Second there was also large selection from the behavioural side of the study to survey participation; from about n≈700 installing the app to n≈100 filling out the t1 questionnaire. What consequences does this selection have? I can see external validity consequences for the former and both internal and external validity consequences to the latter.

Thank you for this suggestion. We added a limitations section to the discussion section where we discuss these concerns.

• There are a lot of separate tables and figures. I would suggest trying to combine tables and figures to make more efficient use of space and help the reader. For instance, Table 4 and 5 could be combined into one. Figures 4, 5, 6, and 7 could be combined.

Following this suggestion we combined the Tables and Figures.

• Incidentally, I couldn’t find figure for results of self-reported waste (p. 10). For all other results there was a figure.

We included a figure on the self-reported waste (Figure 4, now Figure 4a). However, perhaps it was not clear that this figure refers to self-reported residual waste. Therefore, we clarified the figure title (self-reported residual waste).

• If Section 3.4 was exploratory please mention that this is the case. (I do think this section is useful!)

We added that we aimed to test if the changes in the process variables were related to changes in waste behaviour. However, as we did not find any changes in the process variables we exploratorily tested the correlations between the process variables and the indicators of waste behaviours.

• It wasn’t clear to me why a social comparison feedback would lead to weak self-efficacy increase. If a recipient of the information is doing well relative to others, couldn’t this increase self-efficacy?

Indeed, the behaviour of others may influence self-efficacy as well (Bandura, 1982). However, the behaviour of others is likely to influence self-efficacy less than one’s own behaviour. We now clarified throughout the paper that we expect social comparison feedback to influence self-efficacy. However, we expect that the influence is weaker compared to goal comparison and historical comparison feedback.

• I was a bit surprised to see the low Cronbach’s alpha for the separating waste items. Could you mention briefly, even in the supplementary is fine, which items covaried and which ones did not?

The scale for the premeasure does not improve if one of the items is removed. In the post measure Cronbach’s alpha increases to .51 if the item measuring organic waste is removed. If the item on textile is also removed Cronbach’s alpha then further increases to .63. We added this information in a footnote.

• Please use “descriptive norms” label not “social norms” for this measure.

We now consistently use ‘descriptive social norms’ throughout the paper.

• 19 practical implications heading is not formatted correctly and the final paragraph of the paper “Authors should discuss the results…” is left in from, I presume, the submission instructions.

Indeed, we accidentally left this text in the paper, we now removed it. Furthermore, we checked the formatting of our paper.

Reviewer 2 Report

The paper presents an original contribution by analyzing the behavioral approach to household waste reduction. I belive the contribution has potential but, in my opinion, some major issues must to be fixed in order to be published.

1. The participation rate is too low meaning that only 79 people fill the two questionnaires up to more than 700: why? Have you tried to send again the invitation letter or to contact them by phone?
2. Regarding the treatment and the control groups:
   1. do you have the possibility to select another small group of people and analyze their "access" to the bins without providing any information? This could be useful to have a "true" control group.
   2. What is the procedure to select the neighborhood? In providing information about others' behavior, did you control (and match according to) households' size?
3. Regarding the measure of waste reduction: if I correctly understood, you directly control for the number of times people open the bins whereas is not clear to me how you measures the number of bags. Is it based on self-reported measure? Did you control for the consistency of responses to (try to) avoid the effect of cheating?
4. Regarding the main results you present:
   1. be careful in specifying in all cases the dependent variable and provide some descriptive statistics;
   2. by looking at the figures in the paper it is really unexpected that you do not find any significant results in any of the regression you presented. I believe you should try different specifications.
   3. why did you not use mixed effect and control for demographics. In my opinion, the household's size is a key variable to be included.  
   4. in my opinion, if you run estimation using the right model and constrol for all the information you have, it could be the case that your arrive at different conclusions.
5. Other minor issues:
   1. be consistent in the way you write references (round or squared brackets)
   2. In Figure 1, you should put in the caption the translation of the content of the screenshot to guarantee the readability also to no Dutch people
   3. you may also add these references:

      Faruqui, Ahmad, Sanem Sergici, and Ahmed Sharif. "The impact of informational feedback on energy consumption—A survey of the experimental evidence." Energy 35.4 (2010): 1598-1608.

      Tiefenbeck, Verena, et al. "Real-time feedback reduces energy consumption among the broader public without financial incentives." Nature Energy 4.10 (2019): 831-832.

Author Response

Comments and Suggestions for Authors

The paper presents an original contribution by analyzing the behavioral approach to household waste reduction. I belive the contribution has potential but, in my opinion, some major issues must to be fixed in order to be published.

1. The participation rate is too low meaning that only 79 people fill the two questionnaires up to more than 700: why? Have you tried to send again the invitation letter or to contact them by phone?

Thank you for the kind words on our paper. We recruited participants by sending a postal letter to all households in the neighbourhood that use the underground bin to dispose their waste. The letter was sent by the municipality to increase the chance that would inhabitants open and read the letter. Furthermore, we advertised the study in local media and by putting up signs in the neighborhoods. We also distributed prizes to participants who filled out both questionnaires to increase the response rate. Finally, we sent push notifications via the mobile application to remind participants to fill out the questionnaire. Unfortunately, 327 participants out of the 723 who installed the app filled out one of the questionnaires. Furthermore, despite our effort, only 79 participants filled out both questionnaires. We clarified this in the method section. We did not contact people in the neighborhood by phone because we did not have their phone number.

1. Regarding the treatment and the control groups:
1. do you have the possibility to select another small group of people and analyze their "access" to the bins without providing any information? This could be useful to have a "true" control group.

Indeed, in collaboration with the waste collection company we selected a comparable neighbourhood and analyzed their use of the bin during the same time period in order to have comparison group that was not exposed to the intervention. The results suggest that participants in the comparison neighbourhood did not reduce their waste, while the intervention groups did. However, as reviewer 1 also seemed to indicate, this comparison neighbourhood is not a true control group because participants were not randomly assigned to either this condition or one of the feedback conditions. Perhaps there are other differences between the neighbourhoods that explain why this comparison neighbourhood did not reduce waste during the same time period. To clarify this, we now also added the results of this neighbourhood to Figure 3. However, we do not label this neighbourhood as ‘true control group’, but as ‘comparison neighbourhood’.

1. What is the procedure to select the neighborhood? In providing information about others' behavior, did you control (and match according to) households' size?

We selected a neighbourhood that included enough inhabitants to test the intervention, that used underground bins to dispose of waste in order to be able to monitor their waste and where there was still room to further reduce the amount of household waste. We clarified this in the method section.

We did not control for household size when providing households with social comparison feedback. The information from the questionnaire (where we asked for household size) was not linked to the app due to privacy reasons. We now discuss in the discussion section that social comparison feedback may be more effective when the comparison group is matched to the household receiving the feedback.

1. Regarding the measure of waste reduction: if I correctly understood, you directly control for the number of times people open the bins whereas is not clear to me how you measures the number of bags. Is it based on self-reported measure? Did you control for the consistency of responses to (try to) avoid the effect of cheating?

We measure waste by monitoring how often households open the underground bin. By opening the underground bin with their card households can dispose of one waste bag at a time. They need to scan their card again if they want to dispose another waste bag. We clarified this in the method section.

1. Regarding the main results you present:
1. be careful in specifying in all cases the dependent variable and provide some descriptive statistics;

Thank you for this suggestion. We included the means of the dependent variables per condition in the graphs and make sure that we always clarify what the dependent variable is.

1. by looking at the figures in the paper it is really unexpected that you do not find any significant results in any of the regression you presented. I believe you should try different specifications.

We did find significant results in the multilevel analysis. Specifically, we found that all intervention groups reduced the actual number of waste bags from t0 to t1. Furthermore, we found that all groups increased self-reported household waste reduction from t0 to t1. However, we expected that we would only find this for the feedback conditions and not for the control condition. The figures show a similar pattern, namely that the increase in the feedback conditions is not larger compared to the increase in the control condition.

Additionally, we discussed our analyses with a statistician and the analyses we conducted are suitable to test our hypotheses.

1. why did you not use mixed effect and control for demographics. In my opinion, the household's size is a key variable to be included.  

We randomly assigned participants to the conditions and therefore assume that demographics and household size are equally distributed among the groups and do not influence the results. Furthermore, from a practical perspective it is important to test the overall effects without controlling for demographics and household size. Because for practitioners it is simply relevant to know whether overall there are reductions in waste. However, following the reviewer’s suggestion we conducted the analyses including household size as a covariate. We indeed find that a larger household size is related to more waste bags (fixed effect = 6.44, p < .001). However, the other findings remain similar; there is still a main effect of time, goal comparison feedback, historical feedback and social comparison feedback and there are no interaction effects. When we included household size in the analysis for residual waste we again found that larger households produce more residual waste (fixed effect =190.65, p < .001). Again, the other findings remain similar to the findings when we did not include household size as a covariate. We still find a main effect of goal comparison feedback. However, we no longer find a significant main effect of social comparison feedback on residual waste. Furthermore, household size was not significantly related to reducing waste, reusing waste, and separating waste. We added in a footnote in the results section that when we included household size as a covariate and that the results remain similar.

1. in my opinion, if you run estimation using the right model and constrol for all the information you have, it could be the case that your arrive at different conclusions.

See our responses above. We consulted a statistician to discuss if we conducted the correct analyses, which we indeed did. Furthermore, as suggested by the reviewer we ran the analysis including household size as a covariate and the results remained similar.

1. Other minor issues:
1. be consistent in the way you write references (round or squared brackets)

We made sure that all references are now in squared brackets.

1. In Figure 1, you should put in the caption the translation of the content of the screenshot to guarantee the readability also to no Dutch people

We changed Figure 1 to an English version.

1. you may also add these references:

Faruqui, Ahmad, Sanem Sergici, and Ahmed Sharif. "The impact of informational feedback on energy consumption—A survey of the experimental evidence." Energy 35.4 (2010): 1598-1608.

Tiefenbeck, Verena, et al. "Real-time feedback reduces energy consumption among the broader public without financial incentives." Nature Energy 4.10 (2019): 831-832.

Thank you for the suggested papers, we included these references in the paper when we refer to previous research that has found that feedback can reduce energy use.  

Reviewer 3 Report

The manuscript entitled “Feedback to minimize household waste – A field experiment in the Netherlands”presents aresearch related to the sustainable waste management. The topic of the research is up to date as nowadays the production of wastes contributes to several environmental problems. The authors analyzed thoroughly the idea of waste minimization and also considered feedback and comparison standards. The authors tried to verify what is the impact of feedback on waste minimization.

General comments:

• I think that slightly confusing is the section described in lines 178-196. The authors mentioned that they will examine the influence of impact on waste minimization and will focus on the reduction of waste. I recommend to present the study in the past tense, not as a plan for the future.
• I am not also sure whether the group of 406 participants is enough for the questionnaire study. How was the representativeness of this group verified?
• Figure 1 is not of a good quality. Also, the English language should be used. As for now, it is not possible to understand what information is given in the application layout. The same in Figure 2 – how would the reader be able to understand the idea as the language is not English there?
• Please explain what are Est, SE and t in Table 3. This table should be placed after the line where it was mentioned for the first time.
• Tables 5-7 are not cited in the manuscript.
• The discussion is too general. I expect to see the confrontation between the outcomes of this study and the results of similar research presented in the scientific literature.
• More information should be given to clarify the statistical attempt of data processing and analysis.
• I expect to see the limitations of performed study. Please strengthen the scientific novelty.
• The authors stated that “future research directions may also be highlighted”. What are these directions? Please specify.
• Please add the chapter “Conclusions” to sum up the most important findings of the study.
• The theme of the manuscript is valuable, nevertheless the content must be thoroughly improved before the publication.

Author Response

General comments:

• I think that slightly confusing is the section described in lines 178-196. The authors mentioned that they will examine the influence of impact on waste minimization and will focus on the reduction of waste. I recommend to present the study in the past tense, not as a plan for the future.

We now left out the section that the reviewer found confusing. Thank you for this suggestion. We rewrote the paragraph ‘present study’ in the past tense.

• I am not also sure whether the group of 406 participants is enough for the questionnaire study. How was the representativeness of this group verified?

We included the results from a power analysis which shows that we needed 179 participants for 80% power. Therefore, the power analysis shows that our sample size is sufficient to test our hypotheses. We added this information to the method section. However, it is important to note that only 12% of the inhabitants of the neighbourhood installed the application to receive feedback and of those, less than half filled in one of the questionnaires. Furthermore, we compared our questionnaire sample to the population of the neighbourhood. There were more men in our sample compared to the neigbourhood and the average age was higher. We included in the discussion section that our results should be interpreted with care and may not apply to the entire neighbourhood. However, importantly, we did randomly assign participants to the conditions, therefore we do not expect our sample to differ between conditions. We now discuss this in the limitations section in the discussion section.

• Figure 1 is not of a good quality. Also, the English language should be used. As for now, it is not possible to understand what information is given in the application layout. The same in Figure 2 – how would the reader be able to understand the idea as the language is not English there?

We adapted both figures accordingly.

• Please explain what are Est, SE and t in Table 3. This table should be placed after the line where it was mentioned for the first time.

Thank you for this suggestion. We now always place the table after we refer to it. Furthermore, we explain in a footnote below that table that Est. refers to Estimate and SE refers to Standard error. 

• Tables 5-7 are not cited in the manuscript.

We now refer to Table 5-7 as well in the text.

• The discussion is too general. I expect to see the confrontation between the outcomes of this study and the results of similar research presented in the scientific literature.

We added to the discussion section how our findings build on previous studies testing feedback. We now discuss that our finding that feedback can change behavior is in line with previous research (e.g. Osbaldiston & Schott, 2012; Abrahamse et al., 2005). However, previous studies have mostly tested the effect of feedback on energy consumption while our study tested the effect on waste. Furthermore, we added that previous research has focused on feedback comparison standards (for an overview see Karlin et al., 2015). Yet, to the best of our knowledge our study is the first to test all feedback comparison standards in one study.

• More information should be given to clarify the statistical attempt of data processing and analysis.

We added a paragraph on data analysis to clarify the data analysis.

• I expect to see the limitations of performed study. Please strengthen the scientific novelty.

We now added a paragraph ‘limitations’ in the discussion section where we discuss the main limitations of our study.

• The authors stated that “future research directions may also be highlighted”. What are these directions? Please specify.

Our apologies, this was a standard text from the example paper that was still included in the paper. We removed this text. Furthermore, throughout the discussion we now provide more specific suggestions for future research. For example, we indicated that future research is needed to test the influence of contextual factors, to measure the amount of actual waste, to randomly assign participants to a true control group as well.

• Please add the chapter “Conclusions” to sum up the most important findings of the study.

Thank you for this suggestion, we added a section ‘conclusion’ to the paper.

• The theme of the manuscript is valuable, nevertheless the content must be thoroughly improved before the publication.

Thank you very much, we have adapted the paper based on your suggestions and the suggestions from the other two reviewers. We think the paper has strongly improved and hope you agree.

Round 2

Reviewer 1 Report

Thank you to the authors for addressing almost all of my comments. The only one that I do not think was fully addressed concerns the comparison neighbourhood. Could you please add some details about the comparison neighbourhood? More specifically, on what basis do you think that it is comparable to the treatment neighbourhood? E.g. Similar housing costs/ wealth/crime/age distributions etc?

Author Response

Thank you for the suggestion. We now added data on the intervention neighbourhood and the comparison neighbourhood. Specifically, we included data on the percentage of women, the average age, the householdsize, and the percentage of owned houses. The neighbourhoods are comparable with regard to the percentage of women, householdsize and the percentage of owned houses. However, the average age is a bit higher in the intervention neighbourhood compared to the comparison neighbourhood. We included this information in the manuscript. 

Reviewer 2 Report

Authors clarified all the points raised in the first revision. The structure and the provided results are clearer in the current version of the manuscript.

Author Response

Thank you. 

Reviewer 3 Report

The authors have significantly supplemented and improved the presented manuscript, minor corrections may still be introduced to the presentation of empirical research results and their discussion. The manuscript is well prepared and the topic is interesting and important.

Author Response

Thank you for your suggestion. We went through the manuscript and tried to clarify our findings in the result and in the discussion section.