Digital Information Credibility: Towards a Set of Guidelines for Quality Assessment of Grey Literature in Multivocal Literature Review

Reviewer 1

Reviewer’s Comments/Concerns

Response/Action(s) Taken

1.  

In the DATA SYNTHESIS&ANALYSIS, why is the country analyzed and how does the country affect this study?

We greatly appreciate the reviewer’s efforts to review the paper carefully and offered the valuable suggestions.  With respect to the concerns raised by the reviewer, we are agreed with their questions.

ACTION:

In the Data Synthesis & Analysis the country is anlaysed because of the following reasons:

A-     In most of the MLR, researchers have included the details of countries and states. The reasons is to investigate where the specified work has been done in which areas. It will open a research direction to the researchers to work in the specified areas. In the literature, numerous researchers have analyzed the studies based on variables including countries, time period, contents, states, libraries, research methodologies, etc. Following are some research studies in which one these various have been analyszed:

1.       White, S. L., Graham, L. J., & Blaas, S. (2018). Why do we know so little about the factors associated with gifted underachievement? A systematic literature review. Educational Research Review, 24, 55-66.

2.       Sahu, A. K., Padhy, R. K., & Dhir, A. (2020). Envisioning the future of behavioral decision-making: A systematic literature review of behavioral reasoning theory. Australasian Marketing Journal, 28(4), 145-159.

3.       Iden, J., & Eikebrokk, T. R. (2013). Implementing IT Service Management: A systematic literature review. International Journal of Information Management, 33(3), 512-523.

4.       Hutchings, K. (2022). Why we need to know more about diversity among the globally mobile: a systematic literature review of non-traditional expatriate research and future research agenda for minority expatriates. Journal of Global Mobility: The Home of Expatriate Management Research, 10(1), 127-161.

5.       Khan, S. U., Niazi, M., & Ahmad, R. (2011). Barriers in the selection of offshore software development outsourcing vendors: An exploratory study using a systematic literature review. Information and Software Technology, 53(7), 693-706.

2

In the DATA SYNTHESIS&ANALYSIS, why do the authors analyze the countries and states of the study at the same time? Is there any duplication?

It is common for Multivocal literature reviews to analyze both countries and states within the same study as mentioned in the concern 1. This is because some research questions or topics may focus on a specific country or region, while others may require an analysis of multiple countries or regions.

Analyzing multiple countries or states within the same study can provide a broader perspective and allow comparisons between regions. For example, a systematic review on the effectiveness of credible information in digital news stories may analyze studies from multiple countries or states to compare the effectiveness of different interventions in different contexts.

However, it is possible for there to be duplication in the studies analyzed if the same study is conducted in multiple countries or states. To prevent duplication, the authors carefully screened and evaluated each study to ensure they do not analyze the same study multiple times. This has done by comparing study characteristics such as study design, sample size, and outcome measures. When the duplication is identified, the authors only included the study once in their analysis.

3

In the analysis of results, only the results were described, but the results were not analyzed.

We are thankful to the anonymous reviewer for the in-depth review and agree with his concern.

ACTION: In section 8, we have properly analyzed and discussed the results in the revised manuscript. For instance, Table 5 presents the proposed guidelines for the quality assessment of Grey literature, which results in developing a new proposed strategy, namely “SADACO.” Similarly, Table 8 shows the quartile-based analysis of our study's top fifteen journals, concluding that most of the publications are in the “Computers in Human Behaviour” Journal as the Eigenvalue is 0.05973.

4

The meaning and data of each part are marked in Figure 4, but not in Figure 3, resulting in the data in Figure 3 is not intuitive enough.

Thank you for raising the concern we are agreed with the comment.

ACTION: As per the reviewer's concern, we have redesigned Figure 4 and added labels to each segment. The changes can be seen in the Revised Manuscript.

5

 The reviewers think that mayb e the amount of data in the article was not enough and suggested increasing the amount of data.

We are thankful to the reviewer for this comment.

ACTION: In the revised manuscript, we have added 50 more  studies using one of the popular method of the Multivocal study i.e., “SnowBalling”. Discussion on snowballing is also added at section 8 in the revised manuscript.

We appreciate the time and efforts of the editor and reviewers in reviewing this manuscript. We have addressed all the reviewers’ concerns and issues indicated in the review report and believe that the revised version could meet the journal publication requirements.

Reviewer 2

1

Carefully check the Authors and Affiliations section and make necessary corrections. Currently, some information are missing (post code, country, etc.), other are not uniformly prepared (Capital Letters in Case of Authors Names, etc.).

We greatly appreciate the reviewer’s efforts to review the paper carefully and offered the valuable suggestions.  With respect to the concerns raised by the reviewer, we are agreed with their questions.

ACTION: We have carefully checked the manuscript and corrected the Authors names, affiliations, postal codes, and countries in the revised manuscript.

2

Several minor editorial and formatting issues are present throughout the whole manuscript. Therefore, a careful examination and proofreading seems necessary.

With respect to the concerns raised by the reviewer, we are agreed with their questions.

ACTION: In the revised manuscript we have corrected the formatting issues along with typos and grammatical mistakes.

3

Consider reformatting some fragments of text and reorganizing them into separate paragraphs. Look at Chapter 2 as an example – this is simply one chunk of text, which makes it hard to read and compare one research outcomes with another.

Thank you for your valuable concern and agreed with the comment.

ACTION: We have divided the Chapter 2 in different paragraphs by their concern relevancy. The changes can be seen in the revised manuscript.

4

Do not repeat a sentence again and again, try to rephrase what you want to say. Use similar words and not just one over and over. English grammar and style requires upgrades.

Thank you for your valuable concern and agreed with the comment.

ACTION: We have proofread the manuscript and many typos, grammatical mistakes have been corrected in the revised manuscript. Similarly, most the repeated sentences in the whole manuscript have been rephrased/removed.

5

At the end of Chapter 1, Authors should highlight the novelty of their paper, e.g., using bullet points. Then, they should describe the contents of their paper – what part or information is located in which chapter, etc.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: We have added the Contribution part at the end of Introduction Chapter with each contribution in the bullet form in the revised manuscript.

6

Chapter 3 is too short to be a standalone one – it should be merged with other fragments of text.

Thank you for highlighting this issue. We are agreed with this concern as the Chapter 3 was very short.

ACTION: As per the reviewer suggestion, we have merged Chapter 3 with Chapter 1 in the revised manuscript as it was related with Chapter 1.

7

Chapter 4 should be reorganized as well, its structure does not seem proper.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: We have reorganized the Chapter 4 with proper structure and the changes can be seen in the revised manuscript.

8

The [Search Strings] could be presented in a table, which would sum up each scenario, etc.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: As per the reviewer suggestion, we have presented the mentioned text in the Table.

9

The same remark goes to [Publication Quality Assessment].

Thank you for this comment. We are agreed with reviewer concern.

ACTION: As per the reviewer suggestion, we have presented Quality Assessment text in the Table in revised manuscript.

10

Some tables from Chapter 4 seem to be graphical elements, which were copy-pasted into the template. This is not the proper way of editing.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: As per the reviewer suggestion, all the Tables have redesigned and graphical elements have been removed.

11

General remark – all figures have too small fonts, with too low resolution, making it hard to read and interpret.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: All the figures have been redesigned with proper fonts and high resolutions. Now the revised Figures would be easy to read and interpret.

12

General remark – check all the tables and figures, as well as their captions, whether they meet the Journal and Template recommendations. Figures have enumeration and caption AFTER the initial figure. Whereas, Tables have their enumeration and caption BEFORE inserting the table itself.

Thank you for this comment. We are agreed with reviewer concern.

ACTION: As per the reviewer suggestion, all the Figures and Tables captions have been set according to Journal guidelines. Figure and Table captions have been properly placed as per the journal guidelines.

13

Where is the Conclusions section? Where is the Future Works chapter? Please look at other Journals from MDPI, including previously published papers, and make necessary corrections in case of your manuscript.

Thank you for this comment as the Conclusion and Future works section was missed mistakenly in the older version. We are agreed with reviewer concern.

ACTION:  We have separately included Conclusion and Future work section in the revised manuscript.

We appreciate the time and efforts of the editor and reviewers in reviewing this manuscript. We have addressed all the reviewers’ concerns and issues indicated in the review report and believe that the revised version could meet the journal publication requirements.

Reviewer 2

1

The revised version of this manuscript has improved a lot, considering the initial submission

We are very thankful to the anonymous reviewer for this comment.

2

However, Authors did not adequately respond to several remarks. Still, some aspects have not been solved, particularly considering the editorial and formatting side, as well as the layout, quality and resolution of tables, figures, graphs, etc.

We are thankful to the reviewer for this comment.

Action: We have reformatted all the tables exactly according to the journal format. For more clarity we have also increased its columns and row spacing.

We have also reformatted the figures, graphs. etc. exactly according to the journal format. To view these images we also improved it quality and resolution.