Publication Trends on the Varying Coefficients Model: Estimating the Actual (Under)Utilization of a Highly Acclaimed Method for Studying Statistical Interactions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript addresses the utilization gap of the varying coefficients model (VCM) in empirical research despite its proven ability to uncover complex statistical interactions. The authors employ bibliometric methods to highlight discrepancies between the VCM's theoretical acclaim and its practical adoption, focusing on its dominance in methodological journals compared to application-based ones. This issue is relevant and thought-provoking, particularly for bridging the gap between statistical innovation and its real-world utility.

 

The paper has several strengths. It emphasizes an underexplored topic by analyzing the systemic challenges faced by a robust methodological tool like the VCM. The bibliometric approach, albeit limited, provides an interesting perspective on how different research domains adopt advanced statistical techniques. Furthermore, the examples provided—such as the hazard perception case—effectively illustrate the model's potential to deliver insights unattainable by traditional regression models with interaction terms.

 

However, the manuscript also faces notable limitations. First, the reliance on Google Scholar as the primary data source for bibliometric analysis raises concerns about data reliability and representativeness. Established bibliometric databases like Web of Science or Scopus would provide a more robust foundation for such analyses. Second, the manuscript lacks depth in its theoretical exploration of the factors influencing VCM adoption. While the gap is identified, there is little discussion about potential underlying causes or how this issue might be addressed. Additionally, the manuscript is heavily focused on a limited set of research domains, leaving out potentially relevant fields where VCM could be gaining traction. Finally, the graphical and tabular presentation of the data could be improved to make the findings more accessible and impactful.

 

Recommendations for Revision

Strengthen Data Reliability: Replace or supplement the Google Scholar-based bibliometric analysis with data from more reliable sources such as Web of Science or Scopus. This would enhance the credibility and generalizability of the findings.

 

Expand Research Domain Scope: Broaden the analysis to include additional fields, such as physics, machine learning, or public health, where the VCM might also hold potential. This would provide a more comprehensive picture of the model's adoption.

 

Discuss Barriers to Adoption: Include a dedicated section exploring potential reasons behind the underutilization of the VCM. Consider discussing:

 

The complexity of implementation in comparison to traditional regression models.

The lack of user-friendly software tools or training materials for VCM.

Perceived vs. actual advantages of VCM in specific empirical contexts.

Enhance Data Presentation: Add more visual aids (e.g., bar charts, network diagrams) to illustrate key findings, such as the distribution of VCM usage across domains or the ratio of methodological to empirical publications.

 

Focus on Actionable Insights: Provide recommendations for how the research community could bridge the gap between methodological innovation and practical adoption. Suggestions might include developing training modules, creating application-focused examples, or enhancing software implementations.

 

Author Response

Dear Reviewer,

Thank you for the constructive and insightful comments on the previous version of the manuscript. After carefully addressing the comments, the manuscript has significantly improved and now provides a stronger and more comprehensive message to the readers of Publications. The responses to the comments are detailed below and can also be read in the attached file.

Sincerely,

The authors

 

This manuscript addresses the utilization gap of the varying coefficients model (VCM) in empirical research despite its proven ability to uncover complex statistical interactions. The authors employ bibliometric methods to highlight discrepancies between the VCM's theoretical acclaim and its practical adoption, focusing on its dominance in methodological journals compared to application-based ones. This issue is relevant and thought-provoking, particularly for bridging the gap between statistical innovation and its real-world utility.

Thank you for the positive feedback on the message of the manuscript.

 

The paper has several strengths. It emphasizes an underexplored topic by analyzing the systemic challenges faced by a robust methodological tool like the VCM. The bibliometric approach, albeit limited, provides an interesting perspective on how different research domains adopt advanced statistical techniques. Furthermore, the examples provided—such as the hazard perception case—effectively illustrate the model's potential to deliver insights unattainable by traditional regression models with interaction terms.

Thank you for recognizing and for crystallizing the strengths of the paper.

 

However, the manuscript also faces notable limitations. First, the reliance on Google Scholar as the primary data source for bibliometric analysis raises concerns about data reliability and representativeness. Established bibliometric databases like Web of Science or Scopus would provide a more robust foundation for such analyses. Second, the manuscript lacks depth in its theoretical exploration of the factors influencing VCM adoption. While the gap is identified, there is little discussion about potential underlying causes or how this issue might be addressed. Additionally, the manuscript is heavily focused on a limited set of research domains, leaving out potentially relevant fields where VCM could be gaining traction. Finally, the graphical and tabular presentation of the data could be improved to make the findings more accessible and impactful.

Thank you for pointing out these notable limitations. Addressing them has made the message of the manuscript substantially more comprehensive, stronger, and meaningful.

 

Recommendations for Revision

Strengthen Data Reliability: Replace or supplement the Google Scholar-based bibliometric analysis with data from more reliable sources such as Web of Science or Scopus. This would enhance the credibility and generalizability of the findings.

The revised version of the manuscript addresses two sound opinions, the reviewer’s and the Editor’s. To summarize these opinions: The reviewer commented that for greater credibility and generalizability, the Google Scholar-based bibliometric analysis should be replaced or supplemented with data from WOS or Scopus. The Editor commented that it might be appropriate to make it clear that while this is an exploratory analysis, WOS and/or Scopus would be used in a follow-up, and that references to previous works in Publications using Google-Scholar for exploratory bibliometric analysis would be encouraged in this context.

The two opinions were addressed in the following ways:

 

Reviewer: The main Google-Scholar based analysis was supplemented by an analysis in WOS. The findings from this analysis are presented in Figure 8 (p.21) and are interpreted above and below the figure in the new section “Cross-validation analysis with Web of Science (WOS)” (starting on p.20). The findings of the analysis suggested that overall, the distribution of VCM publications across domains was in alignment between Google-Scholar and WOS and therefore provided support to the analysis in Google Scholar. The few differences that were found between the distributions showed that there are additional fields in which the VCM might hold potential, as the reviewer had anticipated, and this finding had been integrated into the General Discussion (see response to the following reviewer comment). The purpose of the cross-validation analysis was communicated in the Method section below Figure 5 (p.15).

 

Editor: The readers are now informed that the main analysis in which VCM publications were mapped across methodological and non-methodological outlets in various domains was exploratory. For this reason, it could be conducted in Google-Scholar, capitalizing on the large dataset that this database offers like in previous works that are now cited in the manuscript (from Publications and from additional outlets). It is also mentioned that Google-Scholar was an option here because there was no need to import the data to specialized bibliometric software programs (p.16, 2^nd paragraph). In addition, it is now explained in a new paragraph above the General Discussion that at this stage, the mapping of VCM distribution across domains and outlets could be exploratory, but future mappings will involve multiple databases for greater accuracy (like WOS and/or Scopus which are noted at the beginning of the paragraph, p.23, 1^st paragraph).

 

 

Expand Research Domain Scope: Broaden the analysis to include additional fields, such as physics, machine learning, or public health, where the VCM might also hold potential. This would provide a more comprehensive picture of the model's adoption.

 

The reviewer was correct to note that the VCM might hold potential in domains that were not mentioned in the previous version of the manuscript. The supplemental analysis in WOS (see response above), has indeed pointed to additional such domains. Specifically, as noted above Figure 8 (on p.21), the category “Medicine” in the figure included publications from the category “Public, Environmental and Occupational Health”. This finding was also discussed in the General Discussion (p.26, last sentence of the 2^nd paragraph). In addition, the search in WOS also revealed publications in the Social Sciences (although almost entirely mathematical), as noted in the paragraph below Figure 8 (p.21), and further discussed in the General Discussion (starting at the last line of p.26). Machine learning has not surfaced in the analysis in WOS. However, it was identified in the search in Google Scholar and merged into the category “Information and data science” (see notes below Figure 6 on p.18). The latter category is now more visible after accepting the reviewer’s suggestion to present the findings in a figure rather than in tabular form. Finally, the number of retrieved publications in Physics remained small even after supplementing the analysis in Google Scholar with the analysis in WOS. This is noted in the paragraph below Figure 8 (on p.21) and again in the General Discussion (p.27, the last two sentences of the 3^rd paragraph). Thank you for this comment.

 

Discuss Barriers to Adoption: Include a dedicated section exploring potential reasons behind the underutilization of the VCM. Consider discussing:

 

The complexity of implementation in comparison to traditional regression models.

The lack of user-friendly software tools or training materials for VCM.

Perceived vs. actual advantages of VCM in specific empirical contexts.

 

The reviewer’s comment led to a broader literature review, revealing that failing to utilize statistical innovations in research is a problem that has generated discussions in the literature regarding its possible reasons and potential remedies. This new knowledge has been integrated into a comprehensive General Discussion (starting on p.23) that provides readers both with a general framework for understanding the possible reasons that statistical innovations are not implemented and a more specific view on the obstacles to adopting the VCM. The specific view on the VCM included elaborations on the complexity of implementing the VCM and knowledge from an extensive literature review on the availability (or lack thereof) of user-friendly software tools for implementing the VCM. These points were indeed identified by the reviewer as potential obstacles to using the VCM. The reviewer also suggested looking for perceived vs actual advantages of VCM in specific empirical contexts. Such discussions were not found in the literature despite broadening the search in this direction. However, it is possible to search again if the reviewer finds that the General Discussion should be further expanded. Thank you for this comment and for the guidance on how to approach this broader discussion. The manuscript now offers a broader view on the underutilization of the VCM and potential ways to address it. Accordingly, a third aim was added to the manuscript’s aims on p.8: “To provide insights into the obstacles to utilizing the VCM and how its utilization might be increased”.

  

Enhance Data Presentation: Add more visual aids (e.g., bar charts, network diagrams) to illustrate key findings, such as the distribution of VCM usage across domains or the ratio of methodological to empirical publications.

Thank you for this comment. The Tables in the paper were replaced by figures (Figure 6, on page 18, replaced Table 1, and Figure 7, on page 19, replaced Table 2) and it is indeed much easier now to identify the patterns that the text in the Results section refers to as the basis for the findings.

Focus on Actionable Insights: Provide recommendations for how the research community could bridge the gap between methodological innovation and practical adoption. Suggestions might include developing training modules, creating application-focused examples, or enhancing software implementations.

The revised General Discussion now incorporates both a broad view on actionable insights for bridging the gap between methodological innovation and practical adoption, and more specific discussions on these insights in the context of the VCM. These insights and discussions are presented in the subsection of the General Discussion, “Communicating the VCM to researchers and research students” (which starts on p.26). Thank you for this comment.

 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This paper studies varying coefficient model. In various literatures, it was believed that varying coefficient model is good at handling interactions. However, this paper claims that varying coefficient model does not really perform well in practice.

I have the following comments.

 

1.The intro needs to highlight the main innovations compared with existing works.

 

2. What is the main technical novelty in this paper? At first glance, it seems that this paper is mainly about experiments. I would like to expect if there are some novel methods.

 

3. Some related works need to be discussed. These works also requires forecasting with multiple variables

Menghao Huo and Kuan Lu and Yuxiao Li and Qiang Zhu. CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting. Arxiv 2501.08620.

 

Ke, Zong and Zhou, Shicheng and Zhou, Yining and Chang, Chia Hong and Zhang, Rong. Detection of AI Deepfake and Fraud in Online Payments Using GAN-Based Models. arXiv:2501.07033.

Puning Zhao and Lifeng Lai. Minimax optimal Q learning with nearest neighbors. IEEE Transactions on Information Theory 2024.

4. The authors state that "It is unclear why researchers prefer using the regression model with an interaction 430 term on using the VCM.". Hope that the authors can provide some intuitive reasons.

 

5. For a better expression, I suggest that this paper use more figures. This will make the paper easier to read.

Comments on the Quality of English Language

The English usage is good in general.

Author Response

Dear Reviewer,

Thank you for the constructive and insightful comments on the previous version of the manuscript. After carefully addressing the comments, the manuscript has significantly improved and now provides a clearer focus and a more comprehensive message to the readers of Publications. The responses to the comments are detailed below and can also be read in the attached file.  

Sincerely,

The authors

 

This paper studies varying coefficient model. In various literatures, it was believed that varying coefficient model is good at handling interactions. However, this paper claims that varying coefficient model does not really perform well in practice.

 

I have the following comments.

 

 1.The intro needs to highlight the main innovations compared with existing works.

Thank you. This comment has led to re-reading of the Introduction and finding that indeed, it did not highlight the focus of this paper compared with existing works. Consequently, the revised Introduction now includes the following clarifications on the focus of this work in contrast to previous works:

• “In the current paper, while the prowess of the VCM in studying statistical interactions is demonstrated, the focus is not on its prowess but rather, on the extent to which it is utilized”. (First paragraph of the Introduction, on p.3, highlighted in yellow).

 

• “Note that since this paper is not intended to introduce novel computational methods for VCMs, the formulas in the Background section are not essential for engaging with its main points”. (Opening sentence of the third paragraph of the Introduction, on p.3, highlighted in yellow).

 

2. What is the main technical novelty in this paper? At first glance, it seems that this paper is mainly about experiments. I would like to expect if there are some novel methods.

Once again, the reviewer is correct about the unclarity regarding the paper’s focus. The Background section, which is more technical, might lead readers to look for technical innovations, while the aims are to show that despite its technical prowess, the VCM is largely underutilized, that its utilization varies across domains, and that there are possible reasons and potential remedies for its underutilization. To tackle this unclarity in the Introduction, before presenting the more technical Background section, the third paragraph of the Introduction has been revised in the following ways:

• Adding a new opening sentence to this paragraph (see also the response to comment 1 above):

“Note that since this paper is not intended to introduce novel computational methods for VCMs, the formulas in the Background section are not essential for engaging with its main points”. (Opening sentence of the third paragraph of the Introduction, highlighted in yellow).

• Rephrasing the second sentence of the paragraph to be clearer regarding the aims of the paper:

“Readers can skip the Background section and still address the main points of this paper: that the VCM is underutilized, that its acceptance as a tool for studying statistical interactions varies across research domains, and that there are possible reasons and potential remedies for its current underutilization”. (Second sentence of the third paragraph of the Introduction, highlighted in yellow).

 

3. Some related works need to be discussed. These works also requires forecasting with multiple variables

 

Menghao Huo and Kuan Lu and Yuxiao Li and Qiang Zhu. CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting. Arxiv 2501.08620.

 

Ke, Zong and Zhou, Shicheng and Zhou, Yining and Chang, Chia Hong and Zhang, Rong. Detection of AI Deepfake and Fraud in Online Payments Using GAN-Based Models. arXiv:2501.07033.

 

Puning Zhao and Lifeng Lai. Minimax optimal Q learning with nearest neighbors. IEEE Transactions on Information Theory 2024.

The reviewer pointed to important works that contribute to forecasting with multiple variables. These works can indeed be compared to previous works on forecasting with the VCM. Yet, it should also be considered that such comparisons might shift the focus of the paper from addressing the extent to which the VCM is utilized to the computational characteristics of the VCM. The reviewer was correct to note in the previous comment on the unclarity regarding the innovation in the paper. Perhaps the reviewer will agree that it was the technical aspects of the Background section that were responsible for this unclarity (see response above), and therefore, that referring to novel methods might lead the readers to look for computational advancements in the paper. However, if the reviewer disagrees with this perspective, integrating the suggested works to provide a broader view on forecasting with multiple variables can certainly be reconsidered.  

 

4. The authors state that "It is unclear why researchers prefer using the regression model with an interaction 430 term on using the VCM.". Hope that the authors can provide some intuitive reasons.

Thank you for this comment. It has led to extending the literature review to papers on obstacles to adopting statistical innovations and potential remedies. This knowledge has been incorporated into a revised General Discussion (starting on p.23) in which it is addressed in the context of utilizing the VCM. Thanks to this comment the paper now offers readers a deeper perspective on the underutilization of the VCM and on potential ways to move forward. Accordingly, a third aim was added to the manuscript’s aims on p.8: “To provide insights into the obstacles to utilizing the VCM and how its utilization might be increased”.

 

5. For a better expression, I suggest that this paper use more figures. This will make the paper easier to read.

Thank you for this comment. The Tables in the paper were replaced by figures (Figure 6, on page 18, replaced Table 1, and Figure 7, on page 19, replaced Table 2) and it is indeed much easier now to identify the patterns that the text in the Results section refers to as the basis for the findings.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I appreciate the authors’ thoughtful and comprehensive responses to the previous comments. The revised manuscript demonstrates clear and meaningful progress. The incorporation of a cross-validation analysis using the Web of Science, expansion of the domain scope, improvement of visual presentation, and enriched discussion on barriers to VCM adoption significantly strengthen the work.

 

Reviewer 2 Report

Comments and Suggestions for Authors

Thanks for the authors' response. I have read the paper and I have not found other issues.