Using Conditional Random Forest and Feature Ranking Algorithms to Determine the Relative Importance of the Nicotine Metabolite Ratio (NMR) and Demographic and Behavioral Factors on Nicotine Dependence Severity

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript addresses an important topic in the science of smoking and dependence and is a potential significance contribution to the knowledge .

Manuscript review and formatting issues :

 

• Line 58: Spacing issues need to be revised.
• Line 62: Spell out CPD.
• Line 83: Spacing issues.
• Line 100: Add a period between “counties” and “To be.”
• Please reference tables/figures in the text before showing them so the reader can follow the presentation of information. For example, Figure 1 appeared before being mentioned in the text.

Introduction

1. The last part of the introduction should be better integrated with the information about NMR, rather than appearing only at the end as a stepwise addition of evidence. The interaction between these factors is well documented in the literature; therefore, I would like to see more discussion of socio-behavioral and physiological factors in the introduction, along with stronger connections to your focus on NMR.
2. The introduction should also highlight gaps in knowledge more explicitly.

Methods and Materials

• Please provide a rationale for choosing the Pennsylvania Adult Smoking Study 96 (PASS) conducted between June 2012 and April 2014, rather than using a more recent dataset (e.g., 2018, if available).

Study Procedure

• Line 166: The use of stress as a variable should have been given context earlier in the paper (Introduction).
• Similarly, the psychological distress variable lacks context. The statement in line 176 (“Psychological distress is higher in current smokers than in former and never smokers”) is not sufficient to justify the inclusion of this variable.
• The rationale for choosing specific measures and variables (e.g., stress) is not clearly explained in the introduction, which should set the stage for the significance of these factors in influencing smoking behaviors and nicotine dependence.
• The variable “Morning cravings or urges to smoke”: how was this defined and used in the study? This is unclear and also not contextualized in the introduction.
• Other personal or sociodemographic factors: what are these 19 variables? Consider including them in an appendix or supplementary table. It is important to specify these variables, as they likely interact with NMR associations with nicotine dependence. This context should also be introduced earlier in the paper.
• Line 192: What is TSNM? Has it been defined previously? Please spell it out.

Measures

• Nicotine dependence (ND) measured using two questions from the HIS (lines 139–142): please clarify how the categories (low, medium, high) were defined and calculated from the two items (TTFC and CPD).
• Under Table 1 (sample description), please define abbreviations such as AA, LT, N, Y at the bottom of the table, similar to how “H” is defined.

Statistical Analysis

This section is not clearly or cohesively presented. It is overly technical and should be simplified to better reach a broader audience. I recommend rewriting it in a stepwise approach, introducing each test with a brief explanation of its rationale and how it was applied. There are also typographical issues (e.g., line 250: consider deleting “R using”).

 

Results and Tables of Findings

• Formatting needs improvement to enhance clarity and readability.
• Tables 1 and 2 should be merged. The same results should not be presented in two separate tables; distributions by outcome variables and frequencies of sociodemographic characteristics should be combined into a single table.
• In the section titled “3.1 Dependence Measures – Bivariate Analysis,” it is important to report chi-square and t-test results and indicate whether there are differences in outcome measures based on sociodemographic characteristics. The current summary does not clearly present this.
• Table 4a: Confidence intervals (CIs) need to be reported.
• The results section would benefit from reorganization, with greater emphasis on the most clinically significant findings and the key contributions of the study. Currently, this is not clearly conveyed.

 

Discussion

The discussion section needs to be more elaborative, with stronger emphasis on implications for practice, public health, and future research directions.

Comments on the Quality of English Language

Needs editing and the formatting and organizing needs work and improvements.

Author Response

Please see the attached file.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript demonstrates a significant correlation between the severity of nicotine dependence in smokers and the nicotine metabolism ratio (i.e., the 3'-hydroxycotinine to cotinine ratio). Conversely, it proposes the hypothesis that the degree of nicotine dependence in smokers may be estimated using the nicotine metabolism ratio, which can be relatively easily measured from saliva. This hypothesis ultimately boils down to the question of whether there is a relationship between the enzymatic reactions involved in nicotine metabolism in the body and the factors that determine the severity of nicotine dependence. Nicotine is first metabolized to cotinine primarily by CYP2A6, and this cotinine is then oxidized again to trans-3'-hydroxycotinine by CYP2A6. Therefore, the 3'-hydroxycotinine/cotinine ratio is used as a metabolic ratio that serves as an indicator of CYP2A6 activity.

Therefore, the reviewer would like to ask the authors to explain the following: Is there any causal relationship between CYP2A6 activity and the severity of nicotine dependence (especially the assessment method used in this manuscript )?

Author Response

Please see the attached file.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The authors apply Random Forest (RF) regression and classification alongside parametric linear and logistic regression to rank the relative importance of the Nicotine Metabolite Ratio (NMR) and 18 sociobehavioural predictors against three nicotine-dependence scales: FTND, HONC, and HSI. Data come from the Pennsylvania Adult Smoking Study (n = 318). The central finding is that morning waking urge to smoke and perceived stress consistently outrank NMR, and that NMR's apparent importance depends on the modelling approach used.

The research question is timely and clinically relevant, the dataset is well characterized, and the multi-method comparison is a sensible design. However, the manuscript requires major revision before publication.

Abstract

1. The term "non-Nicotinic" in the title is technically inaccurate. "Nicotinic" refers to acetylcholine receptors; the authors mean non-NMR or non-nicotine-metabolism predictors.

Introduction

2. The Introduction provides adequate background on NMR biology and prior inconsistencies in the literature. The motivation for using machine learning, however, is underdeveloped. The authors should clarify what specific gap RF addresses that prior parametric work cannot, and should briefly justify the choice of RF over alternative tree-based methods such as gradient boosting or conditional inference forests. A clearer a priori hypothesis would also strengthen the framing.
3. Minor: "Scholl et all" (line 70) should read "Schnoll et al." and is misspelled relative to reference [18].

Methods

Several methodological issues warrant attention:

4. "Wake Urge" and "Awaken to Smoke" are closely related to the time-to-first-cigarette item used in both HSI and FTND, and "Work Smk" maps onto an FTND item. The high importance of these predictors may therefore be partially structural. Sensitivity analyses excluding constituent items should be presented.
5. Section 2.3.4 does not specify the item wording, response scale, or timing. Given that this variable drives the headline result, full definition is essential.
6. With n = 318 and 18 predictors, importance rankings are sensitive to the random seed and to the train/test split. Repeated splits or cross-validated importance with confidence intervals are needed.
7. Node-purity (Gini) importance is known to favor continuous and high-cardinality variables. Because log(NMR) is continuous, its high Gini ranking may be an artefact. Conditional permutation importance would be a more defensible primary metric.
8. KNN imputation is reasonable given low missingness (≤10.1%), but the value of k, the variables used as neighbors, and whether the outcome was excluded from imputation are not reported. These details should be added.
9. Across 18 predictors × 3 outcomes × 2 model families × 2 importance metrics, no correction is applied. Family-wise error should at least be discussed.
10. Section 2.3.1 defines HSI as three categories (0–1, 2–4, 5–6), but Table 4B uses a binary 0–3 versus 4–6 split. The discrepancy is not explained.
11. Cronbach's α = 0.61 is low. This caveat is acknowledged briefly but should be given more weight when interpreting FTND-based rankings.

Results

The bivariate description in Sections 3.1.1–3.1.3 largely reproduces Table 2 in narrative form and should be condensed; let the tables carry the numerical detail. Several specific concerns:

12. Table 3A/3B legends do not adequately define R1, R2, MDA, and MDG. The reader cannot reconstruct which column maps to which importance metric without inference.
13. No out-of-sample performance metrics (RMSE, AUC, accuracy) are reported for the RF models. Without these, it is unclear whether importance rankings reflect real signal or rank ordering of an underfit model.
14. Table 5 contains numerical inconsistencies with Tables 4A/4B. For example, the HONC log(NMR) coefficient in Table 5 (−0.35, p = 0.0157) does not match Table 4A (0.33, p = 0.0238). All summary figures should be reconciled.
15. The Wake Urge row in Table 2 lacks a p-value column entry, which is conspicuous given its centrality.
16. Typographical errors propagate through the Results. Line 320 contains a stray "257" mid-sentence; lines 322–323 and 351–353 confuse log(NMR) values with "age of starting smoking"; "out-of-bag" appears as "out of back" on line 387; "HHHC" (line 327) and "NMW" (line 430) appear to be typographical errors.

Discussion and Conclusion

The Discussion appropriately frames NMR as one of several factors rather than a sole determinant of dependence, and the call for broader precision-medicine criteria is well placed. However, the section would benefit from:

17. Direct engagement with the predictor–outcome overlap issue noted above, since it shapes the headline finding.
18. A more measured framing of the "machine learning" contribution. A single RF on a modest tabular dataset is a useful comparison rather than a methodological innovation.
19. Clearer guidance on what the findings imply for clinical translation. The Conclusion currently states that "precision therapy may need to be guided by other factors," but does not specify which.
20. Acknowledgement of the limited generalizability of a single regional sample (central Pennsylvania, 86.5% white).

Figures and Tables

All tables and the single figure should be made self-contained, with captions that allow independent reading. Current issues are summarised below.

21. Figure 1: Caption missing; only a title is given. The HONC column lists 7 items rather than the standard 10, and the scoring note "0–8 (lower (i.e. more dependence)" is incorrect — higher HONC denotes greater dependence, with a 0–10 range. Revise items, scoring text, and add a complete caption explaining the scales and their score interpretation.
22. Table 1: Caption is informative but does not state the imputation method or the denominator for percentages. Add a footnote naming the KNN procedure and clarifying "n (%)" denominators.
23. Table 2: Caption should specify the statistical tests used (t-test, chi-square) and the significance threshold. The Wake Urge row is missing p-values. Decimal places are inconsistent across rows.
24. Table 3A/3B: Legends do not define R1, R2, MDA, MDG, or how "Avg Rank" is computed. The caption should state the number of trees, mtry, node size, the train/test split, and whether values come from the training or out-of-bag sample.
25. Several numerical entries do not match the source tables (see Section 2.4). The caption should be revised to indicate that this is a synthesis of Tables 2–4, and the values reconciled with their sources.

Editorial and Minor Issues

• Title: "non-Nicotinic" should be "non-NMR" or "non-nicotine-metabolism."
• Line 70: "Scholl et all" → "Schnoll et al."
• Line 100: missing period after "counties."
• Line 320: stray "257" mid-sentence.
• Lines 322–323, 351–353: log(NMR) values described as "age of starting smoking"; reads as copy-paste error.
• Line 327: "HHHC" — likely HONC or another typographical error.
• Line 387: "out of back" should be "out-of-bag."
• Line 430: "NMW" should be "NMR."
• Line 456: "Machine learning language was used" should be "Machine learning was used."
• Capitalisation of "Random Forest" and "Machine Learning" is inconsistent throughout.
• Reference list formatting is uneven; align with IJERPH style.
• A careful copy-edit pass for typography and consistency is recommended before resubmission.

Author Response

Please see the attached file.

Author Response File: Author Response.docx

Reviewer 4 Report

Comments and Suggestions for Authors

Please see word doc for my comments.

Comments for author File: Comments.pdf

Author Response

Please see the attached file.

Author Response File: Author Response.pdf

Reviewer 5 Report

Comments and Suggestions for Authors

The authors investigated the contribution of the nicotine metabolite ratio (NMR) to the variance in nicotine dependence/addiction in 353 cigarette smokers (participants of the Pennsylvania Adult Smoking Study, PASS). Nicotine dependence/addiction was assessed by three established, questionnaire-based indices.  Not unexpectedly, it was found that the NMR can explain only part of nicotine dependency, other factors such as time to first cigarette after waking up in the morning and perceived stress rank higher.

I have a number of (mostly minor) points to be clarified in a revised version of the manuscript (see the attached annotated pdf file of the original paper). 

Comments for author File: Comments.pdf

Author Response

Please see the attached file.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have addressed concerns with the previous version of the manuscript. I have no more comments, and recommend that this manuscript is acceptable for publication in the IJERPH.