Computational Prediction of the Severity of Adverse Drug Reactions Caused by Drug–Drug Interactions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript "Computational Prediction of the Severity of Adverse Drug Reactions Caused by Drug-Drug Interactions" reports a structure-based computational approach for predicting the severity of drug–drug interaction (DDI)-associated adverse drug reactions (ADRs) using PoSMNA descriptors and PASS DDI models. The authors developed predictive models for 14 clinically important ADRs, classified into Major, Moderate, Minor, and None severity categories, and implemented the models in the publicly accessible AdverDDIPred web server. The study addresses an important problem in pharmacovigilance by attempting to predict DDI severity using only the chemical structures of drug pairs, without requiring biological or pharmacological annotations. The manuscript is generally well written and technically sound. However, some methodological issues require clarification before the manuscript can be considered for publication.

1. The current case study includes only a few well-known drug combinations, which are insufficient to demonstrate model generalizability. Thus, if possible, validate the models using an external DDI dataset from another independent source.
2. Although balanced accuracy is reported, the manuscript does not describe whether any techniques were used to reduce the impact of class imbalance, and needs to be updated for issues related to oversampling and undersampling. 
3. A comparison table summarizing predictive performance, input requirements, interpretability, and computational complexity would substantially strengthen the manuscript.

4. The Discussion briefly attributes this variation to mechanistic complexity but does not sufficiently analyze why structural descriptors perform well for some ADRs and poorly for others. A more detailed biological interpretation would improve the manuscript.

5. Confidence intervals are not reported for AUC, sensitivity, specificity, or balanced accuracy. Reporting 95% confidence intervals would improve statistical rigor.

6. Including additional examples representing both correctly and incorrectly predicted DDIs would provide a more balanced evaluation.

Author Response

The manuscript "Computational Prediction of the Severity of Adverse Drug Reactions Caused by Drug-Drug Interactions" reports a structure-based computational approach for predicting the severity of drug–drug interaction (DDI)-associated adverse drug reactions (ADRs) using PoSMNA descriptors and PASS DDI models. The authors developed predictive models for 14 clinically important ADRs, classified into Major, Moderate, Minor, and None severity categories, and implemented the models in the publicly accessible AdverDDIPred web server. The study addresses an important problem in pharmacovigilance by attempting to predict DDI severity using only the chemical structures of drug pairs, without requiring biological or pharmacological annotations. The manuscript is generally well written and technically sound. However, some methodological issues require clarification before the manuscript can be considered for publication.

 

1. The current case study includes only a few well-known drug combinations, which are insufficient to demonstrate model generalizability. Thus, if possible, validate the models using an external DDI dataset from another independent source.

 

Response: We thank the reviewer for this important suggestion. We agree that validating our models using an independent external dataset would provide a stronger assessment of their generalizability. However, we were unable to identify a publicly available independent dataset containing drug–drug interaction pairs annotated simultaneously by a specific adverse drug reaction and by the corresponding severity category used in our study (“Major”, “Moderate”, “Minor”, and “None”). Widely used resources, such as DrugBank and other DDI databases, contain information on known drug interactions and, in some cases, textual descriptions of their clinical consequences. However, they do not provide a systematic and directly compatible classification of DDI-associated adverse reactions according to the severity levels used for training our models. Consequently, these resources cannot be employed as an independent external test set without introducing additional manual annotation, subjective interpretation, or substantial incompatibility with the class definitions of the present study.

There is an additional clarification provided below.

The computational algorithms used in this study for the development of the SAR model have been extensively tested and validated across a wide range of applications, and they have been compared to alternative approaches (see, e.g., doi:10.1021/ci000383k; doi:10.1002/minf.202300262; doi:10.1007/s11172-019-2683-0). These studies demonstrate the high accuracy and strong predictive power of the PASS algorithms.

Leave-One-Out Cross-Validation is highly data-efficient method that provides an unbiased estimate of the in-sample generalization error. Compared to validation based on a single test set, cross-validation offers a more comprehensive and realistic assessment of model quality, and is therefore preferred (see, e.g., doi:10.1021/ci000383k; doi:10.1186/s13321-014-0047-1). The "compounds out" cross-validation algorithm used in this study is even more powerful.

 

2. Although balanced accuracy is reported, the manuscript does not describe whether any techniques were used to reduce the impact of class imbalance, and needs to be updated for issues related to oversampling and undersampling. 

 

Response: We thank the reviewer for this comment. No oversampling, undersampling, or synthetic-sample generation was used in the original PASS models. The PASS algorithm is designed to be insensitive to unbalanced training samples. Moreover, it is also weakly sensitive to incomplete data (doi:10.1021/ci000383k). We have clarified this in the Methods section.

Previous version: SAR analysis was performed using a special version of PASS DDI (Prediction of Activity Spectra for Substances with respect to Drug–Drug Interactions) [23,26]. In PASS DDI, classification models are constructed based on PoSMNA descriptors using an algorithm that is based on a modified naive Bayesian approach [27,28]. After training, PASS DDI calculates probability estimates for various types of biological activity, including ADRs associated with pairs of compounds, based on their structural formulas.

Revised version: SAR analysis was performed using a special version of PASS DDI (Prediction of Activity Spectra for Substances with respect to Drug–Drug Interactions) [30,31]. In PASS DDI, classification models are constructed based on PoSMNA descriptors using an algorithm that is based on a modified naive Bayesian approach [34,35]. After training, PASS DDI calculates probability estimates for various types of biological activity, including ADRs associated with pairs of compounds, based on their structural formulas. No explicit class-rebalancing procedures, including oversampling, undersampling, synthetic-sample generation, class weighting, or cost-sensitive learning, were applied during model development. All PASS DDI models were trained using the original class distributions. The modified naive Bayesian algorithm implemented in PASS has previously been shown to be relatively insensitive to unequal class sizes and incomplete activity annotations [34]. Balanced accuracy was used only as an evaluation metric that assigns equal importance to sensitivity and specificity; it was not used to modify the training samples or to optimize the classification threshold.

 

3. A comparison table summarizing predictive performance, input requirements, interpretability, and computational complexity would substantially strengthen the manuscript.

 

Response: We thank the reviewer for this suggestion. We have added a comparison table summarizing the required input data, molecular representations, classification approaches, class-imbalance handling, interpretability, and relative computational complexity of PASS DDI/PoSMNA and the evaluated machine-learning baselines. Predictive performance of these methods on the bleeding dataset is presented separately in Figure 2.

Previous version: No conventional structure-based machine-learning benchmark was included in the original manuscript.

Revised version: To provide a conventional machine-learning reference for the proposed approach, a separate benchmark was performed for the bleeding endpoint. The benchmark included a prevalence-based dummy classifier, logistic regression, random forest, and XGBoost trained using Morgan and RDKit molecular fingerprints. Because of the substantial computational cost of evaluating all combinations of algorithms, molecular representations, severity categories, and validation schemes across all 14 ADR-specific datasets, this comparison was restricted to bleeding as a representative endpoint.
Table 3. Comparison of PASS DDI/PoSMNA and conventional structure-based machine-learning methods used in the bleeding benchmark.

 

 

 

Table 3. Comparison of PASS DDI/PoSMNA and conventional structure-based machine-learning methods used in the bleeding benchmark.

Method	Input data	Molecular representation	Classification approach	Class-imbalance handling	Interpretability	Relative computational complexity
PASS DDI	2D structures of both drugs	PoSMNA	Modified naive Bayesian approach	No explicit resampling or class weighting	Pa/Pi estimates; pair-specific structural descriptors	Moderate
Dummy	Class labels	None	Prevalence-based prediction	None	High	Very low
Logistic regression	2D structures of both drugs	Morgan / RDKit fingerprints	Linear binary classifier	Balanced class weights	Moderate	Low
Random forest	2D structures of both drugs	Morgan / RDKit fingerprints	Tree ensemble	Balanced class weights	Moderate	Moderate
XGBoost	2D structures of both drugs	Morgan / RDKit fingerprints	Gradient-boosted trees	scale_pos_weight	Moderate	Moderate–high

 

 

4. The Discussion briefly attributes this variation to mechanistic complexity but does not sufficiently analyze why structural descriptors perform well for some ADRs and poorly for others. A more detailed biological interpretation would improve the manuscript.

 

Response: We agree with the reviewer's comment and have expanded the Discussion using DILI as an example to provide a more detailed mechanistic interpretation of the observed variation in predictive performance across ADR types. In particular, we discuss DILI as an example of a complex multifactorial adverse reaction. Structural descriptors are expected to be more informative when toxicity is driven by intrinsic chemical liabilities, such as structural alerts, reactive metabolite formation, or direct membrane and mitochondrial damage. In contrast, hepatotoxicity is also strongly influenced by hepatic metabolism, transporter-mediated exposure, immune susceptibility, and dose-dependent bioactivation, which are not explicitly represented by molecular structure alone. This mechanistic heterogeneity may therefore explain why hepatotoxicity remained comparatively difficult to predict, despite the relatively large training datasets.

Previous version: The obtained results show that the proposed approach can be used to predict the severity of DDI-associated ADRs; however, predictive accuracy depends substantially on the type of effect considered and on the quality of the source data. AUC values varied widely, approximately from 0.620 (Bradycardia, Major) to 0.996 (Cardiac depression, Moderate) depending on the ADR category and severity class, indicating that task complexity differs across ADR types. Notably, a complex ADR type such as hepatotoxicity was characterized by lower predictive accuracy even when larger training datasets were available.

Revised version: The obtained results show that the proposed approach can be used to predict the severity of DDI-associated ADRs; however, predictive accuracy depends substantially on the type of effect considered and on the quality of the source data. AUC values varied widely, approximately from 0.620 (Bradycardia, Major) to 0.996 (Cardiac depression, Moderate) depending on the ADR category and severity class, indicating that task complexity differs across ADR types. Notably, a complex ADR type such as hepatotoxicity was characterized by lower predictive accuracy even when larger training datasets were available. The variable performance observed for hepatotoxicity may reflect the heterogeneous mechanistic basis of drug-induced liver injury (DILI). Structure-based descriptors are expected to be more informative when hepatotoxicity is associated with intrinsic chemical liabilities, such as structural alerts, formation of reactive metabolites, or direct membrane and mitochondrial damage. However, DILI is also strongly influenced by biological context, including hepatic metabolism, transporter-mediated exposure, immune susceptibility, and dose-dependent bioactivation. These factors are not represented explicitly by molecular structure alone. Consequently, structural descriptors may capture only part of the mechanistic basis of DILI, which may explain why predictive performance for hepatotoxicity remained relatively modest despite the comparatively large number of training examples.

 

5. Confidence intervals are not reported for AUC, sensitivity, specificity, or balanced accuracy. Reporting 95% confidence intervals would improve statistical rigor.

 

Response: We thank the reviewer for this suggestion. In the revised manuscript, we have added 95% confidence intervals for AUC values to Table 1. Confidence intervals for sensitivity, specificity, and balanced accuracy could not be obtained directly from the PASS DDI validation procedure, which provides confidence-interval estimates for ROC AUC but reports the remaining performance measures as point estimates. We therefore report 95% confidence intervals for AUC, while sensitivity, specificity, and balanced accuracy are retained as point estimates. This distinction has been explicitly indicated in the legend to Table 1.

Previous version: Table 1. Performance metrics for predicting the severity of adverse drug reactions caused by paired drug combinations.
ADR: adverse drug reaction; Class: DDI hazard class; N: number of drug pairs belonging to the class in the dataset; AUC: area under the receiver operating characteristic curve; Sen.: sensitivity; Spec.: specificity; BA: balanced accuracy.

Revised version: Table 1. Predictive performance of PASS DDI models for severity classification of adverse drug reactions associated with drug–drug interactions.
ADR: adverse drug reaction; Class: DDI severity class; N: number of drug pairs belonging to the corresponding class in the dataset; AUC: area under the receiver operating characteristic curve obtained using compounds-out cross-validation; CI: 95% confidence interval for AUC; Sen.: sensitivity; Spec.: specificity; BA: balanced accuracy; AUC5-fold: area under the receiver operating characteristic curve obtained using conventional pair-wise stratified five-fold cross-validation. Classification was performed using the predefined PASS criterion Pa > Pi.

 

6. Including additional examples representing both correctly and incorrectly predicted DDIs would provide a more balanced evaluation.

 

Response: We thank the reviewer for this suggestion. We agree that presenting only successful predictions could provide an overly favorable impression of model performance. We therefore revised the case-based assessment to include both predictions that were consistent with published clinical evidence and an unsuccessful severity prediction. In particular, the warfarin–amiodarone interaction is now explicitly reported as a false-negative prediction for the Major severity category: although the model predicted the Minor and Moderate bleeding categories, it underestimated the clinically relevant severity of the interaction. The corresponding table and discussion have been revised accordingly, and we now explicitly state that these case-based examples are illustrative and should not be interpreted as an independent validation dataset.

Previous version: As shown in Table 3, the results of the case-based assessment were generally consistent with published clinical data on DDI-associated ADRs. Candidate drug combinations for this assessment were identified from published clinical reports and DDI data. For the final case-based assessment, only those drug pairs that were absent from the training datasets used to build the corresponding predictive models were selected. For the combination of warfarin and amiodarone, the Major class did not receive the highest probability estimate (Pa/Pi = 0.090/0.691); however, higher Pa values were observed for the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) classes. This interpretation is consistent with published data indicating that this combination increases the risk of bleeding primarily by enhancing the anticoagulant effect of warfarin and requires dose adjustment and monitoring of anticoagulant intensity [13]. Thus, for this drug pair, the prediction reflects not an inevitable major bleeding event, but rather a clinically significant and potentially manageable bleeding risk.

Revised version: The case-based assessment included examples of predictions that were consistent with published clinical evidence as well as examples illustrating limitations of the model. The individual cases are discussed below. For the combination of warfarin and amiodarone, the Major class was not predicted (Pa/Pi = 0.090/0.691) and should therefore be considered a false-negative prediction for the Major severity category. At the same time, the model predicted both the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) categories. Thus, although the model identified an association of this drug pair with bleeding, it underestimated its severity [20]. This example illustrates an important limitation of the structure-based approach: clinically significant severity may depend on pharmacokinetic interactions, dose, monitoring, and patient-specific factors that are not represented by molecular structure alone. Accordingly, the case-based assessment should be regarded as illustrative rather than as an independent validation of the model.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript addresses an important problem in drug safety: estimating how severe adverse reactions may be when they arise from drug-drug interactions. This is a useful shift from simply predicting whether an interaction exists, since clinicians often need to judge which interactions deserve the most attention. The study covers 14 adverse reaction types, which gives the work a broad scope. The structure-based PoSMNA descriptor approach is also useful, especially because it can be applied when only chemical structures are available. The compounds-out cross-validation strategy is appropriate and helps reduce the risk of information leakage. The authors are also careful not to overstate the results, particularly when discussing weaker performance for more complex outcomes such as hepatotoxicity. Overall, the study is well designed, and the main limitations seem to reflect the difficulty of the prediction task rather than major flaws in the work.

Major Concerns

1. The models rely only on chemical structure, so they cannot capture several factors that may strongly affect ADR severity. These include pharmacokinetics, metabolism, dose, patient characteristics, and the clinical context in which the drugs are used together. This matters most for complex outcomes such as hepatotoxicity, where metabolism and individual susceptibility often play a large role. The manuscript describes the tool as a preliminary screening method rather than a clinical decision system, which is appropriate. The manuscript would benefit from making this limitation more explicit. The authors could also discuss whether future versions might include basic clinical or pharmacological variables, such as age, renal or hepatic function, dose information, or metabolic pathway data.
2. The handling of negative examples needs clearer explanation. In this study, a negative example seems to mean that no association was recorded, rather than that the interaction or ADR was confirmed to be absent. This could introduce mislabeled cases, since some true ADRs may be unreported, understudied, or undetected in the available data. The manuscript explains why this approach is necessary, but readers would benefit from clearer guidance on interpreting negative predictions. It would also help to discuss possible ways to reduce this uncertainty, such as uncertainty estimates, label-noise-robust methods, or expert review of selected negative examples.
3. Model performance varies across ADR types and severity classes. The lower performance for some major reactions suggests that sample size is not the only challenge. These outcomes may also be harder to predict because they involve more varied clinical presentations, less consistent annotation, and more complex biological mechanisms. Hepatotoxicity is a useful example, since it remains difficult to predict even with relatively large datasets. The authors should expand this discussion and explain which ADR mechanisms are more suitable for structure-based prediction and which are less suitable. They could also comment on whether ADR-specific models, improved class balancing, or added mechanistic information might help for the more difficult categories.

Minor Concern

1. Some figure and table captions should be more informative. As written, readers may need to return to the main text to understand the main point of a figure or table. Expanding the captions would make the manuscript easier to follow, especially for readers who look at the figures and tables first.

 

Author Response

This manuscript addresses an important problem in drug safety: estimating how severe adverse reactions may be when they arise from drug-drug interactions. This is a useful shift from simply predicting whether an interaction exists, since clinicians often need to judge which interactions deserve the most attention. The study covers 14 adverse reaction types, which gives the work a broad scope. The structure-based PoSMNA descriptor approach is also useful, especially because it can be applied when only chemical structures are available. The compounds-out cross-validation strategy is appropriate and helps reduce the risk of information leakage. The authors are also careful not to overstate the results, particularly when discussing weaker performance for more complex outcomes such as hepatotoxicity. Overall, the study is well designed, and the main limitations seem to reflect the difficulty of the prediction task rather than major flaws in the work.

Major Concerns:

 

1. The models rely only on chemical structure, so they cannot capture several factors that may strongly affect ADR severity. These include pharmacokinetics, metabolism, dose, patient characteristics, and the clinical context in which the drugs are used together. This matters most for complex outcomes such as hepatotoxicity, where metabolism and individual susceptibility often play a large role. The manuscript describes the tool as a preliminary screening method rather than a clinical decision system, which is appropriate. The manuscript would benefit from making this limitation more explicit. The authors could also discuss whether future versions might include basic clinical or pharmacological variables, such as age, renal or hepatic function, dose information, or metabolic pathway data.

 

Response: We thank the reviewer for this important comment. We agree that pharmacokinetic, pharmacological, and patient-specific information could improve prediction of ADR severity, particularly for complex and multifactorial adverse reactions in some cases like Jin et al., “Deep learning identifies synergistic drug combinations for treating COVID-19” study. At the same time, the present study intentionally addresses a strictly structure-based formulation of the problem. Many contemporary machine-learning approaches can benefit from pretrained molecular representations or from integrating additional biomedical information. However, such approaches may depend on auxiliary data that are incomplete or unavailable for less extensively studied compounds. In contrast, PoSMNA descriptors are generated directly from the structural formulas of the two compounds and do not require target profiles, metabolic pathway annotations, protein-interaction data, or other compound-specific biomedical information.

We therefore consider the limited input requirements of the proposed approach to be one of its practical advantages rather than solely a limitation. Nevertheless, we agree that structural information alone cannot capture all determinants of ADR severity. Future extensions could complement the present structural representation with pharmacokinetic, pharmacological, or clinical variables when sufficiently comprehensive and standardized datasets become available.

 

2. The handling of negative examples needs clearer explanation. In this study, a negative example seems to mean that no association was recorded, rather than that the interaction or ADR was confirmed to be absent. This could introduce mislabeled cases, since some true ADRs may be unreported, understudied, or undetected in the available data. The manuscript explains why this approach is necessary, but readers would benefit from clearer guidance on interpreting negative predictions. It would also help to discuss possible ways to reduce this uncertainty, such as uncertainty estimates, label-noise-robust methods, or expert review of selected negative examples.

 

Response: We thank the reviewer for this comment. We agree that “None” is a conditionally negative class: it indicates that no corresponding ADR association was recorded in the source data, rather than confirming its true absence. Therefore, we have clarified that a “None” prediction should be interpreted as insufficient model evidence for the ADR, and not as proof of safety of the drug combination. To assess the sensitivity of the model to the "None" class, we performed repeated random undersampling of “None” while retaining all “non-None” examples. Across the analyzed splits, the standard deviation of AUC was 1.14 percentage points, indicating limited sensitivity to random negative selection. However, this analysis cannot exclude systematic label noise caused by unreported ADRs. We added this limitation and identified positive–unlabeled learning, label-noise-robust methods, uncertainty calibration, and expert review of selected negative examples as directions for future work.

 

3. Model performance varies across ADR types and severity classes. The lower performance for some major reactions suggests that sample size is not the only challenge. These outcomes may also be harder to predict because they involve more varied clinical presentations, less consistent annotation, and more complex biological mechanisms. Hepatotoxicity is a useful example, since it remains difficult to predict even with relatively large datasets. The authors should expand this discussion and explain which ADR mechanisms are more suitable for structure-based prediction and which are less suitable. They could also comment on whether ADR-specific models, improved class balancing, or added mechanistic information might help for the more difficult categories.

 

Response: We thank the reviewer for this helpful suggestion. We agree that the observed variation in performance likely reflects not only sample size but also mechanistic heterogeneity across ADR types. Some ADRs are more directly linked to intrinsic chemical structure, for example through structural alerts, off-target binding, or simple physicochemical liabilities, and are therefore more amenable to structure-based prediction. In contrast, other ADRs arise from more complex biological cascades that are only partially encoded by molecular structure alone. DILI is a clear example: although certain structural features may indicate hepatotoxic liability, DILI is often driven by metabolic activation, formation of reactive metabolites, mitochondrial dysfunction, transporter effects, and immune-mediated injury, making it substantially harder to predict from structure alone. We have expanded the Discussion to explain this distinction more explicitly. We also note that ADR-specific models, better class balancing, and the inclusion of mechanistic or biological features may improve prediction for these more complex categories.

Previous version: The obtained results show that the proposed approach can be used to predict the severity of DDI-associated ADRs; however, predictive accuracy depends substantially on the type of effect considered and on the quality of the source data. AUC values varied widely, approximately from 0.620 (Bradycardia, Major) to 0.996 (Cardiac depression, Moderate) depending on the ADR category and severity class, indicating that task complexity differs across ADR types. Notably, a complex ADR type such as hepatotoxicity was characterized by lower predictive accuracy even when larger training datasets were available.

Revised version: The obtained results show that the proposed approach can be used to predict the severity of DDI-associated ADRs; however, predictive accuracy depends substantially on the type of effect considered and on the quality of the source data. AUC values varied widely, approximately from 0.620 (Bradycardia, Major) to 0.996 (Cardiac depression, Moderate) depending on the ADR category and severity class, indicating that task complexity differs across ADR types. Notably, a complex ADR type such as hepatotoxicity was characterized by lower predictive accuracy even when larger training datasets were available. The variable performance observed for hepatotoxicity may reflect the heterogeneous mechanistic basis of drug-induced liver injury (DILI). Structure-based descriptors are expected to be more informative when hepatotoxicity is associated with intrinsic chemical liabilities, such as structural alerts, formation of reactive metabolites, or direct membrane and mitochondrial damage. However, DILI is also strongly influenced by biological context, including hepatic metabolism, transporter-mediated exposure, immune susceptibility, and dose-dependent bioactivation. These factors are not represented explicitly by molecular structure alone. Consequently, structural descriptors may capture only part of the mechanistic basis of DILI, which may explain why predictive performance for hepatotoxicity remained relatively modest despite the comparatively large number of training examples.

 

Minor Concern

1. Some figure and table captions should be more informative. As written, readers may need to return to the main text to understand the main point of a figure or table. Expanding the captions would make the manuscript easier to follow, especially for readers who look at the figures and tables first.

 

Response: We thank the reviewer for this suggestion. We have revised the figure and table captions to make them more informative and self-contained. In particular, the revised captions now specify the evaluated performance metrics and severity categories, clarify the interpretation of the cross-validation schemes and graphical elements, and provide additional information required to interpret the presented results without referring extensively to the main text. The captions of the case-based assessment and methodological figures were also expanded to clarify the interpretation of Pa/Pi estimates and the purpose of the presented examples.

Previous version: Figure 1. The distribution of predictive performance values for 14 ADRs is presented by severity class. The Minor class is not included in the diagram because it contains only two values.

Revised version: Figure 1. Distribution of ROC AUC values across ADR-specific models by severity class. Values are shown for the Major, Moderate, and None severity categories across the 14 evaluated ADR types. The Minor category is not shown because it was represented in only two ADR-specific datasets.

Previous version: Figure 2. Web interface of the models on the Way2Drug website.

Revised version: Figure 3. Web interface of the AdverDDIPred application implemented on the Way2Drug platform. Users can submit a drug pair using compound names, SMILES strings, or molecular structures and obtain Pa and Pi estimates for DDI-associated ADR severity categories satisfying the predefined PASS criterion Pa > Pi.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

Firstly, I would like to appologize to authors and Editor for submitting my review a bit late. I was just trying to be as detailed as possible.
The manuscript addresses a relevant and timely problem: prediction of DDI-associated ADR severity from chemical structure alone and the PoSMNA/PASS DDI approach, the "compounds-out" cross-validation scheme, and the accompanying AdverDDIPred web application are genuine strengths. The study has scientific merit. However, several issues, some of them substantive, should be addressed before the manuscript can be considered for publication.

Comments and suggestions:

1. Sensitivity and specificity are near-identical across nearly all rows, suggesting a per-class operating point. In Table 1, Sen. and Spec. coincide to three decimals in the overwhelming majority of entries (e.g. hypotension/None 0.833/0.833; nephrotoxicity/None 0.875/0.876; immunosuppression/Major 0.698/0.698). For independently computed metrics on imbalanced classes this is statistically implausible unless the decision threshold is chosen per class to equalize the two (e.g. a Youden-type or equal-error operating point). If that is the case, balanced accuracy is reported at an optimized threshold and is optimistically biased. The authors should state explicitly how the threshold is set for each class and severity level, whether it is fixed a priori or tuned, and if tuned report performance at a fixed threshold as well, or provide threshold-independent summaries alongside AUC.
2. No comparison with any baseline or existing method. The proposed models are not benchmarked against representative machine-learning or deep-learning approaches to DDI/ADR prediction, nor against trivial baselines (majority-class, random, or single-drug feature models). Without at least one point of reference, the practical advantage and novelty of the approach cannot be assessed. A comparison against structure-based ML baselines on the same datasets is essential.
3. The central methodological claim that naive cross-validation inflates performance is asserted but not demonstrated in this work. The "compounds-out" scheme is presented as a key contribution, with information leakage under k-fold or leave-one-pair-out validation cited as the motivation (referencing prior work [21]). Since this is framed as a strength of the present study, the authors should quantify it here: report performance under both naive and compounds-out validation on these datasets so the magnitude of the leakage effect is visible. This is a low-cost, high-impact experiment.
4. External validation is absent. Performance is evaluated only via the proposed internal cross-validation and a small set of illustrative cases. Validation on an independent external dataset (e.g. a held-out source not derived from DrugMAP/TwoSides) would substantially strengthen the claims of robustness and generalizability.
5. The "None" class is conditionally negative, and the consequences for the reported metrics are under-analyzed. The authors acknowledge that absence of a recorded association does not establish true absence of an ADR, and that false negatives may contaminate the None class. This is discussed qualitatively but its quantitative impact is not addressed. Given that None is by far the largest class in every dataset (tens of thousands of pairs vs. hundreds for Major), even a small mislabeling rate could materially shift specificity and AUC. The authors should discuss the plausible direction and magnitude of this effect, and ideally probe sensitivity of the metrics to the negative-selection thresholds.
6. The case-based assessment (Table 3) is interpreted selectively. For warfarin+amiodarone the Major class does not receive the highest Pa (0.090/0.691; Pa < Pi), yet the text frames this as a correct prediction of "manageable bleeding risk." This reads as a post-hoc rationalization of a case the model arguably misclassified, particularly since warfarin+amiodarone is clinically a serious, closely monitored interaction. The authors should either apply a consistent decision rule across all cases in Table 3 or explicitly acknowledge the miss rather than reframing it as success. As anecdotal illustrations, these cases should also not be over-interpreted as validation.
7. Mean-by-class metrics in Table 2 average over structurally non-comparable tasks. The number of severity classes varies across ADRs — some datasets include a Minor class, some do not; hyperglycemia and hypokalemia lack a Major class entirely. Averaging performance across ADRs with differing class structures makes the aggregate figures difficult to interpret. The authors should note this heterogeneity and consider whether a weighted or stratified summary is more informative.
8. Class imbalance is discussed only qualitatively. The manuscript notes the imbalance and its likely effect on the Major class, but reports no mitigation strategy (resampling, class weighting, cost-sensitive learning) and no quantitative analysis of its influence. A brief treatment of what, if anything, was done — and what effect it had — would improve methodological rigor.
9. An error analysis is missing. Representative false-positive and false-negative predictions, ideally linked to structural or mechanistic features, would give useful insight into where and why the models fail — especially for the poorly predicted ADR types (bradycardia/Major AUC ≈ 0.62; lowering seizure threshold ≈ 0.65).
10. Claims of clinical applicability should be moderated though the authors are already fairly cautious. The Discussion and Conclusions already state repeatedly that this is a preliminary screening tool and that predicted severity is not equivalent to patient outcome. The recommendation is to ensure this framing is consistent throughout (including the Abstract) rather than to add new caveats, since the necessary qualifications are largely present.
11. The Discussion is partly interpretive but leans on restating Results. It does add mechanistic interpretation (e.g. hepatotoxicity, vascular-tone effects), but a more critical placement of the findings relative to the specific prior studies cited in the Introduction — rather than summary of the tables — would strengthen it.

Overall, the study is a worthwhile contribution to computational pharmacology. Comments 1–3 in particular should be addressed before the manuscript is suitable for publication; the remaining points would improve rigor and interpretability.

Good luck!

Author Response

Firstly, I would like to appologize to authors and Editor for submitting my review a bit late. I was just trying to be as detailed as possible.

The manuscript addresses a relevant and timely problem: prediction of DDI-associated ADR severity from chemical structure alone and the PoSMNA/PASS DDI approach, the "compounds-out" cross-validation scheme, and the accompanying AdverDDIPred web application are genuine strengths. The study has scientific merit. However, several issues, some of them substantive, should be addressed before the manuscript can be considered for publication.

 

Comments and suggestions:

1. Sensitivity and specificity are near-identical across nearly all rows, suggesting a per-class operating point. In Table 1, Sen. and Spec. coincide to three decimals in the overwhelming majority of entries (e.g. hypotension/None 0.833/0.833; nephrotoxicity/None 0.875/0.876; immunosuppression/Major 0.698/0.698). For independently computed metrics on imbalanced classes this is statistically implausible unless the decision threshold is chosen per class to equalize the two (e.g. a Youden-type or equal-error operating point). If that is the case, balanced accuracy is reported at an optimized threshold and is optimistically biased. The authors should state explicitly how the threshold is set for each class and severity level, whether it is fixed a priori or tuned, and if tuned report performance at a fixed threshold as well, or provide threshold-independent summaries alongside AUC.

 

Response: We thank the reviewer for drawing attention to this point. The reviewer is correct that the close agreement between sensitivity and specificity is a consequence of the operating rule used by PASS rather than an incidental result. PASS does not classify an example using a conventional fixed posterior-probability threshold such as Pa≥0.5. For every ADR and severity category, the algorithm calculates two estimates, Pa and Pi, which characterize membership in the subsets of active and inactive examples, respectively. These estimates are derived from activity-specific distributions of the primary B statistic. The distributions are constructed during model training by leave-one-out calculations performed on the training set. In PASS Pa and Pi are also interpreted as estimates of two types of prediction errors. Additionally, 1-Pa equals Sensitivity, and 1-Pi equals Specificity. The predefined PASS decision rule is: Pa>Pi. This rule was applied uniformly to all ADR types and severity categories, and was not adjusted separately to maximize sensitivity, specificity, Youden’s index, or balanced accuracy. The same rule is also used in the AdverDDIPred implementation. Because the decision boundary corresponds to the point at which the two error estimates are equal, Pa=Pi, it represents an approximately equal-error operating point. Consequently, sensitivity and specificity are expected to be similar, although they need not be numerically identical because of finite sample sizes and the modified compound-disjoint cross-validation procedure. Thus, the similarity between sensitivity and specificity in Table 1 results from the intrinsic construction of the PASS decision rule rather than from post hoc optimization using the test data. Importantly, the distributions of Pa and Pi, and the corresponding boundary Pa=Pi, were estimated exclusively using the training data at each cross-validation step. Test-set labels were not used either to construct these distributions or to select an operating threshold. Therefore, balanced accuracy was not optimized on the test set. Nevertheless, we agree that the original manuscript did not explain this feature of the PASS algorithm sufficiently clearly and could give the impression that a separate threshold had been selected for every row of Table 1. We have revised the Methods section and the legend to Table 1 to state explicitly that:

1. the decision criterion was fixed a priori as Pa>Pi;

2. the same criterion was used for all ADRs and severity classes;

3. the activity-specific Pa and Pi mappings were derived only from the corresponding training data;

4. no threshold was selected using the validation results; and

5. ROC AUC is additionally reported as a threshold-independent measure of discrimination.

 

PASS DDI does not use a conventional posterior-probability threshold, such as Pa≥0.5. For each ADR and severity category, the algorithm calculates Pa and Pi, which estimate the membership of a drug pair in the active and inactive classes, respectively. By construction, Pa and Pi can also be interpreted in relation to the two types of prediction error, whereas 1−Pa and 1−Pi provide estimates of sensitivity and specificity, respectively. The predefined PASS classification criterion Pa>Pi was used for all ADRs and severity classes. The boundary Pa=Pi therefore corresponds to an approximately equal-error operating point. The activity-specific mappings of the internal B-statistic to Pa and Pi were constructed exclusively from the training data within each cross-validation iteration. Neither test-set labels nor validation metrics were used to select or optimize the decision threshold. Thus, balanced accuracy was not optimized post hoc on the test data. ROC AUC was additionally reported as a threshold-independent measure of discrimination.

Previous version: In the present work, for a new pair of structures, PASS DDI calculates two probability estimates: Pa, which is the probability that a drug pair will cause a specific ADR of a given severity, and Pi, which is the probability that the drug pair will not cause an ADR of that severity. The greater the difference Pa − Pi, the more likely the drug pair is to cause the ADR of the corresponding severity according to the available dataset.

Revised version: In the present work, for a new pair of structures, PASS DDI calculates two probability estimates: Pa, which is the probability that a drug pair will cause a specific ADR of a given severity, and Pi, which is the probability that the drug pair will not cause an ADR of that severity. The greater the difference Pa − Pi, the more likely the drug pair is to cause the ADR of the corresponding severity according to the available dataset. PASS DDI does not use a conventional posterior-probability threshold such as Pa ≥ 0.5. For each ADR and severity category, the algorithm calculates Pa and Pi estimates from activity-specific distributions of the internal B-statistic generated using the training data. Pa reflects the estimated likelihood that a drug pair belongs to the corresponding activity class, whereas Pi reflects the estimated likelihood of inactivity with respect to that class. Classification was performed using the predefined PASS criterion Pa > Pi, which was applied uniformly to all ADRs and severity categories. The activity-specific mappings of the B-statistic to Pa and Pi were derived exclusively from the training data within each cross-validation iteration. Neither test-set labels nor validation metrics were used to select or optimize the decision criterion. Thus, balanced accuracy was not optimized post hoc on the validation data. ROC AUC was additionally reported as a threshold-independent measure of discrimination.

 

2. No comparison with any baseline or existing method. The proposed models are not benchmarked against representative machine-learning or deep-learning approaches to DDI/ADR prediction, nor against trivial baselines (majority-class, random, or single-drug feature models). Without at least one point of reference, the practical advantage and novelty of the approach cannot be assessed. A comparison against structure-based ML baselines on the same datasets is essential.

 

Response: We thank the reviewer for this important comment. We agree that the original manuscript lacked an adequate comparison with baseline machine-learning methods, which limited the interpretation of the practical performance of the proposed approach. We were unable to identify a publicly available method that predicts ADR-specific severity categories from paired chemical structures using the same endpoints and class definitions, and that could therefore be directly applied to our datasets without substantial modification. We consequently added a benchmark against several widely used structure-based machine-learning approaches trained and evaluated on exactly the same drug-pair datasets. The comparison included a prevalence-based dummy classifier as a trivial baseline, logistic regression, random forest, and XGBoost, together with Morgan and RDKit molecular fingerprints and alternative methods for combining the representations of the two drugs, including fingerprint concatenation.

Previous version: No conventional structure-based machine-learning benchmark was included in the original manuscript.

Revised version: To provide a conventional machine-learning reference for the proposed approach, a separate benchmark was performed for the bleeding endpoint. The benchmark included a prevalence-based dummy classifier, logistic regression, random forest, and XGBoost trained using Morgan and RDKit molecular fingerprints. Because of the substantial computational cost of evaluating all combinations of algorithms, molecular representations, severity categories, and validation schemes across all 14 ADR-specific datasets, this comparison was restricted to bleeding as a representative endpoint.
Table 3. Comparison of PASS DDI/PoSMNA and conventional structure-based machine-learning methods used in the bleeding benchmark.

 

 

 

3. The central methodological claim that naive cross-validation inflates performance is asserted but not demonstrated in this work. The "compounds-out" scheme is presented as a key contribution, with information leakage under k-fold or leave-one-pair-out validation cited as the motivation (referencing prior work [21]). Since this is framed as a strength of the present study, the authors should quantify it here: report performance under both naive and compounds-out validation on these datasets so the magnitude of the leakage effect is visible. This is a low-cost, high-impact experiment.

 

Response: We thank the reviewer for this important comment. In our study, we paid particular attention to the cross-validation procedure. All methods were evaluated using a compound-disjoint cross-validation, in which no individual drug occurring in the test set was allowed to occur in the training set as part of another pair. The results of this approach are presented in Figure 2. Under this strict validation scheme, our proposed PASS DDI/PoSMNA approach showed competitive and, for the evaluated endpoint, superior predictive performance compared to the structure-based machine learning baselines we tested. For comparison, we also evaluated these same baseline models using conventional pair-wise stratified k-fold cross-validation. This procedure produced near-perfect AUC values, substantially higher than those obtained using compound-disjoint validation. For example, for the Major bleeding category, conventional stratified cross-validation yielded AUC values of 0.981–0.997, whereas compound-disjoint validation yielded values of 0.649–0.781. The consistency of this difference across classifiers and molecular fingerprints indicates that conventional splitting causes substantial information leakage because the same individual drugs occur in both the training and test sets. The resulting performance estimates are, therefore, overly optimistic. We have revised the Methods, Results, and Discussion sections to describe the baseline models, molecular descriptors, class-imbalance handling, and validation schemes. We also emphasize that meaningful comparison of DDI prediction methods requires a compound-level separation between training and test sets, rather than the conventional splitting of individual drug pairs.

Previous version: When constructing the training and testing datasets, we accounted for the specific structure of the data: it consists of not individual compounds, but compound pairs. Therefore, the most widely used cross-validation methods, such as k-fold splitting or leave-one-drug-pair-out validation, could result in information leakage between the training and testing sets and, consequently, unjustifiably overestimated predictive accuracy. Using such an inappropriate validation scheme, a model may rely on information about individual drugs that are frequently associated with a specific effect, rather than learning SAR patterns specific to drug pairs.

Revised version: The validation strategy had a pronounced effect on the estimated predictive performance. For the Major bleeding category, conventional pairwise-stratified cross-validation yielded ROC AUC values ranging from 0.981 to 0.997 across the evaluated classifiers and fingerprint representations. Under drug-disjoint cross-validation, the corresponding ROC AUC values ranged from 0.649 to 0.781. Thus, conventional pairwisepair-wise splitting produced substantially more optimistic estimates than compound-level separation. The same comparison was performed for all four bleeding severity categories (Major, Moderate, Minor, and None), as summarized in Figure 2. Across the evaluated configurations, the magnitude of the difference between validation schemes varied between severity categories and classifiers, but pairwise-stratified cross-validation generally produced more optimistic performance estimates than drug-disjoint validation. This observation demonstrates that compound-level overlap can substantially affect apparent model performance in drug-pair prediction tasks.

 

 

4. External validation is absent. Performance is evaluated only via the proposed internal cross-validation and a small set of illustrative cases. Validation on an independent external dataset (e.g. a held-out source not derived from DrugMAP/TwoSides) would substantially strengthen the claims of robustness and generalizability.

 

Response: We thank the reviewer for this important comment. We agree that validating our models using an independent external dataset would provide a stronger assessment of their generalizability. However, we were unable to identify a publicly available independent dataset containing drug–drug interaction pairs annotated simultaneously by a specific adverse drug reaction and by the corresponding severity category used in our study (“Major”, “Moderate”, “Minor”, and “None”). Widely used resources, such as DrugBank and other DDI databases, contain information on known drug interactions and, in some cases, textual descriptions of their clinical consequences. However, they do not provide a systematic and directly compatible classification of DDI-associated adverse reactions according to the severity levels used for training our models. Consequently, these resources cannot be employed as an independent external test set without introducing additional manual annotation, subjective interpretation, or substantial incompatibility with the class definitions of the present study.

There is an additional clarification provided below.

The computational algorithms used in this study for the development of the SAR model have been extensively tested and validated across a wide range of applications, and they have been compared to alternative approaches (see, e.g., doi:10.1021/ci000383k; doi:10.1002/minf.202300262; doi:10.1007/s11172-019-2683-0). These studies demonstrate the high accuracy and strong predictive power of the PASS algorithms.

 

5. The "None" class is conditionally negative, and the consequences for the reported metrics are under-analyzed. The authors acknowledge that absence of a recorded association does not establish true absence of an ADR, and that false negatives may contaminate the None class. This is discussed qualitatively but its quantitative impact is not addressed. Given that None is by far the largest class in every dataset (tens of thousands of pairs vs. hundreds for Major), even a small mislabeling rate could materially shift specificity and AUC. The authors should discuss the plausible direction and magnitude of this effect, and ideally probe sensitivity of the metrics to the negative-selection thresholds.

 

Response: We thank the reviewer for this comment. To assess the sensitivity of the results to the composition of the conditionally negative “None” class, we performed approximately 100 repeated random undersampling experiments. In each experiment, the number of “None” pairs was reduced to the size of the largest “non-None” class, while all “Major”, “Moderate”, and “Minor” examples were retained unchanged. The standard deviation of the resulting AUC estimates was only 1.14 percentage points, indicating low sensitivity to the random composition of the None subset. The performance estimates for all positive severity classes were identical across all 100 runs.

 

 

6. The case-based assessment (Table 3) is interpreted selectively. For warfarin+amiodarone the Major class does not receive the highest Pa (0.090/0.691; Pa < Pi), yet the text frames this as a correct prediction of "manageable bleeding risk." This reads as a post-hoc rationalization of a case the model arguably misclassified, particularly since warfarin+amiodarone is clinically a serious, closely monitored interaction. The authors should either apply a consistent decision rule across all cases in Table 3 or explicitly acknowledge the miss rather than reframing it as success. As anecdotal illustrations, these cases should also not be over-interpreted as validation.

 

Response: We thank the reviewer for pointing out this inconsistency. We agree that our original interpretation of the warfarin–amiodarone example was overly favorable. According to the predefined PASS criterion Pa>Pi, the Major category was not predicted for this drug pair (Pa/Pi = 0.090/0.691), and this result should therefore be considered a false-negative prediction for the Major severity category. The model did predict the Minor and Moderate bleeding categories, indicating that it captured the association with bleeding but underestimated its severity. We have revised the corresponding paragraph to acknowledge this model error explicitly and have clarified that the case-based examples are illustrative and should not be interpreted as independent model validation.

Previous version: As shown in Table 3, the results of the case-based assessment were generally consistent with published clinical data on DDI-associated ADRs. Candidate drug combinations for this assessment were identified from published clinical reports and DDI data. For the final case-based assessment, only those drug pairs that were absent from the training datasets used to build the corresponding predictive models were selected. For the combination of warfarin and amiodarone, the Major class did not receive the highest probability estimate (Pa/Pi = 0.090/0.691); however, higher Pa values were observed for the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) classes. This interpretation is consistent with published data indicating that this combination increases the risk of bleeding primarily by enhancing the anticoagulant effect of warfarin and requires dose adjustment and monitoring of anticoagulant intensity [13]. Thus, for this drug pair, the prediction reflects not an inevitable major bleeding event, but rather a clinically significant and potentially manageable bleeding risk.

Revised version: The case-based assessment included examples of predictions that were consistent with published clinical evidence as well as examples illustrating limitations of the model. The individual cases are discussed below. For the combination of warfarin and amiodarone, the Major class was not predicted (Pa/Pi = 0.090/0.691) and should therefore be considered a false-negative prediction for the Major severity category. At the same time, the model predicted both the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) categories. Thus, although the model identified an association of this drug pair with bleeding, it underestimated its severity [20]. This example illustrates an important limitation of the structure-based approach: clinically significant severity may depend on pharmacokinetic interactions, dose, monitoring, and patient-specific factors that are not represented by molecular structure alone. Accordingly, the case-based assessment should be regarded as illustrative rather than as an independent validation of the model.

 

7. Mean-by-class metrics in Table 2 average over structurally non-comparable tasks. The number of severity classes varies across ADRs — some datasets include a Minor class, some do not; hyperglycemia and hypokalemia lack a Major class entirely. Averaging performance across ADRs with differing class structures makes the aggregate figures difficult to interpret. The authors should note this heterogeneity and consider whether a weighted or stratified summary is more informative.

 

Response: We thank the reviewer for this comment. We agree that the original presentation of Table 2 was potentially misleading because the class-specific averages were calculated across different subsets of ADR models. We have therefore revised the table to report the number of contributing models for each severity category and to present the values explicitly as unweighted macro-averages.

We have also clarified that the severity classes are not represented in identical sets of ADR-specific models. Each class-specific value in Table 2 is therefore calculated only across the ADR models containing the corresponding severity category. The revised table reports the number of contributing ADR-specific models and the mean number of drug pairs per contributing model for each severity class. We explicitly identify these values as unweighted macro-averages and note that they should be interpreted as descriptive summaries rather than as directly comparable estimates derived from identical sets of prediction tasks.

Previous version: Table 2. Mean ADRs SAR models performance by severity class.

Revised version: Table 2. Unweighted macro-average predictive performance of ADR-specific models by severity class.
Class: DDI severity class; No. of ADR models: number of ADR-specific models contributing to the corresponding severity-class average; Mean N per model: mean number of drug pairs belonging to the corresponding severity class across the contributing ADR-specific models; AUC: area under the receiver operating characteristic curve; Sen.: sensitivity; Spec.: specificity; BA: balanced accuracy.

 

8. Class imbalance is discussed only qualitatively. The manuscript notes the imbalance and its likely effect on the Major class, but reports no mitigation strategy (resampling, class weighting, cost-sensitive learning) and no quantitative analysis of its influence. A brief treatment of what, if anything, was done — and what effect it had — would improve methodological rigor.

 

Response: We thank the reviewer for this comment. No oversampling, undersampling, class weighting, or cost-sensitive learning was used in the original PASS models. The PASS algorithm is designed to be insensitive to unbalanced training samples. Moreover, it is also weakly sensitive to incomplete data (doi:10.1021/ci000383k). PASS accounts for class frequencies internally when constructing the Pa and Pi distributions, while balanced accuracy was used only as an evaluation metric.

To quantify the influence of the dominant “None” class, we performed 100 independent random undersampling experiments, reducing “None” to the size of the largest “non-None” class while retaining all positive examples unchanged. The standard deviation of the resulting performance estimates was only 1.14 percentage points, indicating that the results were relatively insensitive to the size and random composition of the None class (see plot below and summary table). Nevertheless, the limited number of Major examples remains a limitation and may contribute to their lower predictive performance. No resampling was used for development of the primary PASS DDI models; the undersampling experiment was performed only as a post hoc sensitivity analysis requested during peer review.

Previous version: SAR analysis was performed using a special version of PASS DDI (Prediction of Activity Spectra for Substances with respect to Drug–Drug Interactions) [23,26]. In PASS DDI, classification models are constructed based on PoSMNA descriptors using an algorithm that is based on a modified naive Bayesian approach [27,28]. After training, PASS DDI calculates probability estimates for various types of biological activity, including ADRs associated with pairs of compounds, based on their structural formulas.

Revised version: SAR analysis was performed using a special version of PASS DDI (Prediction of Activity Spectra for Substances with respect to Drug–Drug Interactions) [30,31]. In PASS DDI, classification models are constructed based on PoSMNA descriptors using an algorithm that is based on a modified naive Bayesian approach [34,35]. After training, PASS DDI calculates probability estimates for various types of biological activity, including ADRs associated with pairs of compounds, based on their structural formulas. No explicit class-rebalancing procedures, including oversampling, undersampling, synthetic-sample generation, class weighting, or cost-sensitive learning, were applied during model development. All PASS DDI models were trained using the original class distributions. The modified naive Bayesian algorithm implemented in PASS has previously been shown to be relatively insensitive to unequal class sizes and incomplete activity annotations [34]. Balanced accuracy was used only as an evaluation metric that assigns equal importance to sensitivity and specificity; it was not used to modify the training samples or to optimize the classification threshold.

 

Response Figure R1. Distribution of ROC AUC values for the conditionally negative “None” class across 100 independent random undersampling runs. Performance estimates for the positive severity classes were identical across all runs; therefore, only the variability of the None-class AUC is shown.

№ of undersamplings		100
Mean AUC	0.6947
Median AUC	0.6943
Min AUC	0.6626
Max AUC	0.7255
Range of AUC	0.0629
SD of AUC	0.0114
AUC Q1	0.6875
AUC Q3	0.7028

 

 

9. An error analysis is missing. Representative false-positive and false-negative predictions, ideally linked to structural or mechanistic features, would give useful insight into where and why the models fail — especially for the poorly predicted ADR types (bradycardia/Major AUC ≈ 0.62; lowering seizure threshold ≈ 0.65).

 

Response: We thank the reviewer for this comment. According to the data presented in Table 1, there are a total of 3,320 compounds in the "Major" classes, 18,402 in the "Moderate" classes, and 798,500 in the "None" classes. Therefore, any essential analysis of the structural or mechanistic causes of false-positive and false-negative predictions is absolutely impossible.

Nevertheless, we agree that an illustrative example of model failure is useful. We therefore revised the case-based assessment to explicitly retain the warfarin–amiodarone interaction as a representative false-negative severity prediction. For this drug pair, the model predicted the Minor and Moderate bleeding categories but failed to predict the clinically relevant Major category, thereby underestimating the severity of the interaction. We now discuss this case explicitly as a limitation of the model rather than as a successful prediction. A systematic structural or mechanistic attribution of individual false-positive and false-negative predictions would require a separate dedicated analysis and is beyond the scope of the present study.

Previous version: As shown in Table 3, the results of the case-based assessment were generally consistent with published clinical data on DDI-associated ADRs. Candidate drug combinations for this assessment were identified from published clinical reports and DDI data. For the final case-based assessment, only those drug pairs that were absent from the training datasets used to build the corresponding predictive models were selected. For the combination of warfarin and amiodarone, the Major class did not receive the highest probability estimate (Pa/Pi = 0.090/0.691); however, higher Pa values were observed for the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) classes. This interpretation is consistent with published data indicating that this combination increases the risk of bleeding primarily by enhancing the anticoagulant effect of warfarin and requires dose adjustment and monitoring of anticoagulant intensity [13]. Thus, for this drug pair, the prediction reflects not an inevitable major bleeding event, but rather a clinically significant and potentially manageable bleeding risk.

Revised version: The case-based assessment included examples of predictions that were consistent with published clinical evidence as well as examples illustrating limitations of the model. The individual cases are discussed below. For the combination of warfarin and amiodarone, the Major class was not predicted (Pa/Pi = 0.090/0.691) and should therefore be considered a false-negative prediction for the Major severity category. At the same time, the model predicted both the Minor (Pa/Pi = 0.803/0.075) and Moderate (Pa/Pi = 0.767/0.023) categories. Thus, although the model identified an association of this drug pair with bleeding, it underestimated its severity [20]. This example illustrates an important limitation of the structure-based approach: clinically significant severity may depend on pharmacokinetic interactions, dose, monitoring, and patient-specific factors that are not represented by molecular structure alone. Accordingly, the case-based assessment should be regarded as illustrative rather than as an independent validation of the model.

 

10. Claims of clinical applicability should be moderated though the authors are already fairly cautious. The Discussion and Conclusions already state repeatedly that this is a preliminary screening tool and that predicted severity is not equivalent to patient outcome. The recommendation is to ensure this framing is consistent throughout (including the Abstract) rather than to add new caveats, since the necessary qualifications are largely present.

 

Response: We thank the reviewer for this comment. We reviewed the Abstract, Discussion, and Conclusions to ensure consistent framing of the proposed method as a preliminary screening tool rather than a clinical decision-making system. Statements that could imply direct prediction of individual patient outcomes were revised accordingly. No additional caveats were introduced, as the principal limitations were already described in the manuscript. We also verified that the Abstract and Conclusions do not imply direct prediction of individual patient outcomes or immediate clinical decision-making. The revised wording consistently presents AdverDDIPred as a preliminary screening and prioritization tool for identifying potentially concerning drug combinations that may warrant further evaluation.

 

11. The Discussion is partly interpretive but leans on restating Results. It does add mechanistic interpretation (e.g. hepatotoxicity, vascular-tone effects), but a more critical placement of the findings relative to the specific prior studies cited in the Introduction — rather than summary of the tables — would strengthen it.

 

Response: We thank the reviewer for this comment. We agree that the original Discussion placed excessive emphasis on restating the numerical results and did not sufficiently relate our findings to the specific previous studies introduced earlier in the manuscript. We have therefore revised the Discussion to position the present work more explicitly relative to previous approaches to ADR severity assessment and polypharmacy side-effect prediction. In particular, we now contrast the present ADR- and drug-pair-specific severity classification task with previous studies addressing single-drug toxicity severity, quantitative ADR severity assessment, and prediction of polypharmacy-associated side effects. We also clarify that a distinguishing feature of the present approach is the prediction of Major, Moderate, Minor, and conditionally negative None categories for a specific ADR associated with a specific drug pair using structural information alone.

Previous version: No dedicated paragraph directly comparing the present task with previous ADR-severity and polypharmacy-side-effect prediction approaches was included in the original Discussion.

Revised version: Previous studies have addressed related aspects of ADR and DDI prediction using different task formulations. Thakkar et al. developed the DILIst classification for drug-induced liver injury severity at the level of individual drugs [8], whereas Lavertu et al. investigated quantitative assessment of ADR severity using information derived from social media [5]. Other computational approaches, such as the graph-based model of Zitnik et al., focused on predicting polypharmacy-associated side effects of drug combinations rather than explicitly assigning ADR-specific severity categories [16]. In contrast, the present study formulates the problem as severity prediction for a specific ADR associated with a specific pair of drugs and distinguishes Major, Moderate, Minor, and conditionally negative None categories. Thus, the proposed approach complements previous work by addressing the intersection of drug-pair-specific ADR prediction and severity classification using only structural information.

 

 

 

Overall, the study is a worthwhile contribution to computational pharmacology. Comments 1–3 in particular should be addressed before the manuscript is suitable for publication; the remaining points would improve rigor and interpretability.

 

Good luck!

 

We do hope that in its revised form the manuscript could be accepted for publication.

Response: Thank you very much for your detailed comments and suggestions! We appreciate your positive assessment of our work and hope that the revised manuscript is now suitable for publication.

Author Response File: Author Response.docx