Model Adequacy in Assessing the Predictive Performance of Regression Models in Pharmaceutical Product Optimization: The Bedaquiline Solid Lipid Nanoparticle Example

S/N

REVIEWER 1 COMMENTS

AUTHORS’ RESPONSES

1

Comment 1

This study appears to be a follow-up to the previous work titled "A Quality by Design Approach for Optimizing Solid Lipid Nanoparticles of Bedaquiline for Improved Product Performance. “Which was conducted by the same author. This context is important as it highlights the ongoing efforts to design more effective nanocarriers. Mentioning this connection can provide readers with a better understanding of the study's background and its significance in the broader research landscape. I have provided a few suggestions to improve this study.

Thank You.

An additional phrase was added in the introductory section to highlight the connection suggested.

See Lines 47 -50

“In earlier research, this group communicated formulating BQ as SLNs for oral delivery presenting an opportunity to enhance the drug product performance [3]. In the current study the group progressed to compare the adequacy of the first- and second order regression models in accessing the predictive performance in the product optimization for the BQ SLNs formulation”

2

Comment 2

Abstract:

(a)  The abstract is generally clear but could benefit from more concise language. For instance, the phrase "Employed a central composite design and graphical optimization process using three steps" could be rephrased to "A three-step central composite design and graphical optimization process was employed."

(b)   In abstract section, "First-order polynomial showed poor model adequacy that required further refinement" is somewhat vague. It would be helpful to specify what aspects were inadequate and how the second-order models addressed these issues.

Thank you!

The Abstract section has been rephrased as suggested.

See Lines [14-15]

 Thank you for the observation.

This Abstract section has been expounded to explain the reason behind the inadequacy of the first polynomial model and how the second model option addressed the issue.

See Lines [20-23]

The first-order polynomial showed poor model adequacy and required further refinement due to its lack of explanatory power and significant predictors. Conversely, the second-order models provided superior fitness, sensitivity to variability, complexity, and prediction consistency

3

Comment 3

Introduction:

(a)  In the introduction, the background on solid lipid nanoparticles (SLNs) and their benefits is well-presented. However, the transition to discussing bedaquiline (BQ) could be smoother. Consider linking the benefits of SLNs directly to the challenges in treating multi-drug-resistant tuberculosis (MDRTB).

(b)  The purpose of the current study is stated, but the scope could be clearer. For instance, specifying that the study focuses on comparing first-order and second-order regression models for optimizing BQ SLN formulations would enhance clarity.

Thank you for the advice.

The manuscript has been revised to link up the benefits of SLNs directly to the challenges in treating multi-drug-resistant tuberculosis (MDRTB) as suggested.

 See Lines [37-42]         

Prior research demonstrated the potential of nanotechnology in the treatment of pul-monary inflammatory diseases, highlighting the benefit of focused therapeutics with lower toxicity and dosage requirements. Some advantages of nano-drug delivery sys-tems include improved drug product stability, solubility and controlled release. SLNs have been used to deliver poorly water-soluble molecules and have the potential to achieve sustained drug release or targeted delivery to the site of interest [1, 2].

Thank you for the suggestion.

The manuscript has been revised to specify that the study focuses on comparing first-order and second-order regression models for optimizing BQ SLN formulations as suggested.

See lines [52-53]

However, the current study focuses on comparing first-order and second order regression models for optimizing BQ SLN formulation

4

Comment 4

Discussion:

-  (a)  Incorporating recent studies can definitely strengthen the introduction. Here are some recently published papers related to solid lipid nanoparticles (SLNP) and drug delivery that you can reference:

https://doi.org/10.1002/adtp.202300275

DOI: 10.1007/978-1-0716-2954-3_12

https://doi.org/10.1007/978-981-19-8342-9_12

(b) The statistical results are presented comprehensively. However, the interpretation of these results could be more explicit. For example, explaining why certain models have poor fit or prediction capabilities would provide better context.

Thank you for this suggestion,

The manuscript has been revised to reference the recently published papers pertaining to solid lipid nanoparticles (SLNP) and drug delivery as advised.

Lines [37-42]

Thanks, the correction done.

See lines [292-295], [310-312]

The combined metrics of low R2, high variability, negative AIC and BIC values, nega-tive adjusted and predicted R², and low precision, collectively highlight the limitations in the first order polynomial to accurately predict the Pdl model in the unsonicated formulations

The later suggests that the first order polynomial is not appropriate for accurately predicting the ZP model, as evidenced by the combination of a very low R² value, negatively adjusted R², significantly negative forecasted R² and high AIC and BIC values.   A lower AIC or BIC value indicates a better fit [20]

S/N

REVIEWER 2 COMMENTS

AUTHORS’ RESPONSES

1

Comment 1:

To improve clarity and precision, the article could benefit from major corrections, specifically by clarifying the rationale for selecting the quadratic model over the first-order model to better capture nonlinear interactions among formulation variables.

(a) Explicitly providing key metrics, such as R² and p-values, for both models (e.g., R² of 0.895 for PdI in sonicated formulations) would enhance transparency regarding the improvements observed with the second-order model.

(b) Additionally, briefly describing how graphical tools, like contour and response surface plots, guided the optimization of BQ (10-20%) and Tween 80 (15-18%) concentrations would underscore their role in achieving desirable particle stability and size distribution in SLNs.

Thank you for the observation, the paper has been revised to provide include examples that enhance transparency regarding the improvements observed with the second-order model as suggested.

Lines [643-656]

The drawback in the linear model’s poor predictive performance was evident by low R² values (0.3404 for PSD) and negative predicted R² values across all response variables. Also, prominent is the model’s inability to adequately account for the variability among these variables (as in adjusted R²values of 0.5808 for Pdl), thereby allowing for significant variation, and inaccurate predictions, which raises the possibility of inconsistent product quality. These result in suboptimal formulation, longer and more expensive development times due to repeated cycle of formulation testing and adjustments, a lack of reliable data for regulatory submission and detrimental effects on customer confidence and business reputation. Overall, the quadratic (second order) model is superior to the linear (first order) model for the sonicated BQ formulation. It provides higher explanatory power (as with adjusted R²values of 0.7317 for Pdl), better predictive capability although low (as with predicted R²values of 0.0111 for Pdl), improved precision (as with precision ratio 8.5101 compared to 0.9748 for linear model), and lower variability across all response variables (PdI, PSD, and ZP). All these confer on it the better model for optimization purposes. 

Thank you for the observation.

The paper has been revised to briefly describing how graphical tools, like contour and response surface plots, guided the optimization of BQ (10-20%) and Tween 80 (15-18%) concentrations as suggested.

Lines [502-518]

Graphical tools such as contour plots and response surface plots are essential in visualizing the relationship between factors like BQ and T80 concentrations and their impact on the desired response. The use of these tools to guide the optimization of BQ (10-20%) and T80(15-18%) can be explained as follows: first, the contour plots were designed with the use of JMP software to show regions of constant responses based on varying concentrations of BQ (10-20%) and T80 (15-18%). When the concentration of the variables are consistently adjusted, it can identify the optimal regions where the CQAs meet the desirable product target profiles such as minimal PSD. For example, the range of BQ (12-18%) and T80 (16-17%) concentrations, where a PSD of ≤ 500nm is achieved, forms the acceptable region considered as the optimal conditions for achieving that CQA. When the concentration of either input variables deviate from the specified limits, the formulation’s PSD increases moving outside desired target profile. The interplay in varying the BQ and T80 concentrations to identify the region of optimal condition for the response variable, is used to showcase the systematic optimization of the formulation parameters.  On the other hand, the 3D plot provides a three-dimensional view of the response, showing how it varies across the entire range of BQ and T80 concentrations. With this, the optimal region where the desired response is maximized or minimized is identified.

2

Comment 2:

Consistent terminology (e.g., introducing “SLNs” early and using it consistently) would streamline the discussion,

Thank you for the observation.

The manuscript has been revised to ensure that terminologies mentioned in the paper are consistently used all through.   For instance,

See Lines [736] [ 743] etc.

3

Comment 3:

while a few added details on the statistical software (e.g., JMP) and evaluation metrics like the Akaike Information Criterion (AIC) and adjusted R² would clarify the model adequacy assessment.

Thank you for your insightful observations regarding the deficiencies noted in our study. The manuscript has been revised to include role of the statistical software (JMP) and the evaluation metrics mentioned in the model adequacy assessment.

See Lines [164-168] for few added details on JMP statistical software.

This software program performs statistical analysis, modelling, and data visualization. It is used to support DOE and design generation where it interactively builds and refine graphs and tables with the graph builder and tabulate tools, respectively. The software also employs the least squares method for fitting linear regressions such as the first-order model.

See Lines [236-237] [239-241] [309-312] for few added details on evaluation metrics (AIC & BIC and Adjusted R²)

Adjusted R² accounts for the number of predictors in the model, offering a more accurate measure of fit when multiple variables are involved. Predicted R² evaluates the model's ability to predict new data, with higher values indicating strong predictive performance. Additional metrics such as the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were used to balance model fit with complexity, where lower values suggest a more parsimonious model.

The later suggests that the first order polynomial is not appropriate for accurately predicting the ZP model, as evidenced by the combination of a very low R² value, negatively adjusted R², significantly negative forecasted R² and high AIC and BIC values.   A lower AIC or BIC value indicates a better fit [20].

4

Comment 4:

Finally, emphasizing the broader significance of these findings, particularly in enhancing formulation precision and stability in drug delivery systems, would help readers appreciate the value of precise model selection in pharmaceutical product optimization.

Thank you for the observation.

The Conclusion section of the manuscript had been revised to capture the broad significance of findings in enhancing formulation precision and stability in drug delivery systems as suggested.

See Lines [758-761]

Finally, the broader significance of this study can be applied to the importance of precise model selection in pharmaceutical product optimization. Identifying and using the right regression model during product development is essential in improving formulations in drug delivery systems

S/N

REVIEWER 3 COMMENTS

AUTHORS’ RESPONSES

1

Comment 1:

In Table 1, the line numbers overlap the first column of Table 1.

Thank you for the observation.

The manuscript has been revised to separate the line numbers from the first column of table 1.

See Line [190-192]

2

Comment 2:

Please, correct.
Equation 5 is of poor legibility. Please, improve the resolution of equation 5.

Thank you for the observation.

The resolution of equation 5 has been improved as advised.

See Line [199-200]

S/N

REVIEWER 4 COMMENTS

AUTHORS’ RESPONSES

1

 Comment 1:

Model Performance: The poor predictive performance of some models (e.g., negative predicted R² values) is noted but not extensively discussed. More explanation of these results and their implications would be beneficial.

Thank you for the observation, the paper has been revised to include additional information on results and implications of poor predictive performance of some models as suggested.

See Lines [642-655]

The drawback in the linear model’s poor predictive performance was evident by low R² values (0.3404 for PSD) and negative predicted R² values across all response variables. Also, prominent is the model’s inability to adequately account for the variability among these variables (as in adjusted R²values of 0.5808 for Pdl), thereby allowing for significant variation, and inaccurate predictions, which raises the possibility of inconsistent product quality. These result in suboptimal formulation, longer and more expensive development times due to repeated cycle of formulation testing and adjustments, a lack of reliable data for regulatory submission and detrimental effects on customer confidence and business reputation. Overall, the quadratic (second order) model is superior to the linear (first order) model for the sonicated BQ formulation. It provides higher explanatory power (as with adjusted R²values of 0.7317 for Pdl), better predictive capability although low (as with predicted R²values of 0.0111 for Pdl), improved precision (as with precision ratio 8.5101 compared to 0.9748 for linear model), and lower variability across all response variables (PdI, PSD, and ZP). All these confer on it the better model for optimization purposes.

2

Comment 2:

Unsonicated vs. Sonicated Formulations: While the study compares these two types of formulations, the rationale for focusing more on sonicated formulations in later analyses could be more clearly explained.

Thank you for the observation.

The manuscript has been revised to provide rationale for focusing more on sonicated formulations in the later analyses as requested.

Lines [330-336]

Earlier step using first order models in optimizing both formulations, shows that in sonicated formulation, statistical significance, higher precision, and improved predictive model fits polydispersity index (Pdl) and, a better model fitness and complexity was achieved in the formulation’s particle size distribution (PSD). This implies that sonication drastically changes the physical characteristics of the nanoparticles, such as particle size and surface charge, which are essential for their stability and functionality. Hence, the need for greater emphasis on sonicated formulations in subsequent investigations.