Next Article in Journal
Bio-Based Polyurethane and Its Composites towards High Damping Properties
Next Article in Special Issue
Extensive Sampling of Molecular Dynamics Simulations to Identify Reliable Protein Structures for Optimized Virtual Screening Studies: The Case of the hTRPM8 Channel
Previous Article in Journal
The State of the Art of Piezo1 Channels in Skeletal Muscle Regeneration
Previous Article in Special Issue
PLATO: A Predictive Drug Discovery Web Platform for Efficient Target Fishing and Bioactivity Profiling of Small Molecules
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Monte Carlo Models for Sub-Chronic Repeated-Dose Toxicity: Systemic and Organ-Specific Toxicity

Laboratory of Chemistry and Environmental Toxicology, Department of Environmental Health Sciences, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Int. J. Mol. Sci. 2022, 23(12), 6615;
Received: 18 May 2022 / Revised: 9 June 2022 / Accepted: 10 June 2022 / Published: 14 June 2022
(This article belongs to the Special Issue State-of-the-Art Molecular Informatics in Italy)


The risk-characterization of chemicals requires the determination of repeated-dose toxicity (RDT). This depends on two main outcomes: the no-observed-adverse-effect level (NOAEL) and the lowest-observed-adverse-effect level (LOAEL). These endpoints are fundamental requirements in several regulatory frameworks, such as the Registration, Evaluation, Authorization and Restriction of Chemicals (REACH) and the European Regulation of 1223/2009 on cosmetics. The RDT results for the safety evaluation of chemicals are undeniably important; however, the in vivo tests are time-consuming and very expensive. The in silico models can provide useful input to investigate sub-chronic RDT. Considering the complexity of these endpoints, involving variable experimental designs, this non-testing approach is challenging and attractive. Here, we built eight in silico models for the NOAEL and LOAEL predictions, focusing on systemic and organ-specific toxicity, looking into the effects on the liver, kidney and brain. Starting with the NOAEL and LOAEL data for oral sub-chronic toxicity in rats, retrieved from public databases, we developed and validated eight quantitative structure-activity relationship (QSAR) models based on the optimal descriptors calculated by the Monte Carlo method, using the CORAL software. The results obtained with these models represent a good achievement, to exploit them in a safety assessment, considering the importance of organ-related toxicity.

1. Introduction

The European Commission (EC) requires continuous efforts to develop new approaches in the safety assessment of chemicals consistent with the regulatory framework. The European Directive on cosmetics, for example, laid out a new vision in Europe for the safety assessment of cosmetics, moving from the consolidated, animal-based, toxicological approach to a novel paradigm that bans the use of animals for toxicity tests [1,2]. Regulators explicitly require a deep examination of RDT, since it gives crucial details on the critical adverse effects resulting from repeated exposure to a certain substance in a limited period [3,4]. REACH, which prescribes within its annexes an assessment on RDT [3], firmly supports the application of new-approach methodologies in order to provide forward-looking solutions in the field of risk assessment.
RDT studies usually employ a wide array of standards; indeed, they present different exposure times and routes as well as various models, such as rodent or non-rodent species [5]. The exposure times can be divided into three groups: (i) sub-acute toxicity studies, which have a duration of up to 28 days (usually 2–4 weeks); (ii) sub-chronic toxicity, which has a duration of about 90 days; (iii) chronic (long-term) toxicity (repeated doses over a major part of the life-span) [4].
As toxicologically relevant pieces of evidence, the NOAEL and LOAEL are derived from the RDT studies. The NOAEL is the highest experimental dose at which no significant adverse response is observed, and the LOAEL is the lowest dose at which adverse effects arise compared to a control group [6]. NOAEL can be used to determine the important toxicological values considered in the evaluation of human exposure, such as the acceptable daily intake (ADI) and the reference dose (RfD) [7].
In the safety evaluation of cosmetics, when no benchmark dose (BMD) can be calculated, NOAEL is used to calculate the margin of safety (MoS) [4]—an essential parameter in the product information file (PIF)—which is vital for placing the product in the European market [2] and fundamental for indicating the potential toxicity for human health of substances in a cosmetic formulation [4].
NOAEL and LOAEL are marked by a substantial uncertainty since they are not intended for the dose−response curves but as single-dose estimates that largely depend on the experimental design [6]. As a result, other approaches such as the BMD have been proposed in place of the classical methodologies [4,8]. Unfortunately, the availability of data useful for BMDs is quite limited.
RDT studies require high costs and considerable time expenditures. For this reason, in silico methods are needed to offer a valid, future-oriented alternative [9]. Moreover, only a few attempts have been made to apply QSAR methods to simultaneously model NOAEL and LOAEL [6,10,11].
The aim of this work was to develop new QSAR models to predict NOAEL and LOAEL, focusing in particular on organ-specific toxicity, to obtain fundamental information on systemic toxicity and the advanced solutions in the research used for the in silico models. To achieve this, we used the Monte Carlo algorithm, with the CORAL software (, accessed on 11 June 2021). Other attempts to develop models for NOAEL and LOAEL using CORAL have been made in recent years [6,10,11]. In this work, we extended the method by applying the algorithm to new data related to organs, particularly the liver, kidney and brain, focusing on the toxicity of a specific organ. The models will be implemented on the VEGA platform (, accessed on 2 December 2021) and will be freely available.
Finally, these models will be used to refine the battery of models within the VERMEER Cosmolife software [2] (, accessed on 3 May 2022), a new tool developed in the LIFE VERMEER project (LIFE16 ENV/IT/00016—, accessed on 3 May 2022). In particular, the section of the tool related to the dose–response assessment will be improved, to boost the potential of the tool and provide new solutions in the field of risk assessment [2].

2. Results

2.1. Characterization of the Datasets

The datasets contain chemicals from different sources covering several functional categories, such as pesticides, drugs, food additives and cosmetics. Figure 1 shows the histograms of the ToxPrint chemotypes identified in the datasets, reported in Tables S1–S4 of the Supporting Information. The chemotypes matching the structures in the datasets are shown if the frequency is over 10%. Out of 729 chemotypes, respectively, 376, 318, 211 and 328 were found in the general, kidney, brain and liver datasets. The most frequent ToxPrint chemotypes in all datasets were aromatic benzenes, C=O carbonyl generic, halides, alkanes, carboxamide, aminocarbonyl, heterocycles, aliphatic–aromatic ether and aromatic amines, all with over a 16% frequency in the datasets. The chemical space, covered by chemicals in terms of the structural and physico-chemical properties, is described in Tables S25–S28 (Supplementary Materials). Datasets present a similar structural and physico-chemical profile. This represents important evidence, which allows us to say that these models could be suitable for predicting external data with a chemical space in line with what we defined here.

2.2. Models Developed

The CORAL models were based on the hybrid optimal descriptors extracted from the simplified molecular-input line-entry system (SMILES) and molecular graph. The models for the predictions of NOAEL and LOAEL in rats for sub-chronic repeated-dose toxicity are given in Equations (1)–(8), while Table 1 shows the statistical characteristics of the models. We calculated the validation metrics for each model, such as the determination coefficient (R2), the leave-one-out (LOO) cross-validated determination coefficient (Q2), predictive R2 (R2 pred), the criteria of predictability Q2F3, the concordance correlation coefficient (CCC), the index of ideality of correlation (IIC), the mean absolute error (MAE), the root mean squared error (RMSE) and the Fischer F-ratio (F) [12,13,14]. The predictions and experimental values, together with the extracted attributes of the models, are reported in Tables S9–S24 in the Supplementary Materials.
The most important criterion for assessing the predictive potential of a model is the statistical quality of an external validation set. In the general models, the predictive ability R2 in the validation sets reached 0.55 and 0.53, respectively, for NOAEL and LOAEL. The external validation metrics of the CCC and Q2F3 for NOAEL were 0.69 and 0.53, and 0.63 and 0.32 for LOAEL. The LOO cross-validation technique Q2 was used to assess the robustness of the models: it ranged from 0.50 to 0.54 for NOAEL and 0.49 to 0.54 for LOAEL. A value of 0.5 is an acceptable threshold for the Q2 reported in the literature [13].
The performances were better in the models for the organ-specific toxic effects than in the general models. In the kidney models, the predictive ability R2 in the validation sets reached 0.59 for NOAEL and 0.69 for LOAEL. The external validation metrics, CCC and Q2F3, were 0.75 and 0.84 for NOAEL, and 0.82 and 0.81 for LOAEL. The Q2 ranged from 0.48 to 0.58 for NOAEL and from 0.49 to 0.63 for LOAEL. In the brain models, R2 in the validation sets reached 0.53 for NOAEL and 0.69 for LOAEL. The agreement between the experimental and calculated values, expressed by the metric CCC, was 0.67 for NOAEL and 0.80 for LOAEL. The criterion of predictability Q2F3 was 0.81 for NOAEL and 0.82 for LOAEL. Moreover, Q2 ranged between 0.46 and 0.51 for NOAEL and 0.44 and 0.61 for LOAEL. Last but not least, in the liver models, R2 reached 0.55 and 0.61 in the validation sets for NOAEL and LOAEL, respectively. The metric CCC reached similar values for NOAEL and LOAEL. The criterion of predictability Q2F3 was 0.55 for both NOAEL and LOAEL. The robustness of the models expressed by Q2 reached similar values: higher than 0.72 for NOAEL and LOAEL. It can be noticed that LOAEL performs better than NOAEL in all the cases. Considering the complexity of modeling these endpoints, the overall performance is quite satisfactory for all the models. It is possible to observe that the performance of the liver models is better than the other organ-specific models, showing higher robustness, expressed by Q2, and lower MAE and RMSE values.

2.2.1. General Models

The general NOAEL model was built up using the equivalent distribution in the active training set (n = 140), passive training set (n = 140), calibration set (n = 140), and validation set (n = 141). The model was calculated as follows:
NOAEL   = 2.1123367   ( ±   0.0064558 ) + 0.1651535   ( ±   0.0010034 ) ×   DCW ( 1 ,   30 )
The general LOAEL model was built up using the equivalent distribution in the active training set (n = 142), passive training set (n = 142), calibration set (n = 137), and validation set (n = 137). The model was calculated as follows:
NOAEL   = 2.1123367   ( ±   0.0064558 ) + 0.1651535   ( ±   0.0010034 ) ×   DCW ( 1 ,   30 )

2.2.2. Kidney Models

The NOAEL kidney model was built up using the equivalent distribution in the active training set (n = 95), passive training set (n = 95), calibration set (n = 45), and validation set (n = 45). The model was calculated as follows:
NOAEL kidney   = 2.1060212   ( ±   0.0059588 ) + 0.1735682   ( ±   0.0016214 ) × DCW ( 1 ,   10 )
The LOAEL kidney model was built up using the equivalent distribution in the active training set (n = 97), passive training set (n = 102), calibration set (n = 46), and validation set (n = 38). The model was calculated as follows:
LOAEL kidney   = 2.4150252   ( ±   0.0053276 ) + 0.1027443   ( ±   0.0010802 ) × DCW ( 1 ,   10 )

2.2.3. Brain Models

The NOAEL brain model was built up using the equivalent distribution in the active training set (n = 23), passive training set (n = 22), calibration set (n = 23), and validation set (n = 22). The model was calculated as follows:
NOAEL brain   = 2.3962455   ( ±   0.0337524 ) + 0.2029147   ( ±   0.0106681 ) ×   DCW ( 1 , 33 )
The LOAEL brain model was built up using the equivalent distribution in the active training set (n = 22), passive training set (n = 23), calibration set (n = 22), and validation set (n = 23). The model was calculated as follows:
LOAEL brain = 2.5305357   ( ±   0.0383210 ) + 0.1089317   ( ±   0.0053590 ) × DCW ( 1 , 4 )

2.2.4. Liver Models

The NOAEL liver model was built up using the equivalent distribution in the active training set (n = 97), passive training set (n = 94), calibration set (n = 30), and validation set (n = 31). The model was calculated as follows:
NOAEL liver   = 1.7245212   ( ±   0.0040067 ) + 0.0530130   ( ±   0.0002568 ) ×   DCW ( 2 , 15 )
The LOAEL liver model was built up using the equivalent distribution in the active training set (n = 96), passive training set (n = 95), calibration set (n = 30), and validation set (n = 31). The model was calculated as follows:
LOAEL liver   = 1.7148691   ( ±   0.0038133 ) + 0.0529304   ( ±   0.0002685 ) ×   DCW ( 2 , 14 )

3. Discussion

In the present study, the in silico models were developed for predicting NOAEL and LOAEL for the sub-chronic toxicity data (90 days of exposure). The development of the in silico models for RDT represents a challenging task [9,15]. The uncertainty that distinguishes the process derives, in particular, from the experimental design [6]. Indeed, different designs can produce contrasting NOAEL and LOAEL for the same chemical [9]. This variability makes the development of computational models extremely challenging. In recent years, only a few NOAEL and LOAEL models have been developed [6,10,15]. Table 2 shows the statistical characteristics that are in common with the models developed here, which we will discuss in Section 3.1. The complexity of developing models for NOAEL and LOAEL arises also from the fact that NOAEL and LOAEL are related to an extended range of effects that could arise in different target organs. These aspects, as well as the extensive modes and mechanism of actions, are difficult to consider separately, and often only partial information is provided. In this view, all of these features contribute to the final toxicity value. This study proposed new models for examining the critical effects that could arise in target organs, bearing in mind the abovementioned criticisms and limitations. To the best of our knowledge, this is one of the few attempts to develop NOAEL and LOAEL models for organ-specific toxicity. Predicting target-organ toxicity is a focal point in the next-generation risk assessment (NGRA) [16] and in the future, predictions of organ-level effects, in particular for the liver, should represent the basis of in silico modeling [17]. Moreover, data with a more defined understanding of the mechanisms/effects and/or adverse outcome pathways (AOP) is needed to improve the prediction strategy [17]. A thorough investigation of the ADME processes of a chemical, i.e., its ability to penetrate certain biological barriers and its distribution within the human body, is fundamental in the forward-looking NGRA approach [16]. Additionally, the physiologically-based kinetic (PBK) models, which are fundamental in determining the exposure to possible toxicants, can provide fundamental indications of concerns linked to target organs [18]. Considering this context, the QSAR models for organ-specific toxicity, which we have developed in this work, can provide outcomes useful for creating a future integrated-testing strategy.
Finally, our work follows the regulatory updates and aims to improve the risk-assessment procedure, introducing new alternative methods. NOAEL and LOAEL are fundamental endpoints for human health risk assessment under different regulatory contexts, such as cosmetics and REACH [3,4]. A significant use of time and money characterizes the in vivo RDT studies as well as the substantial number of animals involved. According to the cosmetics regulation, the in vivo RDT studies are not part of the testing procedures anymore [1], and REACH [3] calls for research to reduce the number of animals, according to the 3R principle [19]. Therefore, there is a pressing demand for new and valid alternatives for these endpoints.
However, the uncertainty of the NOAEL and LOAEL values, which is strongly dependent on the experimental design, tends to be reflected in the predictions obtained with the in silico models. Moreover, the link between the descriptors and systemic effects could not be linearly explained, since there is no weighty mechanistic principle [20]. Despite all these limitations, a reliable predictivity for NOAEL and LOAEL may be obtained using a combination of several techniques, such as QSARs, read-across, physiologically-based pharmacokinetic modeling (PBPK), the threshold of toxicological concern (TTC), and in vitro methods.

3.1. Comparison with Other Models

To the best of our knowledge, there are no NOAEL and LOAEL regression models related to organ-specific toxicity reported in the literature. We will therefore compare the general NOAEL and LOAEL models with the published and implemented models in free software.
In the last few years, other models have been developed for NOAEL and LOAEL in rats for sub-chronic repeated-dose toxicity (90 days of exposure). One model is currently freely available for predictions in the VEGA software [10,21]. The authors in [10] looked to studies in rodents to build NOAEL models. They considered studies with values for 28 days of treatment, dividing by 3 in order to approximate 90 days [4]. The dataset collected consisted of 140 chemicals with toxicity data from the Integrated Risk Information System (IRIS), Hazard Evaluation Support System (HESS) and Munro databases (, accessed on 5 January 2021) and from the OECD QSAR Toolbox, version 3.2 (, accessed on 5 January 2021). The performance of the model [10] is reported in Table 2.
In 2020, two new models (NOAEL and LOAEL) were developed [6]. These models were built using toxicity data from the Fraunhofer RepDose database, a non-public database. Studies only for the sub-chronic data from 90 days of oral toxicity in rats (Rattus norvegicus) were included. The models were based on 326 compounds. The statistical characteristics of the base model and the model based on chemicals inside the applicability domain (AD) are reported in Table 2. In this work, the models are only based on the public sources collected recently, covering more compounds than the other models reported in Table 2. The presence of the various functional groups and the chemical domains described by the physico-chemical properties, highlight the chemical heterogeneity of our datasets.
The most significant criterion for comparing the predictive potential of the models considers the statistical quality of an external (validation) set. For the NOAEL model, the validation set in [6] consisted of 38 compounds reaching R2 = 0.65. When excluding the compounds outside the AD, the predictive capacity reached R2 = 0.69, with 33 compounds covered. In the present work, the NOAEL model was based on a validation set of 141 compounds and reached R2 0.55. Excluding compounds outside the AD improved the R2, reaching 0.61 with a 77% coverage of compounds. The NOAEL model in [10] reached an R2 of, respectively, 0.60 and 0.61 in the whole validation set and in the validation inside the AD. However, the validation set had fewer compounds than the previous ones.
The LOAEL model in [6] reached R2 = 0.59. The predictive ability reached R2 = 0.62, excluding compounds outside the AD, with 33 compounds inside it. In this work, the LOAEL model was based on a validation set of 137 compounds and reached R2 0.53. R2 was similar, excluding compounds outside the AD with 74% of compounds.

4. Materials and Methods

4.1. Dataset Collection and Preparation

The datasets used to develop the QSAR models were built using the NOAEL and LOAEL experimental sub-chronic (90 days of exposure) oral-toxicity data in rats, from different publicly available databases, in line with their conformity to the OECD guideline, 408 [22]. These sources include the Munro and HESS databases, both of which are available from the OECD QSAR Toolbox, version 4.4 (, accessed on 5 January 2021), the IRIS database from the U.S. Environmental Protection Agency (EPA) (, accessed on 5 January 2021), the COSMOS database [23] (, accessed on 5 January 2021), the U.S. EPA’s ToxRefDB, v. 2.0 [24], and the European Food Safety Authority (EFSA) OpenFoodTox database [25,26]. In order to ensure consistent data, we took into account only data from the sub-chronic toxicity studies of oral exposure in rats (90 days, but also 91 and 92 days in order to extend the number of data), removing variability and inconsistencies due to interspecies’ differences [27]. Further, 2D SMILES chemical structures were retrieved for each substance. In some cases, SMILES were already present in the databases, otherwise, they were retrieved using a semi-automated data curation and quality checking workflow, implemented in the KNIME Analytics Platform, v. 4 [28,29].
Once a full dataset was set up under these rules, we rejected inorganic compounds, metal complexes and data related to mixtures or polymeric structures. Ionized structures were neutralized and counterions were excluded. Duplicates were analyzed and detected by checking SMILES at the 2D level, and by the CAS number. For compounds with multiple data, the lowest value was preserved, adopting a conservative approach. The experimental values (in mg/kg body weight (bw) per day) were converted to a logarithmic scale. Finally, to make the data more consistent, only chemicals with an NOAEL equal to or lower than LOAEL were maintained.
Another step was to retrieve data related not only to general-systemic toxicity but also to organ-specific toxicity. Therefore, following the same procedure, we built one dataset for general NOAEL and LOAEL and separate datasets using NOAEL and LOAEL data where the indication of the adverse effect is associated with a specific target/organ for the liver, kidney and brain. This provides a general dataset of 573 unique chemicals. For the organ-related data, we obtained 353 chemicals for the liver, 289 for the kidney and 91 for the brain. The datasets containing IDs, CAS numbers, SMILES and experimental values are reported in the Supporting Information, Tables S1–S4. A further curation aimed at the development of a specific model will be described in the next sections. The complete structure of these datasets is shown in Table 3.

4.2. Characterization of the Datasets

The chemical structures of the molecules (Supplementary Materials, Tables S1–S4) were analyzed with the ChemoTyper tool, v. 1.0, which searches and identifies 729 ToxPrint chemotypes (, accessed on 16 April 2021). The complete lists and histograms of the chemotypes identified in the datasets can be found, respectively, in Tables S5–S8 and in Figures S1–S4 (Supplementary Materials).
Chemicals were also analyzed on the basis of their physico-chemical properties. These were calculated using the ‘Molecular Properties’ node of the analytics platform, KNIME (, accessed on 16 April 2021). The properties included the molecular weight, hydrogen bond acceptors, hydrogen bond donors, rotatable bonds count, Lipinski’s rule of five, topological polar surface area, atomic polarizabilities, bond polarizabilities and MannholdlogP. Tables S25–S28 in the Supplementary Materials show the mean, maximum and minimum values of the selected properties for each dataset.

4.3. Further Curation

To build the models, we removed the compounds with a low-frequency distribution of NOAEL and LOAEL values, as they were considered noise. For the liver dataset, an analysis was done to identify the possible structural or response outliers and activity cliffs, employing the Small Dataset Curator tool, v. 1.0.0 [30,31] and the istSimilarity tool, v. 1.0.7 [32]. The final number of compounds and the range of the NOAEL and LOAEL endpoints are summarized in Table 3. Figures S5–S8 in the Supplementary Materials show the frequency distribution of the experimental values.

4.4. Model Development and Optimal Descriptors

We used the CORAL software, v. 2020 (, accessed on 6 June 2021), to build the QSAR models with the Monte Carlo method. The models were based on the so-called hybrid optimal descriptor, accounting for the correlation weights (CW) of both the SMILES and hydrogen-suppressed molecular graph (HSG) attributes [33,34].
The datasets were split randomly into the active training, passive training, calibration and validation sets [33,35]. Each subset had its own task. The task for the active training set was to calculate and identify which correlation weights gave the largest correlation between the experimental and predicted endpoint for the active training set. The task for the passive training set was to analyze if these correlation weights gave a reasonable correlation coefficient for similar compounds. The calibration set was used to detect overtraining. The validation set was used to evaluate the predictive potential of the model, using substances never used in the phase of model building.
The general form of a CORAL model can be described by the following one-variable Equation (9):
Endpoint = C 0 + C 1 ×   DCW ( T * , N * )
where C0 and C1 are the intercept and the slope, DCW is the optimal descriptor, T* and N* are the best threshold and the best number of epochs of the Monte Carlo optimization. The best threshold is an integer to classify a molecular feature as active or rare. The building process never involves rare features. Both thresholds give the best statistical performance for the calibration set.
The optimal descriptor—a function of molecular features—that we applied, comprises the following components, as listed in Equation (10):
DCW ( T , N ) = DCW α ( T , N ) + DCW β ( T , N ) + DCW γ ( T , N ) + DCW δ ( T , N )
DCW ( T , N ) = x 1 × CW ( S k ) + x 2 × CW ( SS k ) + x 3 × CW ( SSS k )
DCW β ( T , N ) = CW ( EC 1 k ) + CW ( EC 2 k ) + CW ( EC 3 k ) + CW ( NNC k )
DCW γ ( T , N ) = y 1 × CW ( APP [ Cl , N ] ) + y 2 × CW ( APP [ Cl , O ] ) +   y 3 × CW ( APP [ Cl , S ] ) + y 4 × CW ( APP [ Cl , P ] )
DCW δ ( T , N ) = i j = i + 1 M ij M = F Cl Br N O S P = # F a 12 a 13 a 14 a 15 a 16 a 17 a 18 a 19 Cl a 23 a 24 a 25 a 26 a 27 a 28 a 29 Br a 34 a 35 a 36 a 37 a 38 a 39 N a 45 a 46 a 47 a 48 a 49 O a 56 a 57 a 58 a 59 S a 67 a 68 a 69 P a 78 a 79 = a 89 #
where a 12 = CW ( APP [ F , Cl ] ) ;   a 13 = CW ( APP [ F , Br ] ) ;   ;   a 89 = CW ( APP [ = , # ] ) . Symbols ‘=’ and ‘#’ are indicators of the double and triple covalent bonds. Some examples of the atom-pairs proportions (APP) for APP[Cl,N] can be the following:
  • (Cl.N)..1.1.A molecule contains one chlorine atom and one nitrogen atom;
  • (Cl.N)..1.2.A molecule contains one chlorine atom and two nitrogen atoms;
  • (Cl.N)..2.1.A molecule contains two chlorine atoms and one nitrogen atom.
The others APP attributes are defined in the same manner. Table 4 contains the types of attributes involved in the models with corresponding references.
The correlation weights were based on the index of ideality of correlation (IIC) and on the correlation intensity index (CII), both of which were calculated using the compounds in the calibration set, as described in the literature [14,41].
Table 5 shows details of the eight models developed. Screenshots of the methods M1–M8, utilized to build up the models, are shown in Figures S9–S16 (Supplementary Materials).

4.5. Definition of the Applicability Domain

The applicability domain (AD) implemented in CORAL is based on a probabilistic measure defect SMILES , which measures the quality of the molecular features extracted from the compound [33,36]. It is defined according to the distribution of the molecular features (Sk) extracted from SMILES or HSG in the active training and calibration sets. The defect of Sk was calculated as:
d k = | P ( S k ) P ( S k ) | N ( S k ) + N ( S k )
where P(Sk) and N(Sk) are, respectively, the probability and frequency of Sk in the active training set. P′(Sk) and N′(Sk) are, respectively, the probability and frequency of Sk in the calibration set.
The defect of the SMILES was calculated as:
defect SMILES = k = 1 NA d k
where NA is the number of non-blocked molecular features extracted from SMILES or HSG.
The criterion for the reliability of a prediction is defined as below in Equation (13):
defect SMILES < 2 × D ¯
where D ¯ is the average of the statistical SMILES-defect for the compounds in the active training set [33]. Compounds having defect SMILES greater than the calculated threshold 2 × D ¯ were considered outside the AD. In contrast to a traditional interpretation of the applicability domain, this approach provides a semi-qualitative criterion, named the statistical defect. Indeed, this is a measure of the presence of the molecular features observed in SMILES. If the majority of the molecular features are rare, then the SMILES have a large defect, while if the majority of the molecular features are represented many times in both the training set and in the calibration set, the defect is small. Tables S9−S24 in the Supplementary Materials report the statistical quality and the calculated threshold for the models applying the AD procedure described.

5. Conclusions

The present study provides new QSAR regression models for NOAEL and LOAEL, in particular, new models for NOAEL and LOAEL for the liver, kidney and brain. Considering the difficulty to obtain a robust model for these endpoints, predictions are quite satisfying, particularly for the organ-specific models. This is encouraging, with a view to a future integrated approach, where the prediction of NOAEL and LOAEL can be used to support a new multi-tier assessment strategy. Particularly convincing are the results for the liver, an organ of central importance for the risk assessment of various kinds of chemicals. These QSAR models will be implemented in the VEGA platform (, accessed on 2 December 2021). The VEGA system will also show the six most similar substances and an evaluation of the reliability of the predictions. They have the important advantage of being freely available for the end users. Therefore, they can be used by the scientific community, worldwide, to support their activities. These models can be used for a hazard characterization of chemicals as well as for screening, the prioritization of chemicals and as supporting evidence for a risk-assessment approach.
Moreover, these models offer an innovative perspective for the risk-assessment tools developed by the authors and their colleagues, in the context of the LIFE VERMEER project (, accessed on 3 May 2022).
Finally, as we also previously highlighted within the text, some limitations were present in this work due to the intricacy of the experimental design, the dependency of the dose administered and the complexity of the toxicity mechanisms behind the adverse effects. It is worth underlining that, in the future, these models could be refined, applying the concept of the BMD, which is more reliable than NOAEL, since it is less influenced by the sample size and selection of the treatment dose. Moreover, with the awareness of their importance to refine the prediction of repeated-dose toxicity, a deeper evaluation of relevant issues, such as the role of the metabolism and mechanism of actions, will be taken into account in order to increase the work we are doing to provide advanced strategies for risk assessment and regulatory purposes.

Supplementary Materials

The following supporting information can be downloaded at:

Author Contributions

Conceptualization, G.S. and G.J.L.; methodology, G.S., G.J.L., A.P.T. and A.A.T.; software, G.J.L., A.P.T. and A.A.T.; validation, G.S. and G.J.L.; resources, G.S., M.M. and D.B.; data curation, G.S.; writing—original draft preparation, G.S., A.P.T., A.A.T. and G.J.L.; writing—review and editing, G.S., G.J.L., D.G. and E.B.; visualization, G.J.L. and D.B.; supervision, E.B.; project administration, E.B.; funding acquisition, E.B. All authors have read and agreed to the published version of the manuscript.


This research was funded by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 963845 (ONTOX).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available in Supplementary Materials.

Conflicts of Interest

The authors declare no conflict of interest.


  1. European Commission. Regulation (EC) No.1223/2009 of the European Parliament and of the Council of 30 November 2009 on Cosmetic Products. Off. J. Eur. Union 2009, 342, 59–209. [Google Scholar]
  2. Selvestrel, G.; Robino, F.; Baderna, D.; Manganelli, S.; Asturiol, D.; Manganaro, A.; Russo, M.Z.; Lavado, G.; Toma, C.; Roncaglioni, A.; et al. SpheraCosmolife: A new tool for the risk assessment of cosmetic products. ALTEX Altern. Anim. Exp. 2021, 38, 565–579. [Google Scholar] [CrossRef] [PubMed]
  3. European Commission. Regulation (EC) No 1907/ 2006 of the European Parliament and of the Council of 18 December 2006 concerning the Registration, Evaluation, Authorization and Restriction of Chemicals (REACH), Establishing a European Chemicals Agency, amending Directive 1999/45/EC and repealing Council Regulation (EEC) No 793/93 and Commission Regulation (EC) No 1488/94 as well as Council Directive 76/769/EEC and Commission Directives 91/155/EEC, 93/67/EEC, 93/105/EC and 2000/21/EC. Off. J. Eur. Union 2006, 1–849. [Google Scholar]
  4. SCCS. Scientific Committee on Consumer Safety (2021). SCCS Notes of Guidance for the Testing of Cosmetic Ingredients and Their Safety Evaluation 11th Revision; European Commission: Luxembourg, 2021; Volume SCCS/1628/21. [Google Scholar]
  5. Gadaleta, D.; Nicolotti, O. A k-NN algorithm for predicting oral sub-chronic toxicity in the rat. ALTEX Altern. Anim. Exp. 2014, 31, 4. [Google Scholar] [CrossRef][Green Version]
  6. Gadaleta, D.; Marzo, M.; Toropov, A.A.; Toropova, A.; Lavado, G.J.; Escher, S.E.; Dorne, J.L.C.M.; Benfenati, E. Integrated In Silico Models for the Prediction of No-Observed-(Adverse)-Effect Levels and Lowest-Observed-(Adverse)-Effect Levels in Rats for Sub-chronic Repeated-Dose Toxicity. Chem. Res. Toxicol. 2020, 34, 247–257. [Google Scholar] [CrossRef]
  7. Galli, C.L.; Corsini, E.; Marinovich, M. Tossicologia, 3rd ed.; Piccin-Nuova Libraria: Padova, Italy, 2016. [Google Scholar]
  8. Sand, S.; Victorin, K.; Filipsson, A.F. The current state of knowledge on the use of the benchmark dose concept in risk assessment. J. Appl. Toxicol. 2008, 28, 405–421. [Google Scholar] [CrossRef] [PubMed]
  9. Pizzo, F.; Benfenati, E. In Silico Models for Repeated-Dose Toxicity (RDT): Prediction of the No Observed Adverse Effect Level (NOAEL) and Lowest Observed Adverse Effect Level (LOAEL) for Drugs. In In Silico Methods for Predicting Drug Toxicity, 1st ed.; Benfenati, E., Ed.; Springer: New York, NY, USA, 2016; pp. 163–176. [Google Scholar] [CrossRef]
  10. Toropov, A.A.; Toropova, A.P.; Pizzo, F.; Lombardo, A.; Gadaleta, D.; Benfenati, E. CORAL: Model for no observed adverse effect level (NOAEL). Mol. Divers. 2015, 19, 563–575. [Google Scholar] [CrossRef][Green Version]
  11. Toropova, A.P.; Toropov, A.A.; Marzo, M.; Escher, S.E.; Dorne, J.L.; Georgiadis, N.; Benfenati, E. The application of new HARD-descriptor available from the CORAL software to building up NOAEL models. Food Chem. Toxicol. Int. J. Publ. Br. Ind. Biol. Res. Assoc. 2018, 112, 544–550. [Google Scholar] [CrossRef]
  12. Gramatica, P.; Sangion, A. A Historical Excursus on the Statistical Validation Parameters for QSAR Models: A Clarification Concerning Metrics and Terminology. J. Chem. Inf. Model. 2016, 56, 1127–1131. [Google Scholar] [CrossRef]
  13. Roy, K.; Kar, S. Chapter 3—How to Judge Predictive Quality of Classification and Regression Based QSAR Models? In Frontiers in Computational Chemistry; Ul-Haq, Z., Madura, J.D., Eds.; Bentham Science Publishers: Sharjah, United Arab Emirates, 2015; pp. 71–120. [Google Scholar] [CrossRef]
  14. Toropova, A.P.; Toropov, A.A. Does the Index of Ideality of Correlation Detect the Better Model Correctly? Mol. Inform. 2019, 38, 1800157. [Google Scholar] [CrossRef][Green Version]
  15. Pizzo, F.; Gadaleta, D.; Benfenati, E. In Silico Models for Repeated-Dose Toxicity (RDT): Prediction of the No Observed Adverse Effect Level (NOAEL) and Lowest Observed Adverse Effect Level (LOAEL) for Drugs. In In Silico Methods for Predicting Drug Toxicity, 2nd ed.; Benfenati, E., Ed.; Springer: New York, NY, USA, 2022; pp. 241–258. [Google Scholar] [CrossRef]
  16. Berggren, E.; White, A.; Ouedraogo, G.; Paini, A.; Richarz, A.-N.; Bois, F.Y.; Exner, T.; Leite, S.; van Grunsven, L.A.; Worth, A.; et al. Ab initio chemical safety assessment: A workflow based on exposure considerations and non-animal methods. Comput. Toxicol. 2017, 4, 31–44. [Google Scholar] [CrossRef] [PubMed][Green Version]
  17. Cronin, M.T.D.; Enoch, S.J.; Mellor, C.L.; Przybylak, K.R.; Richarz, A.-N.; Madden, J.C. In Silico Prediction of Organ Level Toxicity: Linking Chemistry to Adverse Effects. Toxicol. Res. 2017, 33, 173–182. [Google Scholar] [CrossRef] [PubMed][Green Version]
  18. ICCR. Integrated Strategies for Safety Assessment of Cosmetic Ingredients: Part 2. 2018. Available online: (accessed on 23 July 2021).
  19. Russell, W.M.S.; Burch, R.L. The Principles of Humane Experimental Technique. Johns Hopkins Bloom. Sch. Public Health 1959. Available online: (accessed on 23 July 2021).
  20. Worth, A.; Barroso, J.; Bremer, S.; Burton, J.; Casati, S.; Coecke, S.; Corvi, R.; Desprez, B.; Dumont, C.; Gouliarmou, V.; et al. Alternative Methods for Regulatory Toxicology: A State of the Art Review. LU: Publications Office of the European Union. Joint Research Centre Institute for Health and Consumer Protection, Ispra (Italy). 2014. Available online: (accessed on 23 July 2021).
  21. Benfenati, E.; Manganaro, A.; Gini, G. VEGA-QSAR: AI inside a platform for predictive toxicology. CEUR Workshop Proc. 2013, 1107, 21–28. [Google Scholar]
  22. OECD. Test No. 408: Repeated Dose 90-Day Oral Toxicity Study in Rodents; OECD Publishing: Paris, France, 2018; Available online: (accessed on 15 September 2021).
  23. Yang, C.; Barlow, S.M.; Jacobs, K.L.M.; Vitcheva, V.; Boobis, A.R.; Felter, S.P.; Arvidson, K.B.; Keller, D.; Cronin, M.T.; Enoch, S.; et al. Thresholds of Toxicological Concern for cosmetics-related substances: New database, thresholds, and enrichment of chemical space. Food Chem. Toxicol. 2017, 109, 170–193. [Google Scholar] [CrossRef]
  24. Watford, S.; Pham, L.L.; Wignall, J.; Shin, R.; Martin, M.T.; Friedman, K.P. ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology analyses. Reprod. Toxicol. 2019, 89, 145–158. [Google Scholar] [CrossRef]
  25. Dorne, J.; Richardson, J.; Livaniou, A.; Carnesecchi, E.; Ceriani, L.; Baldin, R.; Kovarich, S.; Pavan, M.; Saouter, E.; Biganzoli, F.; et al. EFSA’s OpenFoodTox: An open source toxicological database on chemicals in food and feed and its future developments. Environ. Int. 2021, 146, 106293. [Google Scholar] [CrossRef]
  26. Kovarich, S.; Ceriani, L.; Ciacci, A.; Baldin, R.; Perez Miguel, M.; Gibin, D.; Carnesecchi, E.; Roncaglioni, A.; Mostrag, A.; Tarkhov, A.; et al. OpenFoodTox: EFSA’s chemical hazards database. Zenodo 2020. [Google Scholar] [CrossRef]
  27. Escher, S.E.; Batke, M.; Hoffmann-Doerr, S.; Messinger, H.; Mangelsdorf, I. Interspecies extrapolation based on the RepDose database—A probabilistic approach. Toxicol. Lett. 2013, 218, 159–165. [Google Scholar] [CrossRef]
  28. Berthold, M.R.; Cebron, N.; Dill, F.; Gabriel, T.R.; Kötter, T.; Meinl, T.; Ohl, P.; Sieb, C.; Thiel, K.; Wiswedel, B. KNIME: The Konstanz Information Miner. In Data Analysis, Machine Learning and Applications; Springer: Berlin/Heidelberg, Germany, 2008; pp. 319–326. [Google Scholar]
  29. Gadaleta, D.; Lombardo, A.; Toma, C.; Benfenati, E. A new semi-automated workflow for chemical data retrieval and quality checking for modeling applications. J. Cheminform. 2018, 10, 60. [Google Scholar] [CrossRef][Green Version]
  30. Ambure, P.; Gajewicz-Skretna, A.; Cordeiro, M.N.D.S.; Roy, K. New Workflow for QSAR Model Development from Small Data Sets: Small Dataset Curator and Small Dataset Modeler. Integration of Data Curation, Exhaustive Double Cross-Validation, and a Set of Optimal Model Selection Techniques. J. Chem. Inf. Model. 2019, 59, 4070–4076. [Google Scholar] [CrossRef]
  31. DTC. QSAR Modelling—Manual Small Dataset Curator and Small Dataset Modeler v. 1.0.0. 2020. Available online: (accessed on 13 February 2021).
  32. Kode, K.-C. Kode—Chemoinformatics. 2020. Available online: (accessed on 4 May 2021).
  33. Toropova, A.P.; Toropov, A.A.; Benfenati, E.; Rallo, R.; Leszczynska, D.; Leszczynski, J. Development of Monte Carlo Approaches in Support of Environmental Research. In Advances in QSAR Modeling; Springer: Cham, Switzerland, 2017; Volume 24, pp. 453–469. [Google Scholar] [CrossRef]
  34. Toropova, A.P.; Toropov, A.A. Hybrid optimal descriptors as a tool to predict skin sensitization in accordance to OECD principles. Toxicol. Lett. 2017, 275, 57–66. [Google Scholar] [CrossRef] [PubMed]
  35. Lavado, G.J.; Baderna, D.; Carnesecchi, E.; Toropova, A.P.; Toropov, A.A.; Dorne, J.L.C.; Benfenati, E. QSAR models for soil ecotoxicity: Development and validation of models to predict reproductive toxicity of organic chemicals in the collembola Folsomia candida. J. Hazard. Mater. 2022, 423, 127236. [Google Scholar] [CrossRef] [PubMed]
  36. Manganelli, S.; Benfenati, E.; Manganaro, A.; Kulkarni, S.; Barton-Maclaren, T.S.; Honma, M. New Quantitative Structure–Activity Relationship Models Improve Predictability of Ames Mutagenicity for Aromatic Azo Compounds. Toxicol. Sci. 2016, 153, 316–326. [Google Scholar] [CrossRef] [PubMed]
  37. Toropov, A.A.; Toropova, A.P.; Benfenati, E.; Gini, G.; Puzyn, T.; Leszczynska, D.; Leszczynski, J. Novel application of the CORAL software to model cytotoxicity of metal oxide nanoparticles to bacteria Escherichia coli. Chemosphere 2012, 89, 1098–1102. [Google Scholar] [CrossRef]
  38. Toropov, A.; Toropova, A.; Martyanov, S.; Benfenati, E.; Gini, G.; Leszczynska, D.; Leszczynski, J. Comparison of SMILES and molecular graphs as the representation of the molecular structure for QSAR analysis for mutagenic potential of polyaromatic amines. Chemom. Intell. Lab. Syst. 2011, 109, 94–100. [Google Scholar] [CrossRef]
  39. Toropov, A.A.; Toropova, A.P. Modeling of acyclic carbonyl compounds normal boiling points by correlation weighting of nearest neighboring codes. J. Mol. Struct. THEOCHEM 2002, 581, 11–15. [Google Scholar] [CrossRef]
  40. Toropova, A.P.; Toropov, A.A.; Benfenati, E. The self-organizing vector of atom-pairs proportions: Use to develop models for melting points. Struct. Chem. 2021, 32, 967–971. [Google Scholar] [CrossRef]
  41. Toropov, A.A.; Toropova, A.P. Correlation intensity index: Building up models for mutagenicity of silver nanoparticles. Sci. Total Environ. 2020, 737, 139720. [Google Scholar] [CrossRef]
Figure 1. Histograms of ToxPrint chemotypes identified in the (a) general, (b) kidney, (c) brain and (d) liver datasets.
Figure 1. Histograms of ToxPrint chemotypes identified in the (a) general, (b) kidney, (c) brain and (d) liver datasets.
Ijms 23 06615 g001
Table 1. The statistical characteristics of the models.
Table 1. The statistical characteristics of the models.
active training1400.510.670.640.50 0.720.60142
passive training1400.510.690.650.50 0.810.69142
validation 1410.550.690.56 0.530.650.45172
active training1420.550.710.700.54 0.660.50173
passive training1420.550.680.510.53 0.720.57168
validation 1370.530.630.59 0.320.720.53149
active training950.500.670.670.48 0.560.4894
passive training950.520.540.570.51 0.730.64103
validation 450.590.750.70 0.840.330.2762
active training970.510.680.620.49 0.510.4099
passive training1020.590.580.720.57 0.520.44143
validation 380.690.820.66 0.810.310.2580
active training230.550.710.570.46 0.670.5926
passive training220.610.420.640.52 0.750.6831
validation 220.530.670.68 0.810.350.2823
active training220.540.700.510.44 0.610.5523
passive training230.690.420.830.61 0.910.8248
validation 230.690.800.80 0.820.330.2646
active training970.760.860.720.75 0.380.29297
passive training940.730.830.780.72 0.440.34247
validation 310.550.730.54 0.550.540.3835
active training960.780.870.780.77 0.320.23326
passive training950.780.810.650.77 0.400.32323
validation 310.610.710.61 0.550.480.4046
Table 2. Comparison of QSAR models for NOAEL and LOAEL in rats for sub-chronic repeated-dose toxicity (90 days exposure) in the literature.
Table 2. Comparison of QSAR models for NOAEL and LOAEL in rats for sub-chronic repeated-dose toxicity (90 days exposure) in the literature.
NOAEL This work
active training1400.510.500.67 0.720.60142
passive training1400.510.500.69 0.810.69142
validation 1410.55 0.690.530.650.45172
validation in AD (77%)1090.61 0.740.690.520.38171
LOAEL This work
active training1420.550.540.71 0.660.50173
passive training1420.550.530.68 0.720.57168
validation 1370.53 0.630.320.720.53149
validation in AD (74%)1020.51 0.640.550.580.45105
training1240.570.56 0.680.52164
invisible training1260.500.49 0.770.59125
calibration380.610.55 0.670.4956
validation 380.65 0.680.5268
validation in AD (87%)330.69 0.580.43
training1240.550.53 0.660.51147
invisible training1260.450.43 0.800.63102
calibration380.610.51 0.690.5149
validation 380.59 0.720.5453
validation in AD (87%)330.62 0.590.44
NOAEL [10]
training970.530.51 0.610.47107
Test160.730.67 0.490.3738
validation270.60 0.430.3638
validation in AD (96%)260.61 0.42
Table 3. Datasets examined.
Table 3. Datasets examined.
IDModelInitial Number of CompoundsFinal Number of CompoundsFinal Range Value mg/kg bw/Day (log)
M1General NOAEL573561−1 to 3.591
M2General LOAEL 573558−0.415 to 3.948
M3Kidney NOAEL2892800.097 to 4.301
M4Kidney LOAEL2892830.097 to 3.585
M5Brain NOAEL91900.248 to 4.301
M6Brain LOAEL91900.248 to 4.301
M7Liver NOAEL353252−0.432 to 3.499
M8Liver LOAEL353252−0.415 to 3.789
Table 4. SMILES and molecular graph attributes.
Table 4. SMILES and molecular graph attributes.
S, SS, SSSSingle SMILES atom, a combination of two SMILES atoms and a combination of three SMILES atoms [36,37].
EC1, EC2, EC3Morgan connectivity of first-, second- and third-order [38].
NNCNearest neighbors codes [33,39].
APPAtom pair proportions weighted presence of F, Cl, Br, N, O, S, P, #, = [40].
Table 5. Descriptors applied to build the models M1–M8 listed in Table 3 (+).
Table 5. Descriptors applied to build the models M1–M8 listed in Table 3 (+).
(+) α, β, γ, δ are DCW indices; x1,…,y4 are equation constants; IICw, CIIw are the correlation index weights; T* is the best threshold and N* is the best number of epochs.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Selvestrel, G.; Lavado, G.J.; Toropova, A.P.; Toropov, A.A.; Gadaleta, D.; Marzo, M.; Baderna, D.; Benfenati, E. Monte Carlo Models for Sub-Chronic Repeated-Dose Toxicity: Systemic and Organ-Specific Toxicity. Int. J. Mol. Sci. 2022, 23, 6615.

AMA Style

Selvestrel G, Lavado GJ, Toropova AP, Toropov AA, Gadaleta D, Marzo M, Baderna D, Benfenati E. Monte Carlo Models for Sub-Chronic Repeated-Dose Toxicity: Systemic and Organ-Specific Toxicity. International Journal of Molecular Sciences. 2022; 23(12):6615.

Chicago/Turabian Style

Selvestrel, Gianluca, Giovanna J. Lavado, Alla P. Toropova, Andrey A. Toropov, Domenico Gadaleta, Marco Marzo, Diego Baderna, and Emilio Benfenati. 2022. "Monte Carlo Models for Sub-Chronic Repeated-Dose Toxicity: Systemic and Organ-Specific Toxicity" International Journal of Molecular Sciences 23, no. 12: 6615.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop