As mentioned above, two distinct sets of descriptors were tested to describe the 93 chemical compounds respectively (Table 1 and Table 2). For the self-fitting of training data (highlighted in red), we found that the models derived from physical properties are more efficient than those derived from topological indices for QSAR modeling. It can be seen that almost all the values of σ in this case are negative. However, with regard to independent testing (highlighted in royal blue), it seems that QSAR models derived from the DLI descriptors [17] are much more robust than those derived from general descriptors [18], and in this case almost all the values σ are positive. As an intermediate state, the values of σ derived from cross validation (highlighted in yellow-green) contain several negative and positive ones respectively. In total, the above mentioned result indicated that when projecting the connection table information into physical properties, the general descriptors will lose some structural information of a compound. Such loss of information is different for training and testing datasets since this information is highly dependent on the conformation and structural essence of a molecule.

In conclusion, models derived from DLI are much more stable for both training data and testing data, while general descriptors cannot guarantee such stability and scale in independent data.

It is normally known that QSAR predictions are only reliable within or near the property space used to train the model. Preparing a robust, unbiased and sufficiently large training set is critically important for the building of a proper statistical model. As mentioned above, two data division methods, i.e., Diverse Subset and Cluster plus Diverse Set were applied to divide our dataset into training set and testing set.

In order to statistically reveal the difference between the results influenced by two such kinds of data divisions, pair t-test was performed and the p-value derived from the above two tables (Table 1 and Table 2) was 0.88 (>0.05), which indicates that there is no significant statistical difference for these two data divisions for QSAR analysis. Our result has shown that clustering data before calculating the diverse set does not produce a significant influence on the QSAR models. This result was explained by analysis of the detailed algorithm in calculating the diverse set as follows: The Diverse Subset method used in MOE [19] ranks entries based on the whole dataset diversity, that is, the calculation of Diverse Subset itself is a global diversity comparison procedure. For the Cluster plus Diverse Set method, although an extra preprocess of clustering data exists, Diverse Subset still happens within every sub-cluster and the main difference, compared with the former, is that calculating diverse subset becomes a local procedure based on each clustering. It can be seen that essentially the two division methods have little influence on the final distribution of training data and testing data. Thus, as expected in our results, no significant differences for the results of these two division methods exist.

When building a QSAR model, linear regression methods are normally preferred to the advanced non-linear methods, since the linear models are easier to use for a physical explanation of the prediction results. The most classical liner model in QSAR is PLS, which have been widely used in popular computer-aided drug design software [19–21]. In our study, PLS (MOE-PLS) was first chosen to derive our QSAR models. However, as indicated in Tables 1 and 2, this linear model failed to achieve satisfactory results in QSAR study. The correlation coefficients from self-fitting testing and cross validation testing are all less than 0.65.

Since advanced machine learning methods such as ANN [22], Bayesian inference [23], Random Forest [24] and SVM [25] have been successfully applied in QSAR study [26–36], our QSAR models were rebuilt using the SVR method, which is a derived regression model with powerful fitting ability as well as excellent prediction accuracy [36–39]. In anticipating results, this method behaved well in the self-fitting testing of our training data (R2 is nearly 0.9) as well as in the cross-validation testing. Nevertheless, this method still performed badly in the independent test data, which indicates that such machine learning methods may not be generalized enough in the cyclopamine data. This is probably due to the fact that a substantial diversity exists in our dataset. Among the 93 data, four different scaffolds were found (Figure 1). In addition, there were still six molecules that did not match any of the scaffolds (Figure 2).

When the qualities of the data or the underlying mechanism are not suitable for regression modeling, the binary classification was applied on the data to uncover their probabilities to be active or inactive. MOE has offered a binary filter to filtering the numerical data. Any properties which can be represented in a binary (yes/no) way (like active/inactive, toxic/non-toxic, drug-like/non-drug-like, permeable/non-permeable, etc.) could be mapped onto such a filter. Thus, the binary classification model was used to rebuild the QSAR models to further reveal their intrinsic characteristics. MOE’s binary filters (yes/no) are based on the Bayesian inference technique as mentioned in Material and Methods. Continuous activity data (non-binary) can be transferred to binary values with a specific threshold criterion. In our study the IC50 of the drug compound is used as a cut-off.

As shown in Table 1 and Table 2, the binary model behaved well on both training data and testing data sets. The overall prediction accuracy is improved to nearly 0.8 against NCI-H446 cell line. (Some were up to 0.906). This result has indicated that the binary QSAR classification model is more suitable to guide the direction of designing novel inhibitors of Hedgehog signal pathway.

The SVM classification was also applied to further validate the efficiency of binary classification models compared with regression models. The results shown in Table 1 and Table 2 reconfirmed that for our data the binary classification model is probably more suitable for QSAR analysis.

Four different cell lines (NCI-H446, NCI-H157, SW1990 and BxPC-3) were used to test the cytotoxicity of the 93 compounds. However, only the data of NCI-H446 can produce a reasonable model by QSAR analysis; the prediction accuracy of the models against all the other cell lines is about 0.6.

Why do some specific cell lines not fit well to our QSAR analysis? We speculate that the most likely reason is the non-specific cytotoxicity effect of these compounds to the other three cell lines. For example, HCI-H157 and BxPC-3 do not express the Gli and Smoothened protein, respectively [40,41]. That means that the cytotoxicity effect of these compounds may not directly result from the inhibition of hedgehog signaling. In addition, although sustained hedgehog signaling activity can be detected in SW1990 cells [41], it is very likely that cell lines grown in vitro may lose their dependence on hedgehog signaling for survival [42]. For example, the IC50 of positive compound (cyclopamine) is 9.13 μg/mL for NCI-H446, 38.11 μg/mL for BxPC-3, 61.05 μg/mL for SW1990 and 58.33 μg/mL for NCI-H157. That is to say, firstly, HCI-H466 cells were most sensitive to the hedgehog signaling inhibitor. In addition, the SW1990 possibly mutated and lost the hedgehog signaling in our experiment. In summary, the non-specific effects may result in the variance of the data of the cytotoxicity and finally affect the QSAR analysis.

In our study, SAReport was applied to present a direct instruction on how to modify the structure of a compound and make it a better inhibitor of hedgehog signal pathway. All the structure modifications are listed in the supplementary material. Here the top three structures were selected with their activity improvements according to different modification mechanisms.

The first important finding is that through such SAReport we validated our former finding that only the data to cell line NCI-H446 can obtain a reasonable QSAR modeling result (indicated in Figure 3). Secondly, our SAReport has shown that demethylation, methylation and hydroxylation at a specific position of the inhibitor scaffold may highly improve their activity. As indicated in Figure 3, demethylation at position 8, methylation at position 7 and hydroxylation at position 11 provided three possible ways to improve the inhibitor’s activity. In addition, the SAReport shows that demethylation seems to be the most efficient approach to improve activity among others. This conclusion provides the first proven set of efficient inhibitor structure modification methods in order to improve their activities. All these results will definitely shed new light on the future work of inhibitor synthesis.

93 cyclopamine derivatives together with their activities against four different cell lines (BxPC-3, NCI-H446, SW1990 and NCI-H157) were tested and are listed in the supplementary material.

Two different approaches were applied to divide these experimental data into training set and testing set for our following statistical modeling. Details followed.

There are lots of descriptors to describe a chemical compound, including constitutional descriptors, physiochemical property descriptors, electronic descriptors, topological indices, geometrical descriptors, and quantum chemistry descriptors, etc. However, no set of descriptors is capable of performing spectacularly better than the others. Thus, to build our QSAR model, the widely applicable set of descriptors, i.e., the general descriptors was selected. Also, DLI descriptors was adopted for a complementary comparison.

General descriptors include atomic contributions to van der Waals surface area, log P (octanol/water), molar refractivity and partial charge. These descriptors are applied to the construction of QSAR models for boiling point, vapor pressure, free energy of salvation in water, solubility in water, thrombin/trypsin/factor Xa activity, blood-brain barrier permeability and compound classification. The wide applications of these descriptors have suggested their important usage in the QSAR modeling, combinatorial library design and molecular diversity work.

On the other hand, DLI descriptors acts as an approach to measure drug-like compounds, as first presented by Xu et al. Then it was used and modified as a set of descriptors by MOE. These descriptors characterized the hierarchy of drug structures in terms of rings, links, and molecular frameworks.

Although these two sets of descriptors are both computable from connection table information, they partly complement each other. Normally, general descriptors have a preference for physical prosperities of compounds, while DLI descriptors favor simple topological indices of compounds.

In our computational framework, various statistical models were incorporated to evaluate their performance in QSAR analysis of inhibitors of Hedgehog signal pathway, and we wanted to find the most suitable statistical analysis method for the QSAR modeling of such data. Detailed descriptions of each statistical method are listed below.

SAReport [47] is an important tool for the visualization and analysis of project SAR data introduced by MOE recently. SAReport contains sophisticated analysis methods to help scientists identify important groups and make more effective choices for synthesis.

Briefly, the Suggestions table in SAReport consists of a list of hypothetical molecules, constructed from available pieces, which are predicted to have a high probability of activity. The pool of hypothetical molecules is prepared by enumerating all of the input molecules, and performing single-point mutations at each of the substitute positions, with each of the R-groups that have been observed in the equivalent position for some other molecule in the dataset. The unique list of chimerical molecules is then rated according to an estimate of probability, scaled and balanced to match the distribution of activities found in the input set. The scores are scaled in such a way that a value of 0 indicates that the hypothetical molecule is as likely to be active as an average molecule in the input set, while positive values are more likely. The chimerical molecules are ranked by their probability of activity, multiplied by a weighting factor, which is a measure of cumulative similarity to other molecules in the database. A higher weighting implies that a larger statistical base is available to make the prediction. The most promising candidates are listed first. The molecule from which the candidate was mutated is shown, along with its property information. The new structure is shown to the right, along with the prediction. The percentage value is the increased probability of activity, and the number in brackets is the weighting.