For the bound and unbound/bound test sets, 17 pockets were predicted for each protein on average with our geometry-based site finding method. The TOP1 and TOP3 accuracy differs for different ranking methods. The accuracy of the TOP 1 and TOP 3 in different individual property prediction ranking lists is listed in Table 1. A geometry-based method, SURFNET [8], with its own ranking algorithm is also tested for comparison. It is shown that ranking that presumes binding sites according to conservation score achieves the best performance with a 59% success rate in the top 1 prediction, which means that almost 124 of the 210 proteins in the bound test set are correctly predicted. Ranking with the criterion of “volume and distance from the protein centroid” (shown in the “Distance” column) also performs with better results, which may indicate that the size and the depth of the binding site could be helpful in ligand binding site prediction. However, we found that ranking according to the hydrophobic attribute does not deliver the expected results. We explain this by the fact that the description of hydrophobic properties in our study may be too simple.

In some cases the conservation profiles of proteins are not easily accessible, which may make it impossible to rank presumed binding sites by conservation score. In addition, there is an urgent need for developing an efficient approach to fully integrate various complementary ranking lists from a comprehensive multi-view perspective. Thus in our study, an innovative ranking aggregation method is further applied to address these problems. We integrate the ranking lists of different properties like the combination of “binding site size” and “the distance from the protein centroid”. The corresponding results are listed in Table 2. It is shown that after the ranking aggregation, most of the success rates are improved remarkably and some of them are comparable to the conservation ones. These results indicate that the combination of different individual complimentary properties will generally improve the prediction success rate. In addition, “Volume plus Distance” is found to be an alternative to “Conservation” when proteins with no conservation profiles are predicted. An example (PDB: 2SIM [23]) for such a kind of ranking aggregation is presented in Table 3. It can be seen that the ordering of the correctly predicted binding sites (*Pocket 9) is promoted after ranking aggregation, which leads to the improvement of the TOP 1 success rate. In Figure 1, the surface position of Pocket 9 is visualized with Jmol [24]. However, it is worth noting that when two or more properties that are not complementary are used, such as the information of volume and conservation, the final success rate probably does not show any improvement.

In summary, our study has not only validated the significance of sequence conservation in ligand binding site prediction, but also indicated the usefulness of the size and depth of the binding site in the ranking of binding sites. Furthermore, rather than only considering one property, an innovative multi-view ranking method was applied, which could achieve a much better performance for binding site prediction.

The four properties for the ranking of binding sites are calculated as follows:

The complementary properties listed above might be helpful in ranking presumed binding sites. Such an innovative ranking aggregation method was also applied in our previous study [29]. It is based on the equalitarian philosophical paradigm to seek a consensus list among individual ranking lists. Before defining the two distance measures, some necessary notations should be introduced. Let M_i(1), ···, M_i(k) be the scores associated with the ordered list L_i, where M_i(1) is the best score, M_i(2) is the second best one, and so on. Let r_Li(A) be the rank of A in the list L_i if A is within the top k, and otherwise equal to k + 1. The distance between two ranking lists can be defined as:

which is also named the Spearman’s footrule distance [30]. r^L_j (t) in equation (3) indicates the position of element t in the ordered list j.

In order to discover a comprehensive ranking list that would also be as close as possible to all the given ranking lists, an optimization function is defined:

where W_i is the importance weight of ranking list L_i. It is set to one in our study as we treat the four properties equally. Parameter d, which is calculated according to Spearman distance, represents the distance between the “comprehensive list” δ^* and L_i. The goal of the ranking aggregation is to find δ^* which minimizes the total distance between the “comprehensive list” and every ranking list. To accomplish this goal, the Cross-Entropy method (CE) [31] is used here, which is a general Monte Carlo approach for multi-extremum optimization. The CE algorithm requires users to set a number of parameters. It is recommended that the number of samples N for each stage is set to at least 10 k², and the rarity parameter ρ in the sampling stage of CE [31] used to update the cell probabilities is set to 0.01 when N is relatively large, and 0.1 when N is small (less than 100). All data are aggregated under R statistical environment with the RankAggreg package.

In this study, two datasets, i.e., the 210 bound structures and 48 unbound/bound structures, which are used to evaluate the LIGSITE^csc [16] algorithm are also used as a kind of unbound/bound and bound test set. To assess the quality of binding-site predictions, a standard evaluation method presented previously [4,6,9,16] is applied, which defines a prediction to be a met, if the geometric center of the presumed pocket lies within 4 Å to any atom of the ligand. Predictions that do not meet this criterion are excluded in the calculation of prediction success rates.

We also used another evaluation measurement, i.e., the Matthews Correlation Coefficient [32] (MCC) as a comparison. For each protein, residue predictions were classified as true positives (TP: correctly predicted binding site residues), true negatives (TN: correctly predicted nonbinding site residues), false negatives (FN: incorrectly predicted as nonbinding site residues), false positives (FP: incorrectly predicted as binding site residues). The MCC was computed using Equation 6:

For the bound and unbound/bound test sets, the MCC score for each protein can be calculated with a certain prediction method. In our implementation, different score can be calculated for different ranking methods. To determine the significant differences between different ranking methods as well as their combinations, the one-sided Wilcoxon signed ranked sum test is used based on MCC scores for each protein. The statistical evaluation is performed using R (version 2.15.0).

For the 210 bound structures, the evaluation is very straightforward and we will follow the above described routing procedure. For the unbound/bound dataset, the Biojava development package [33] is first used for the alignment of all the structures, and the ligands in the bound structures are mirrored to the corresponding unbound structures. Finally the predictions are performed for the unbound structures and then they are checked against the bound structures.