Should We Expect a Second Wave of AlphaFold Misuse After the Nobel Prize?

AlphaFold (AF) was the first deep learning tool to achieve exceptional fame in the field of biology [1]. To sum up, we first recall the existence of the CASP (Critical Assessment of Structural Prediction) competition, which allows the evaluation of individual prediction methods by proposing protein structural models. In 2018, the first version of the AF obtained excellent results, close to those of the best approaches available at the time [2,3]. Two years later, in 2020, a particularly significant average improvement was observed [4,5], and then with the communicative power of a company spun off from Alphabet, a great increase in media coverage of structural bioinformatics occurred.

Specialists in the field have had to analyse this phenomenon [6,7,8,9]. An essential aspect of this second version of AF is that all the tools were made available to the scientific community [10]. A large number of laboratories could therefore install and use it autonomously. This particularly positive aspect was further enhanced by the EBI’s AF2 database [11], which was initially dedicated to complete proteomes and later included hundreds of millions of sequences [12]. Moreover, for proteins whose structural models were not already available, online versions such as ColabFold could be employed to build a structural model using a simple web browser [13].

Experts in the field have also had to adapt to benefit from the real-world use of AF applications [14,15,16,17,18,19]. In fact, I discovered in discussions with non-specialists (repeating scientific or often non-scientific articles without hindsight) that they have very often assumed that since the protein-folding problem had been solved, all structures were available (I would like to point out two major problems in this sentence: (i) the term protein folding is wrong and should be protein fold, and (ii) the fact that we regularly use the term ‘predicted structures’ instead of ‘structural models’ means that many people now use the term ‘structures’ for AF models in a problematic and ambiguous way).

This last point rendered experimental research almost useless and other predictive approaches obsolete. To repeat what a researcher (who shall remain nameless) told me in a conversation 2 months after identifying the CASP results, ‘If I have a protein sequence, I, the non-specialist, can always get a nice visualisation of a real structure without tiring myself. The field of protein structural biology research is definitely over’.

Unfortunately, reality is more complicated. The DeepMind group previously pointed out in their articles that only 1/3 of the residues were of atomistic quality, and this figure only increased to 58% for correct folding; thus, for 42%, local quality could not be assured [11]. Secondly, an independent consortium clearly showed that for human proteome, for example, AF2 increased the number of proteins that could be used to propose a structural model of interest by 10% compared to a comparative modelling approach [20]. Finally, technical articles have highlighted practical problems related to unsuccessful AF results. Personally, I support the conclusions of the short article by Dame Janet Thornton, which gives examples of the positives and negatives of structural models, who included an impressive final example. [21]

For my own work, in recent proposals for structural models related to blood group issues, I have used approaches varying from classical comparative modelling to deep learning methods. In the end, I retained the models that I built manually [22,23,24]. am interested in are transmembrane proteins, so this was not an unreasonable approach. Similarly, in a more systematic way, the analysis of the whole human proteome showed systematic local biases for certain local conformations, such as γ-turns, especially cis ω-angle conformations [25]. Another underlying problem was the incomplete nature of the β-strands that form the β-sheets, so there is still room for improvement [25,26,27].

As I have edited and reviewed numerous manuscripts since the advent of this approach, I have had to calm the enthusiasm of non-specialists who have used the AF tool incorrectly. I would like to anonymously transcribe a heated exchange with researchers who had used AF models in conjunction with experiments. The evaluation was not successful because they had used two structural models with confidence limits (pLDDT) of less than 30, i.e., a very simple case to analyse. Their response was that their models could not be criticized because they had been designed by AF2. They had not understood that though AF does represent a methodological revolution, signified by the recent Nobel Prize [28], it only represents an evolution in terms of application.

This recent and well-deserved award brings focus back phrases such as protein folding problem solved, 50 year old problem solved, etc. This renewed focus will lead us to rethink the critical evaluation of structural models. The very user-friendly aspects of tools such as ColabFold, for example, often makes non-specialists forget that a protein is often be in complex with other proteins. They also pose the following problems:

• Alternative splicing events (not taken into account if the correct sequence is not given);
• Post-translational modifications essential for function but which may be poorly known, poorly determined, reversible, etc., and are not taken into account by AF, occur;
• They often require ions, cofactors and partners that can be very complex to introduce (the latest version of AF remains very limited);
• A non-negligible part of the protein structure is disordered and therefore difficult to take into account, even when using by AF;
• A fixed structure with a nice visual appearance represents only part of the structural conformations of the protein of interest, especially in terms of conformers.

In conclusion, the structural bioinformatics community was pleased to be recognised with these new Nobel Prizes, following on from those awarded to Martin Karplus, Michael Levitt, and Arieh Warshel in 2013 [29], but once again we need to point out to non-specialists that there are always limitations, as we lack precise experimental data in a significant number of cases.