Role of Machine and Deep Learning in Predicting Protein Modification Sites: Review and Future Directions

Abstract

Post-translational modifications (PTMs) of proteins are essential for cellular function. Owing to the high cost and time demands of high-throughput sequencing, machine learning and deep learning methods are being rapidly developed for predicting PTM sites. This manuscript presents a comprehensive review of the current research on the application of intelligent algorithms for predicting PTM sites. It outlines the key steps for identifying modified sites based on intelligent algorithms, including data pre-processing, feature extraction, dimension reduction, and classifier development. This review also discusses potential future research directions in this field, providing valuable insights for advancing the state-of-the-art PTM site prediction. Collectively, this review provides comprehensive knowledge on PTM identification and contributes to the development of advanced predictors in the future.

1. Introduction

Protein synthesis follows the central dogma of genetics and involves three primary processes: replication, transcription, and translation. Protein post-translational modification (PTM) occurs during translation. After synthesis, proteins undergo various modifications, such as phosphorylation [1], acetylation [2], methylation [3], ubiquitination [4] and glycosylation [5]. These modifications expand the functional diversity of proteins and significantly increase their complexity. PTMs play crucial roles in cellular and organismal functions, affecting processes such as cell differentiation, apoptosis, protein degradation, protein–protein interactions, and gene expression. Furthermore, PTMs are closely linked to human diseases, and current targeted therapies involve the use of regulatory enzymes associated with these modifications. Therefore, studying protein PTMs is essential for advancing our understanding of biological processes.

Experimental methods are adept at accurately identifying protein modification sites, with mass spectrometry [6] as the predominant approach, complemented by liquid chromatography [7] and radiochemical methods [8]. However, as sequencing technologies continue to advance, an increasing number of protein sequences have been discovered, rendering traditional experimental methods insufficient for managing large datasets. In this context, computational methods have emerged as viable alternatives for analyzing protein sequences and identifying corresponding modification sites. Machine learning and deep learning methods have been successfully used for the identification of modification sites. However, current prediction methods require further improvements [9,10]. In addition, the modification site information provided by computational predictions is speculative, and its biological validity must be experimentally verified.

Machine learning identification methods typically involve six key steps: Initially, data were gathered from the established databases. The collected data were then preprocessed. Subsequently, sequence or structural features were extracted from the protein sequences, including sequence position information, amino acid physicochemical properties, and protein structure. Next, redundant or insignificant features were removed using feature-reduction methods. A suitable model was selected for training, and its performance was evaluated using a test set. Most machine learning identification methods adhere to this fundamental process, as illustrated in Figure 1. Although several articles have summarized the methods for identifying PTM sites [11], this study not only focuses on machine learning workflows to compare and summarize the methods used in each step, but also pays special attention to issues such as the data limitations of modification sites, interpretability of predictive models and features, multi-label prediction, and modification crosstalk. This study aimed to provide an overview of the existing problems and future challenges in this field.

2. Datasets and Data Pre-Processing

2.1. Dataset

With continuous advancements in sequencing technology and proteomics, researchers have developed numerous PTM databases that can be used by other researchers in the field. The following section provides an overview of several popular databases, with additional information in Table 1.

2.1.1. UniProt

UniProt [12,13] is a comprehensive database that offers protein structure, sequence information, functional annotations, Gene Ontology (GO) annotations, subcellular location data, PTM information, and similar proteins. The UniProt database is an authoritative repository of protein sequences and functional information that systematically integrates PTM annotations. After manual curation, these annotations are stored in Swiss-Prot entries and centrally displayed in the dedicated “PTM/Processing” module, covering various modification types, such as phosphorylation and glycosylation, with specific amino acid residue sites of modification clearly annotated. The data sources included published experimental evidence and reliable computational predictions, which are often linked to relevant literature. Moreover, PTM information does not exist in isolation but is deeply interconnected with modules such as “Function” “Disease and Variants” and “Sequence”, collectively elucidating the biological significance of PTMs in regulating protein activity, localization, interactions, and stability. Simultaneously, UniProt provides cross-references to specialized PTM databases, such as PhosphoSitePlus and GlyGen, serving as a comprehensive PTM information hub that guides users in their further exploration.

2.1.2. dbPTM

dbPTM [14,15] is a comprehensive resource for PTM studies. The database contains 2,235,664 experimental PTM sites and over 70 PTM types integrated into more than 40 databases, including 30 benchmark datasets. Additionally, dbPTM provides information on the association between modification sites and diseases, which can be valuable for disease research. Researchers can select specific modification sites for data download by clicking the download bar. Instead of providing the entire protein sequence, the database offers protein fragments with details of the modification site; each fragment is 21 amino acids in length.

2.1.3. CPLM 4.0

The Compendium of Protein Lysine Modifications 4.0 (CPLM 4.0) [16] is a comprehensive data resource that builds on previous versions of the CPLA [17], CPLM [18], and PLMD [19]. CPLM 4.0 focuses on the modification of lysine residues in proteins. This database includes a significant number of modification events that encompass a wide range of unique sites on various proteins. In total, 105,673 proteins were included in CPLM 4.0, with data pertaining to up to 29 different types of protein lysine modifications across 219 species [16].

Table 1. Commonly used datasets for PTM studies.

Name	Website	PTM Type	Statistics
UniProt [12,13]	https://www.uniprot.org/	Multiple	570,420 reviewed proteins, 251,131,639 unreviewed proteins
dbPTM [14,15]	https://biomics.lab.nycu.edu.tw/dbPTM/	Multiple	2,235,664 sites, 70+ PTM types, 40+ integrated databases, 30+ benchmark datasets
PhosphoSitePlus [20]	https://www.phosphosite.org/homeAction	Multiple	59,469 PTM sites, 13 PTM types
CPLM 4.0 [16]	http://cplm.biocuckoo.cn/	Multiple	463,156 unique sites of 105,673 proteins for up to 29 PLM types across 219 species
qPTM [21]	http://qptm.omicsbio.info/	Multiple	11,482,553 quantification events for 660,030 sites on 40,728 proteins under 2596 conditions
PupDB [22]	https://cwtung.kmu.edu.tw/pupdb/	Pupylation	268 pupylation proteins with 311 known pupylation sites and 1123 candidate pupylation proteins
DEPOD [23]	https://depod.bioss.uni-freiburg.de/	Phosphorylation	194 phosphatases have substrate data
O-GlcNAcAtlas [24]	https://oglcnac.org/atlas/	O-GlcNAcylation	16,877 Unambiguous sites, 10,058 ambiguous sites
Phospho.elm [25]	http://phospho.elm.eu.org/	Phosphorylation	42,914 instances, 11,224 sequences
CarbonylDB [26]	https://carbonyldb.missouri.edu/CarbonylDB/index.php/	Carbonylation	1495 proteins, 3781 PTM sites, 21 species
Scop3P [27]	https://iomics.ugent.be/scop3p/index	Phosphorylation	108,130 modifications, 20,394 proteins
O-GlycBase [28]	https://services.healthtech.dtu.dk/datasets/OglycBase/	O-Glycosylation	242 proteins
dbSNO [29]	http://140.138.144.145/~dbSNO/index.php	S-nitrosylation	174 experimentally verified S-nitrosylation sites on 94 S-nitrosylated proteins
UbiNet 2.0 [30]	https://awi.cuhk.edu.cn/~ubinet/index.php	Ubiquitination	3332 experimentally verified ESIs
UbiBrowser 2.0 [31]	http://ubibrowser.bio-it.cn/ubibrowser_v3/	Ubiquitination	1,884,676 predicted high-confidence ESIs, 8,341,262 potential E3 recognizing motifs, 4068 known ESIs from literature
PhosPhAt [32]	https://phosphat.uni-hohenheim.de/	Phosphorylation	10,898 phosphoproteins, 64,128 serine sites, 13,102 threonine sites, 2672 tyrosine sites

Table 1. Commonly used datasets for PTM studies.

Name	Website	PTM Type	Statistics
UniProt [12,13]	https://www.uniprot.org/	Multiple	570,420 reviewed proteins, 251,131,639 unreviewed proteins
dbPTM [14,15]	https://biomics.lab.nycu.edu.tw/dbPTM/	Multiple	2,235,664 sites, 70+ PTM types, 40+ integrated databases, 30+ benchmark datasets
PhosphoSitePlus [20]	https://www.phosphosite.org/homeAction	Multiple	59,469 PTM sites, 13 PTM types
CPLM 4.0 [16]	http://cplm.biocuckoo.cn/	Multiple	463,156 unique sites of 105,673 proteins for up to 29 PLM types across 219 species
qPTM [21]	http://qptm.omicsbio.info/	Multiple	11,482,553 quantification events for 660,030 sites on 40,728 proteins under 2596 conditions
PupDB [22]	https://cwtung.kmu.edu.tw/pupdb/	Pupylation	268 pupylation proteins with 311 known pupylation sites and 1123 candidate pupylation proteins
DEPOD [23]	https://depod.bioss.uni-freiburg.de/	Phosphorylation	194 phosphatases have substrate data
O-GlcNAcAtlas [24]	https://oglcnac.org/atlas/	O-GlcNAcylation	16,877 Unambiguous sites, 10,058 ambiguous sites
Phospho.elm [25]	http://phospho.elm.eu.org/	Phosphorylation	42,914 instances, 11,224 sequences
CarbonylDB [26]	https://carbonyldb.missouri.edu/CarbonylDB/index.php/	Carbonylation	1495 proteins, 3781 PTM sites, 21 species
Scop3P [27]	https://iomics.ugent.be/scop3p/index	Phosphorylation	108,130 modifications, 20,394 proteins
O-GlycBase [28]	https://services.healthtech.dtu.dk/datasets/OglycBase/	O-Glycosylation	242 proteins
dbSNO [29]	http://140.138.144.145/~dbSNO/index.php	S-nitrosylation	174 experimentally verified S-nitrosylation sites on 94 S-nitrosylated proteins
UbiNet 2.0 [30]	https://awi.cuhk.edu.cn/~ubinet/index.php	Ubiquitination	3332 experimentally verified ESIs
UbiBrowser 2.0 [31]	http://ubibrowser.bio-it.cn/ubibrowser_v3/	Ubiquitination	1,884,676 predicted high-confidence ESIs, 8,341,262 potential E3 recognizing motifs, 4068 known ESIs from literature
PhosPhAt [32]	https://phosphat.uni-hohenheim.de/	Phosphorylation	10,898 phosphoproteins, 64,128 serine sites, 13,102 threonine sites, 2672 tyrosine sites

2.2. Data Pre-Processing

Data pre-processing involves three primary steps. First, the protein sequence was segmented to generate fragments. The next step was to collect trustworthy negative data to construct the datasets. Finally, the problem of imbalanced datasets was investigated to mitigate potential adverse effects. The data pre-processing workflow is shown in Figure 2.

2.2.1. Sequence Slice

In current studies on PTM, most researchers have chosen the peptide representation method outlined in Equation (1).

$$S=P_{-\epsilon }\dots P_{-2}{P}_{-1}{P}_0{P}_1{P}_2\dots P_{\epsilon }$$

(1)

$P_0$ ignifies the central amino acid in PTM site recognition studies, which aims to determine whether these amino acids undergo modifications. For example, in methylation recognition studies, $P_0$ represents either lysine or arginine. The $P_{-\epsilon }\dots P_{-2}{P}_{-1}$ represents the upstream $\epsilon$-th amino acid from the central amino acid $P_0$, whereas $P_1{P}_2\dots P_{\epsilon }$ denotes the downstream $\epsilon$-th amino acid from the central amino acid $P_0$. Therefore, the length of the peptide is $2\epsilon +1$. It is customary to use ‘-’ or ‘X’ as placeholders when there are insufficient upstream or downstream amino acids.

The lengths of the peptides used in PTM site studies have varied from one research project to another. Lai et al. [33] developed an auto-machine learning method to predict the lysine lactylation sites in 51 amino acids. Wei et al. [34] used 11 residues to predict methylation sites. Li et al. [35] used peptides of 31 amino acids. Sua et al. [36] trained on 27-peptide sequence segments and tested 20 of them. Lyu et al. [37] segmented proteins into 35-residue segments with cysteine at the center. Auliah et al. [38] used a local sliding window of 57 residues to predict pupylation sites. Bao et al. [39] created 27-tuple peptides for K-PTMs. The peptide lengths available in dbPTM and PhosphoSitePlus were 21 and 15 bp, respectively. The choice of peptide length may affect the prediction results, leading researchers to explore peptides of various lengths. Khalili et al. [40] investigated window sizes ranging from seven to 35 and found that a window size of 13 yielded the best performance in their models.

2.2.2. Sequence Redundancy

CD-HIT, a method proposed by Li et al. [41], has been widely used to eliminate homologous protein data. CD-HIT employs a clustering approach to identify and remove similar protein sequences from the dataset. CD-HIT is widely used in PTM-site studies to eliminate homologous protein sequences and residual fragments. Some studies have aimed to remove redundant protein sequences [35,42], whereas others have focused on eliminating fragmented residue sequences [43,44].

2.2.3. Selected Reliable Negative Sequences

Although databases commonly provide information on modification sites, they do not provide information on non-modification sites. Peptides without annotation information (known as unlabeled data) can be categorized into two groups: first, the site has not been identified as a modification site, and second, additional research is required to verify whether the site has been modified. Therefore, we obtained reliable positive sequences; however, there may be potential issues with negative data. The modification site prediction problem can be described as a positive-unlabeled (PU) problem. Two methods can be used to address the PU problem in the modification site prediction.

Segments without modification information were considered negative samples [33]. The proposed method is simple and manageable. Several studies have used this method to develop models. However, these methods ignore the potential for such modifications. Gao et al. [45] established three criteria for identifying non-phosphorylated sites: (1) the fragment must not be labeled as a positive site, (2) the fragment should be within the sequence containing a positive site, and (3) the negative site must be solvent-inaccessible. In [40,46], all residues from a protein with a minimum of three confirmed positive sites were considered to be negative sites.

Another method aims to build models using limited positive data and large amounts of unlabeled data. Ning et al. [47] used semi-supervised learning and a support vector machine (SVM) to select reliable negative data. Jiang et al. [48] introduced the PUL-PUP algorithm to acquire negative data. PUL-PUP initially used similarity to identify negative data that are distant from positive data. Subsequently, PUL-PUP iteratively trains the SVM to expand the reliable negative data.

2.2.4. Balanced Dataset

In PTM studies, the amount of positive data is less than that of negative data, leading to a data imbalance. Imbalanced datasets may have detrimental effects on model training. Various methods have been proposed, such as data-based, algorithm-based, and hybrid-based methods.

(1)

Data based methods

Data-based methods can be divided into oversampling [49], undersampling [50], and hybrid sampling [51,52,53]. Random oversampling (ROS) randomly duplicates minority samples to create a balanced dataset. The synthetic minority oversampling technique (SMOTE) [49] generates new samples by considering the k-nearest neighbors of each minority class sample and is widely used to address imbalanced datasets, such as SulSite-GTB [54]. Adaptive synthetic sampling (ADASYN) [50] is an advanced method that can generate samples based on the learning difficulties of individual minority groups. Balanced datasets can also be obtained through undersampling, which involves reducing the number of majority samples in the dataset. Random undersampling (RUS) selects sequences from the majority of subsets and is widely used for PTM prediction. NearMiss [55] selects a subset of majority class samples closest to minority class samples as representatives. ENN [56] is a KNN-based method that avoids interference from samples. Hybrid sampling methods, such as SMOTETomek [57,58] and SMOTEENN, combine oversampling and undersampling.

(2)

Algorithm-based method

Algorithm-based methods solve imbalanced datasets by shifting their focus toward minority class samples, including cost-sensitive learning [59], ensemble learning [60,61], and one-class classification [61,62]. Cost-sensitive learning uses cost functions to build classifiers by minimizing the misclassification cost and adjusting the cost of misclassified samples based on a cost matrix [63,64,65]. Commonly used methods include weighted SVM [66], fuzzy SVM [67,68], and cost-sensitive neural networks [69]. Ensemble learning can reduce the bias caused by a single learner and enhance the model efficiency. Bagging, boosting, stacking and hybrid models are the most representative ensemble learning models. RUSBOOST [70] trains weak classifiers by constructing multiple balanced datasets using a combination of random undersampling (RUS) and AdaBoost. Jia et al. [71] used an ensemble method to predict the O-GlcNAcylation sites. One-class learning, or novelty detection, is a useful approach for handling significant imbalances between positive and negative samples. This technique builds models for the minority class, for example, one-class SVM [72].

(3)

Hybrid-based methods

Based on the aforementioned methods for handling imbalanced datasets, researchers have proposed hybrid methods that combine different balancing strategies to further enhance model performance. For instance, Islam et al. [73] used undersampling and K-nearest neighbor (KNN) to balance the data. In [70], a combination of RUS and AdaBoost was used to balance the dataset. Reference [51] implemented hybrid sampling and bagging classifier to address the problem of imbalanced dataset.

2.2.5. Data Splitting

To train a model and objectively assess its generalization capability, the dataset must be divided into training and test sets according to specific guidelines. The holdout method, a widely used and straightforward approach, involves splitting data according to a predefined ratio, such as 70% for training and 30% for testing. Alternatively, a temporal partitioning strategy can be employed using data collected before 2022 for training and data from 2022 to 2025 for testing. Regardless of the chosen method, it is crucial to ensure that the training and test sets have similar data distributions and that no test data are included in the training set.

3. Feature Engineering

Feature engineering mainly includes two parts: feature extraction and feature dimensionality reduction. This section separately introduces the application of these two components in PTM prediction.

3.1. Feature Extraction

Machine learning cannot directly recognize the sequence data. Therefore, researchers must design algorithms to complete feature extraction. Feature extraction involves encoding sequences to transform them into features suitable for machine learning models. Feature extraction methods are categorized into four types: (1) sequence-based, (2) physicochemical-based, (3) annotation-based and (4) deep learning-based.

3.1.1. Sequence-Based Feature

Sequence-based methods commonly use the composition, position, and other relevant information on amino acids to achieve a numerical representation of protein sequences. In sequence-based methods, a placeholder (‘-’ or ‘X’) is considered a particular type of amino acid. So, there are 21 amino acid residues in sequence segments.

Amino acid composition (AAC) [74,75,76,77] is a common feature extraction method based on the frequency of amino acids in the sequence segments. AAC can yield 21 features, including 20 amino acids and one placebo feature. The composition of k-spaced amino acid pairs (CKSAAPs) uses the frequency of amino acid pairs with separation of k spaces to represent the protein sequence, where the value of k is set by the researcher. CKSAAP has 441 features (from AA, AC, to XX) when k is 0. One-hot encoding [76] is commonly used in deep learning, where each amino acid is converted into a vector of length 21. Finally, a protein fragment of length L was depicted as a two-dimensional (2D) matrix of size L × 21. The use of machine learning methods to extract features is an innovative approach. The K-nearest neighbor (KNN) Score [78,79,80] was used to characterize the fragments.

In addition to the methods discussed above, several other sequence-based feature extraction methods exist, for instance, the conjoint triad descriptor (CTriad) [77,81], dipeptide composition (DPC) [77,82], amino acid pair composition (AAPC) [74,76,83], pair potential [45,84], four-body statistical pseudo-potential [45,85], local structural entropy [45,86], information on proximal PTMs [47], position-specific amino acid propensity (PSAAP) [47,87,88], enhanced amino acid pair (EAAC), enhanced group amino acid pair (EGAAC) [76,89], and position weight amino acid composition (PWAAC) [54] descriptors.

3.1.2. Physicochemical-Based Feature

The physicochemical properties of amino acids directly influence their interactions with enzymes, behavior in three-dimensional protein structures, and suitability as substrates for modification. Based on these properties, the 20 amino acids can be regarded as functional units, each exhibiting distinct characteristics. These properties were assessed for each amino acid using a well-established metric. For physicochemical properties, placeholders (‘-’ or ‘X’) were either disregarded or assigned a default value of 0.5.

The AAindex database, comprising 566 amino acid properties [25,77,90,91,92], can be integrated with computational techniques such as gray models, principal component analysis, and clustering to enhance feature extraction efficiency. The secondary structure (SS) [77,93] was determined using SPIDER2, which converted fragments into a 63-dimensional vector that included probability scores for α-helix, β-helix, and coil for each amino acid (with placeholders). The composition, transition, and distribution (CTD) method [77,94,95] introduced by Dubchak et al. categorizes 20 amino acids into three groups based on eight properties, ultimately transforming the fragments into an 188-dimensional vector.

Some commonly used feature extraction methods include backbone torsion angles (BTAs) [77,91,96], accessible surface area (ASA) [45,77,97,98,99], physicochemical properties (PCPs), other binding sites for any chemical groups [91], positively charged amino acid composition (PCAAC), regions discorded by DISOPRED2 [91,100], BioJava [91,101], disorder [45,102], gray pseudo amino acid composition [47], and encoding based on grouped weight (EBGW) [54].

3.1.3. Annotation-Based Feature

Protein annotation typically encompasses basic, structural, and functional details, as well as other pertinent information. These data aid in comprehending the structure, function, and significance of proteins within organisms and are frequently used to describe protein fragments.

There are numerous annotation-based methods, such as the position-specific scoring matrix (PSSM) [73,74,76], evolutionary-based profile bigrams [73,103,104,105], gene ontology (GO) [91,106], InterPro [91,107], KEGG [91,108], Pfam [91,109], STRING [91,110], functional domain [91], active site [91], natural variants [91], BLOSUM62 scoring matrix (B62) [74,111], evolutionary conservation score [45,112,113], and pseudo-position-specific scoring matrix (PsePSSM) [54,114].

PSSM is the most popular method. PSSM is a 20 × L matrix, where L represents the length of the fragment. Each column corresponds to a residue position in the protein sequence, and each row represents one of the 20 possible amino acids. Each element (i, j) in the matrix represents the probability or score of the j-th position in the protein sequence being mutated into the i-th amino acid during evolution. This score typically reflects the degree of conservation and preference for a specific amino acid at that position.

3.1.4. Deep Learning-Based Feature

Deep learning is extensively employed in PTM site recognition research, serving dual purposes: as a classifier for prediction and as a tool for extracting features from the network structures. This section focuses on its role as a feature extraction tool.

Convolutional neural networks (CNNs) extract features and reduce dimensionality via convolutional and pooling layers, whereas recurrent neural networks (RNNs) capture the sequence context. Some studies have integrated these approaches into hybrid feature extraction methods. Natural language processing (NLP) has rapidly developed in recent years. Protein sequences are similar to those in natural languages in several respects. First, both datasets are sequential. Second, both contain contextual information. Several studies have used language models to extract protein fragment features. Bidirectional encoder representations from transformers (BERTs) are typically used to predict PTM sites. Alkuhlani et al. [115] used six protein language models, ProtBERT-BFD [116], ProtBERT [116], ProtALBERT [116], ProtXLNet [116], ESM-1b [117], and TAPE [118] to identify PTM sites based on BERT [119], Albert [120], and XLNet [121]. Qiao et al. used BERT to build a novel predictor, BERT-Kcr, for protein Kcr sites prediction [10]. Lyu et al. [37] used word embedding to encode protein fragments, whereas Wang et al. [122] predicted plant ubiquitination by using the word2vec feature extraction method. PTM is intrinsically linked to enzymes, with enzyme-substrate relationships deduced from the physical and chemical properties of the modification sites through feature engineering. Deep-PLA [123] has demonstrated the effective integration of enzyme-specific constraints into deep neural network architectures, offering a relevant case study for this topic.

Sequence-based features primarily leverage the relative positions and compositional information of amino acids in protein sequences. Although these features provide high interpretability, they often fail to capture the physicochemical properties of amino acids adequately. Physicochemical property features, such as the AAindex, are generally fixed values that are determined beforehand. These features do not reflect the actual state of an amino acid within the three-dimensional structure of a specific protein or cellular environment. In reality, amino acids interact, work together, and produce powerful synergistic effects, and fixed values cannot represent them because they are nonlinear, dynamic, and complex. Annotation-based features offer an alternative perspective for studying PTM. However, this method relies heavily on homologous sequences. Annotation-based features are unreliable for novel proteins without known homologous sequences. Additionally, regions of high evolutionary conservation do not always correspond to modification sites, whereas some modification sites may lack high conservation. Sequence features extracted using deep learning are typically derived from one-dimensional amino acid sequences. This generation process resembles a black box, limiting the biological interpretability of these features. Because these features are primarily based on sequence data, they fail to effectively incorporate the three-dimensional structural information of proteins in real space. Although some studies, such as the LkaM-PTM [124] model, have begun to integrate structural information into feature construction, enabling deep learning-based sequence features to accurately reflect spatial structures remains challenging. In addition to the methods mentioned above, prediction methods based on modern protein structures (such as AlphaFold2 [125]) also deserve further investigation.

Therefore, integrating multiple feature types has become the mainstream method for overcoming the limitations of single-feature methodologies. Studies typically perform dimensionality reduction and feature selection on high-dimensional feature sets to identify the most informative features from a vast feature space, thereby constructing a compact yet highly discriminative feature subset.

3.2. Feature Reduction

We introduced four types of feature extraction methods. Several studies have used multiple feature extraction methods to obtain comprehensive feature sets. However, the ensemble method often leads to the challenge of having an excessive number of feature dimensions. Redundant and nonessential features can reduce the efficiency of the predictor. Important features were retained through feature reduction, whereas those with lower importance were removed. Consequently, the final feature vector exhibited a lower dimensionality and higher relevance. Feature-reduction approaches can be divided into two types: feature selection, which only reduces the number of features, and feature transformation or dimensionality reduction, which focuses on decreasing complexity by transforming existing features. These two methods are commonly used to optimize feature sets in modification-site prediction studies.

Auliah et al. [38] used a chi-squared test to perform feature selection. The chi-square test is a widely used hypothesis-testing method that is important in statistical analysis. The chi-squared test was used to examine whether the two variables were independent of each other. Maximal-relevance maximal-distance (MRMD) was used to rank the importance of features [34]. Li et al. [126] combined the analysis of variance (ANOVA) with incremental feature selection (IFS) to find the most vital feature subset. The minimum redundancy maximum relevance (mRMR) [127,128,129,130] is often used to select optimal features from the entire feature set. He et al. [131] proposed a feature selection method called MRMD3.0, which comprises two steps: the first step contains nine feature ranking methods (tree importance, ANOVA, variance threshold, chi-squared, linear model method, mutual information, mRMR, MRMD, and recursive feature elimination) and four link analysis strategies (PageRank, Trust Rank, Leader Rank, and HITS). The second step uses the IFS to select the best feature subset. Ensemble methods are typically used for the feature selection. Yu et al. [132] selected features using XGBoost [133]. Principal component analysis (PCA) is widely used to reduce the dimensionality of data. This method can describe existing high-dimensional feature sets using fewer comprehensive features. Singular value decomposition (SVD) is another standard method used for feature dimensionality reduction in the literature.

Unlike in traditional machine learning, deep learning does not typically involve an explicit, standalone process for feature dimensionality reduction. However, certain intrinsic components of deep learning models, such as pooling layers and dropout layers, inherently perform functions similar to feature selection and dimensionality reduction while executing their primary roles, such as downsampling or mitigating overfitting.

4. Classifiers

This section is dedicated to an individual treatment of how machine learning and deep learning methods are applied to PTM site identification.

4.1. Machine Learning

Support vector machines (SVMs) separate protein fragments by creating an optimal hyperplane for classification, which is particularly effective for small sample data. Xu et al. [134] used SVM to identify protein lysine glycation using sequences. Bao et al. [39] used an SVM and multilayer neural networks to predict various PTM sites. Auliah et al. [38] used multiple classifiers to assess the recognition efficiency of PUP-fusion. Decision trees (DTs) use if and then judgment rules for protein fragment classification. K-Nearest Neighbor (KNN) predicts unknown protein fragments based on the nearest samples. Ning et al. [135] used KNN as a classifier to identify formylation sites and explored the impact of different values of K on experimental results. Artificial Neural Networks (ANNs) are popular classifiers in bioinformatics that mimic the structures and functions of biological neural networks [136]. Several studies have used ANN to predict PTM sites [137,138].

Ensemble methods can enhance the prediction accuracy. Ensemble methods can be categorized into three types: Bagging, Boosting, and Stacking. Bagging is similar to the voting process. Basic classifiers have been used to predict protein fragments, resulting in various outcomes. The category of unknown protein fragments was determined based on the most frequent category. Random Forest (RF) [139,140] is a representative bagging algorithm commonly used for PTM site prediction. Hasan et al. [141] used RF to predict S-sulfenylation sites. The Cascade Forest [142,143] uses a layered approach comprising multiple forest structures, where the input for each layer is derived from the output feature information of the preceding layer. This methodology facilitates incremental feature extraction and allows for adaptive adjustments to the complexity of the model. Qian et al. [144] proposed a novel predictor, the SUMO-forest, based on cascade forests. Boosting refines the base learner by adjusting the data sample weights and ultimately determines the segment classes through weighted voting. Gradient tree boosting (GTB) [145,146] is a popular boosting method that uses multiple DT with excellent performance and has been applied in multiple fields. Wang et al. [54] proposed SulSite-GTB to predict S-sulfenylation sites based on GTB. The stacking method uses multiple base learners to identify protein fragments and generate classification results. These classification results are then used as features for another learner to learn and produce final classification results. He et al. [147] used stacking ensemble layers to build a predictor in which the base learners were convolutional neural networks with different specifications. In addition to the aforementioned ensemble methods, other hybrid methods have been developed. Zhang et al. [148] integrated five classifiers, RF, SVM, GBDT, KNN and Logistic Regression, to predict lysine malonylation sites.

In PTM site prediction, machine learning methods offer several advantages, including lightweight architectures, fast training, and the ability to produce effective models from limited data when supported by high-quality feature engineering. ML methods also retain strong interpretability in the decision-making process. However, these methods rely on researcher-crafted feature sets, such as physicochemical properties and evolutionary information derived from protein sequences. This manual feature extraction paradigm may fail to capture the complex latent patterns in sequences.

4.2. Deep Learning

CNNs consist of convolutional, pooling, and fully connected layers. The convolution layer extracts features from the input data, and the pooling layer selects these features. The fully connected layer classifies unknown protein fragments. Wang et al. [149] used CNN to predict multiple PTMs. Zhao et al. [150] used a CNN to predict the Kcr. CNN-SuccSite [76] was developed as a CNN model for predicting lysine succinylation sites and comprises an input layer, two convolution layers, two max-pooling layers, two fully connected layers, and an output layer. Wei et al. [151] created a one-dimensional (1D) CNN to predict Kcr sites. However, RNNs are advantageous for sequential data because of their ability to handle sequences of varying lengths and capture temporal dependencies. RNNs provide rich contextual information and include two common variants: long short-term memory (LSTM) [152] and gated recurrent units (GRUs). Lyu et al. [37] constructed a five-layer LSTM model featuring an input layer, word embedding layer, LSTM layer, dense layer, and output layer to predict cysteine sulfophenylation sites. Li et al. [153] proposed a transfer learning model based on LSTM to predict lysine propionylation, whereas Mul-SNO [154] combined bidirectional long short-term memory (BiLSTM) and bidirectional encoder representations from transformers (BERTs) to predict S-nitrosylation sites. Yu et al. [155] used a CNN-LSTM hybrid network for feature extraction and prediction. Currently, some studies have integrated deep learning with machine learning to enhance prediction efficiency. Ning et al. [156] combined 4-layer DNN and penalized logistic regression for succinylation site prediction. PROSPECT [157], proposed by Chen et al., integrates two CNNs and an RF to predict phosphorylation sites.

Transformer [158] is a deep learning model that uses a self-attention mechanism. Its primary innovation is the ability to capture global dependencies among all elements in a sequence through parallel computation, which enhances the model’s capability to contextualize information within that sequence. Transformer architecture comprises an encoder and decoder, with each layer featuring multi-head self-attention mechanisms and feed-forward neural networks. This design enabled the model to dynamically assess the significance of each amino acid in the input sequence. Meng et al. [159] proposed TransPTM, a transformer-based neural network model for predicting nonhistone acetylation sites. Liang et al. [160] proposed an effective model named DeepMM-Kcr, which is based on multiple features and a multi-head self-attention mechanism.

The core idea of transfer learning is to apply knowledge, including model parameters and feature representations, acquired from solving one task (the source task) to another related but distinct task (the target task). This application enhances the learning efficiency and performance of the new tasks. In some instances, the data for certain modification sites may be limited, potentially resulting in insufficient annotated data to train high-performance models. The utilization of transfer learning methods can effectively address this challenge. Xu et al. [161] developed DTL-NeddSite, a convolutional neural network-based predictor that leverages deep transfer learning and one-hot encoding. The model was first trained on a large dataset of lysine PTM sites and then fine-tuned using neddylation site data to construct the target model. Soylu et al. [162] developed the DEEPPTM model by integrating a protein embedding approach using ProtBERT with an attention-based Vision Transformer (ViT) to enhance the modification prediction accuracy and elucidate the relationships between modification types and protein sequences.

Deep learning can directly learn hierarchical feature representations from raw protein sequences in an end-to-end manner, thereby eliminating the need for labor-intensive manual feature engineering that relies on prior knowledge of the protein. The DL method often achieves higher prediction accuracy when it is supported by large-scale datasets. However, it also has notable limitations: the models typically require large amounts of annotated data, substantial computational resources. Effectively integrating multi-source heterogeneous biological data, such as gene expression profiles, protein–protein interaction networks, and evolutionary information, remains a key challenge in enhancing model generalization. Furthermore, designing neural network architectures that can capture long-range dependencies in sequences while incorporating structural spatial information is critical in deep learning model research. The integration of attention mechanisms with graph neural networks presents a promising approach for preserving the topological relationships between sequences and structures when modeling residue interactions. Leveraging pre-training strategies to learn general representations from unlabeled data can mitigate the scarcity of annotated samples. Furthermore, incorporating interpretability techniques to explore biologically significant regions highlighted by the model aids in elucidating the underlying molecular mechanisms, thereby advancing deep learning from ‘black-box’ prediction to mechanistic discovery.

5. Measurement

The prediction of PTM sites is a binary classification problem. The modified sites were divided into positive and unlabeled data. Typically, unlabeled data are considered negative samples. Researchers typically use accuracy (ACC), Matthews correlation coefficient (MCC), F-measure, and area under the receiver operating characteristic curve (AUC) to assess classifier performance. The formulas for these metrics are as follows:

$$\mathrm{A}\mathrm{C}\mathrm{C}={\displaystyle \frac{TP+TN}{TN+TP+FN+FP}}$$

(2)

$$\mathrm{M}\mathrm{C}\mathrm{C}={\displaystyle \frac{\left(TP\times TN\right)-\left(FN\times FP\right)}{\sqrt{\left(TP+FN\right)\left(TN+FP\right)\left(TP+FP\right)\left(TN+FN\right)}}}$$

(3)

$$\mathrm{F}1={\displaystyle \frac{2\times \frac{TP}{TP+FP}\times \frac{TP}{TP+FN}}{\frac{TP}{TP+FP}+\frac{TP}{TP+FN}}}$$

(4)

$$Sn={\displaystyle \frac{TP}{TP+FN}}$$

(5)

$$Sp={\displaystyle \frac{TN}{TN+FP}}$$

(6)

where TP denotes the number of correct classifications in the positive dataset. TN represents the number of correct classifications in a negative dataset. FN is the number of false-negative results, where FP is the number of false positives.

The receiver operating characteristic (ROC) curve was originally used for radar-signal detection to differentiate between signals and noise. Subsequently, the researchers adopted it for the model evaluation. The horizontal axis of the ROC curve represents the false positive rate (FPR), and the vertical axis represents the true positive rate (TPR). Owing to the curved nature of the ROC, assessing the quality of the model can be challenging. Therefore, in practical applications, the area under the curve (AUC) serves as a measure of the model performance and is particularly beneficial for handling unbalanced data.

6. Summary of Predictors

With advancements in machine and deep learning, there has been a surge in research focusing on predicting PTM sites.

Table 2. Review of PTM prediction models in recent years.

PTM	Tools	Dataset	Window Size	Feature Extraction	Classifier	Result			Website	Ref.
						ACC	AUC	MCC
Crotonylation	BERT-Kcr	used by Lv et al. [163]	31	BERT	BiLSTM	82.0%	0.905	0.640	http://zhulab.org.cn/BERT-Kcr_models/data	[10]
Lactylation	Auto-Kla	UniProt	51	Token embedding, position embedding, transformer encoder	AutoML, MLP	91.21% ± 1.58%	0.92 ± 0.0062	0.554 ± 0.023	https://github.com/tubic/Auto-Kla	[33]
S-sulphenylation	DeepCSO	UniProtKB	35	NUM, EAAC, BE, AAindex, CKSAAP, PSSM	LSTM, CNN, RF, SVM	Arabidopsis thaliana			http://www.bioinfogo.org/DeepCSO	[37]
						78.6% ± 0.7%	0.852 ± 0.018	0.417 ± 0.032
						Homo sapiens
						77.7% ± 0.6%	0.822 ± 0.011	0.367 ± 0.028
Phosphorylation	--	dbPTM	21	AAindex, Binary-encoding, ASA, secondary structure, disordered regions, BP, MF, CC, protein functional, domain data from InterPro, KEGG pathway and functional annotation	RF, SVM	Serine			--	[44]
						--	0.95	0.78
						Threonine
						--	0.97	0.77
						Tyrosine
						--	0.99	0.57
Succinylation	SSKM_Succ	Training data: PLMD and UniProt Test data: dbPTM	21	Information of Proximal PTMs, Grey Pseudo Amino Acid Composition, K-Space, PSAAP	SVM, RF, NB	80.18%	--	0.546	https://github.com/yangyq505/SSKM_Succ.git	[47]
S-sulfenylation	SulSite-GTB	Carroll Lab, RedoxDB and UniProtKB	21	AAC, DPC, EBGW, KNN, PSAAP, PsePSSM, PWAAC	GTB	88.53%	0.94	0.77	https://github.com/QUST-AIBBDRC/SulSite-GTB/	[54]
Phosphoglycerylation	iDPGK	PLMD	15	AAC, PCAAC, AAPC, BLOSUM62, PSSM	DT, RF, SVM	74.9%	--	0.49	http://mer.hc.mmh.org.tw/iDPGK/	[74]
Succinylation	CNN-SuccSite	PLMD 3.0	31	PspAAC, CKSAAP, PSSM	CNN	86.79%	--	0.489	http://csb.cse.yzu.edu.tw/CNN-SuccSite/	[76]
Glycosylation	PTG-PLM	UniProt	31	ProtBERT-BFD, ProtBERT, ProtALBERT, ProtXLNet, ESM-1b and TAPE	CNN, SVM, LR, RF, and XGBoost	Ngly Site			https://github.com/Alhasanalkuhlani/PTG-PLM	[115]
						96.5%	0.978	0.902
						Kgly site
						64%	0.64	0.28
Formylation	LFPred	UniProt, PLMD and dbPTM	41, information entropy	AAC, BPF, AAI	KNN	79.3%	--	0.55	--	[135]
S-sulfenylation	S-Sulfenylation	Conducted by Xu et al. [164] and Hasan et al. [141]	21	PseAAC, SVV, SM, PRIM, R-PRIM, FV, AAPIV, RAAPIV	BP-NN	96.89%	0.931	0.862	https://www.github.com/ahmad-umt/S-Sulfenylation	[137]
Sumoylation	SUMO-Forest	UniProt	21	PSAAP, PseAAC, SP, BK	Cascade Forest	Cascade Forest-based cost-matrix			https://github.com/sandyye666/SUMOForest	[144]
						98.69%	0.98	0.89
						Cascade Forest based F-measure
						98.54%	0.99	0.89
						--	0.797	0.287
						sumoylation
						--	0.868	0.431
Crotonylation	--	collected verified Kcr sites on non-histone proteins from papaya	From 2 to 37	BE, CKSAAP, AAC, EAAC, EGAAC	CNN	85.64%	0.853	0.335	http://www.bioinfogo.org/pkcr	[150]
Crotonylation	DeepKcrot	Collected from [165,166,167]	29	EGAAC, WE	LSTM, CNN, RF	RFEGAAC			http://www.bioinfogo.org/deepkcrot	[151]
						0.851	0.784	0.228
						LSTMWE
						0.860	0.839	0.306
						CNNWE
						0.869	0.861	0.338
Propionylation	--	PLMD and UniProt	17	RNN, LSTM	Transfer learning, SVM	--	0.705	0.317	http://47.113.117.61/	[153]
Succinylation	HybridSucc	PLMD 3.0, PhosphoSitePlus and dbPTM	--	PseAAC, CKSAAP, OBC, AAindex, ACF, GPS, PSSM, ASA, SS, and BTA	DNN, PLR	--	0.885	--	http://hybridsucc.biocuckoo.org/	[156]
Nitrosylation	Mul-SNO	training set: Li et al. [168], independent test set: DeepNitro	31	BiLSTM, BERT	RF, lightgbm, xgboost	80%	0.80	0.59	http://lab.malab.cn/∼mjq/Mul-SNO/	[154]
Phosphorylation	PROSPECT	UniProt	27	one-of-K, EGAAC and CKSAAGP	CNNone-of-K, CNNEGAAC and RFCKSAAGP	--	0.821	0.37	http://PROSPECT.erc.monash.edu/	[157]
Crotonylation	DeepMM-Kcr	the same as those used by lv et al. [163]	31	token embedding, Positional embedding, one-hot, AAindex, PWAA	Transformer	85.56%	0.9310	0.7119	https://github.com/yunyunliang88/DeepMM-Kcr	[160]
Neddylation	DTL-NeddSite	from the literature	41	EAAC, One-hot, WE	Transfer learning	--	0.818	--	https://github.com/XuDeli123/DTL-NeddSite	[161]
Multi-PTM	DeepPTM	CPLM	21	ProtBERT	ViT	--	0.793 (succinylation)	--	https://github.com/seferlab/deepptm	[162]
Glutarylation	iGlu_AdaBoost	Conducted by Al-barakati et al. [169] from PLMD, NCBI, and SWISS-PROT	23	188D, CKSAAP, and EAAC	AdaBoost	72.07%	0.63	0.36	--	[170]
Malonylation	Kmalo	PLMD and LEMP	11~39	AAC, one hot encoding, Pse-AAC, AAindex, PSSM	hybrid models contain multiple CNNs, random forests and SVM	Mammalian proteins			https://fdblab.csie.ncu.edu.tw/kmalo/home.html	[171]
						86.6%	0.943	0.480
						Plant proteins
						69.1%	0.772	0.195
Ubiquitination	DeepTL-Ubi	PhosphoSitePlus, mUbiSida and PLMD	31	one-hot	transfer deep learning method	M. musculus			https://github.com/USTC-HIlab/DeepTL-Ubi	[172]
						60.4%	--	--
						A. nidulans
						67.9%	--	--
						T. gondii
						55.6%	--	--
Phosphorylation	--	iPhos-PseEn	13	BE	CNN, BLSTM	Phosphoserine (S)			--	[173]
						92.7%	0.996	0.582
						Phosphoserine (T)
						91.4%	0.994	0.501
						Phosphoserine (Y)
						93.6%	0.995	0.488
Phosphorylation	DF-Phos	dbPAF and Phospho.ELM	33	CTD, DDE, EAAC, EGAAC, a series of PseKRAAC, GrpDDE, kGAAC, LocalPoSpKaaF, QSOrder, SAAC, SOCNumber, ExpectedValueGKmerAA, ExpectedValueKmerAA, ExpectedValueGAA, ExpectedValueAA	Deep Forest	78%	--	0.51	https://github.com/zahiriz/DF-Phos	[174]
Crotonylation	--	UniProt and pkcr	31	AAC, AAPC, BE, CKSAAP, EAAC, EGAAC and PSSM	SVM, RF	90%	--	0.80	--	[175]
Lactylation	DeepKla	previous research, Botrytis cinerea	51	embedding	CNN, RNN	93.59%	--	0.8783	http://lin-group.cn/server/DeepKla/	[176]
Lactylation	DeepKlapred	previous research [176]	51	Position Embedding, QSOrder, CTD, DDE, DistancePair	Transformer	96.9%	--	0.938	https://awi.cuhk.edu.cn/~biosequence/DeepKlapred	[177]
Acetylation	TransPTM	UniProt	25	One-hot, ProtT5	Transformer	88%	0.83	0.45	https://www.github.com/TransPTM/TransPTM	[159]

-- indicates no relevant content; NUM: Numerical Representation for Amino Acid; ASA: Solvent accessible area; AAC: amino acid composition; DPC: dipeptide composition; EBGW: encoding based on grouped weight; KNN: k nearest neighbors; PSAAP: position-special amino acid propensity; PsePSSM: Pseudo-position specific scoring matrix; PWAAC: Position weight amino acid composition; PseAAC: pseudo amino acid composition; SP: statistics property; BK: bi-gram and k-skip-bi-gram; PspAAC: position-specifc amino acid composition; BPF: binary profile feature; AAI: amino acid index; AAPC: amino acid pair composition; PSSM: Position specific scoring matrix; B62: BLOSUM62; GPAAC: Grey Pseudo Amino Acid Composition; SVV: site vicinity vector; SM: statistical moments; PRIM: position relative incident matrix; R-PRIM: reverse position relative incident matrix; FV: frequency vector; AAPIV: accumulative absolute position incidence vector; RAAPIV: reverse accumulative absolute position incidence vector; DBPB: di-amino acid BPB; DDE: Dipeptide Deviation from Expected Mean value; EAAC: Enhanced Amino Acid Composition; Enhanced Grouped Amino Acid Composition; PseKRAAC: Pseudo K_tuple Reduced Amino Acid Composition; GrpDDE: Group Dipeptide Deviation from Expected Mean; kGAAC: k Grouped Amino Acid Composition; LocalPoSpKaaF: Local Position Specifi c k Amino Acids Frequency; QSOrder: Quasi Sequence Order; SAAC: Split Amino Acid Composition; SOCNumber: Sequence Order Coupling Number; ExpectedValueKmerAA: Expected Value for K-mer Amino Acid; ExpectedValueGAA: Expected Value for each group Amino Acid; ExpectedValueAA: Expected Value for each Amino Acid; BP: biological process; MF: molecular function; CC: cellular component.

summarizes various studies, including PTM types, datasets, window sizes, feature extraction methods, classifiers, results, and web servers.

According to Table 2, combined with the results obtained from keyword searches in the Web of Science, we generated Figure 3. This figure illustrates the percentage distribution of research papers published over the past five years, focusing on 15 types of modification sites. As illustrated in Figure 3, there is a significant imbalance in the research focus across various modification sites. Most studies focused on a limited range of modification types, including phosphorylation, ubiquitination, acetylation, and glycosylation, which together represent approximately 61.8% of the published literature. Conversely, other modification sites, notably propionylation and formylation sites, have received minimal attention.

Figure 4 illustrates the percentage distribution of various feature extraction methods in the existing literature, highlighting their relative importance. Traditional feature extraction methods continue to dominate, with sequence composition features (e.g., AAC and CKSAAP) and evolutionary information features (e.g., PSSM) being the most prevalent. Furthermore, features based on the physicochemical properties of amino acids (e.g., AAindex) have been widely employed in numerous models. In recent years, embedding methods that utilize pretrained language models, such as BERT, have demonstrated significant potential, positioning deep-learning-based feature extraction as a current research focus. Most models strategically combine multiple feature types to comprehensively capture diverse aspects of sequence information. Although some studies have incorporated certain structure-related features (such as ASA and SS), research that directly utilizes three-dimensional structural information remains limited.

As shown in Table 2, deep learning exhibits exceptional performance in various PTM prediction tasks. Models utilizing deep neural network architectures, such as CNN, LSTM, and transformers, have been used to predict emerging modification types, including crotonylation and lactylation, achieving a peak ACC of 96.9%. In contrast, traditional machine learning methods, while remaining somewhat competitive in predicting classical modification types, such as phosphorylation, offer advantages in training efficiency and model interpretability. However, their performance is often limited by the quality of manual feature engineering. Notably, current research reveals a significant technological convergence trend, particularly through the integration of learning strategies that combine features extracted by deep learning with traditional machine learning classifiers. This approach not only sustains high performance but also enhances the model’s generalization capabilities. Such systematic comparisons provide empirical evidence for method selection in the field of PTM prediction and suggest that future research should concentrate on developing novel computational frameworks that achieve both high predictive accuracy and strong biological interpretability.

7. Challenges and Future Directions

Significant progress has been made in the identification of PTM sites using machine learning and deep learning techniques. However, several noteworthy aspects warrant further investigations.

7.1. Data Limitations

The study of PTM sites based on machine and deep learning requires a large and accurate dataset for training the model. As previously mentioned, there are three main issues concerning data on modification sites: (1) the absence of completely reliable negative examples, (2) the significant imbalance present in the datasets, and (3) the current modification site databases reveal an underrepresentation of certain modification types. For example, the dbPTM database lists 194 O-palmitoleoylation sites, including both experimentally validated and predicted instances. This is insufficient to support the training requirements of deep or machine learning.

Several studies have proposed effective solutions to these issues. For example, Wen et al. [178] constructed the PTMAtlas database, which comprises 397,524 PTM sites, and developed a DeepMVP model that outperforms existing tools across multiple PTM types. The construction of comprehensive and representative datasets remains a critical challenge in the recognition of PTMs. Negative instance data, which are typically more abundant than positive instance data, are often unreliable and contribute to the imbalanced datasets. To address this issue, further investigation into the application of semi-supervised learning and one-class learning approaches in dataset construction and model building processes is warranted. Transfer learning is a deep learning technique that leverages the knowledge gained from one task to improve performance on a related but distinct task. By pre-training a model on a large dataset and then fine-tuning it on a smaller task-specific dataset, transfer learning can mitigate the challenges posed by limited training data, thereby addressing the issue of underfitting that may arise when working with insufficient data for certain modified sites.

7.2. Interpretability

In recent years, deep learning techniques have shown considerable value in bioinformatics, particularly for the prediction of protein modification sites. However, these models typically utilize complex nonlinear network architectures, leading to a lack of interpretability of their internal mechanisms. This ‘black box’ nature undermines researchers’ trust in the predictive outcomes of these models and ignites discussions concerning the reliability of algorithmic decisions in biomedical applications. The establishment of interpretable models is crucial for studying modification sites. These approaches seek to identify key sequence features that determine modification sites, indicating not only where modifications occur but also why they occur at specific locations. Furthermore, developing interpretable methods that correlate sequence-level importance with three-dimensional protein structures can elucidate the spatial and physicochemical constraints of these modifications.

In [179], MIND-S utilized structural models and protein structure graph networks to improve the PTM prediction. Integrating protein structure-based features with other feature types can effectively enhance model interpretability. Ultimately, the core value of interpretability analysis lies in converting model predictions into experimentally verifiable scientific hypotheses, thereby exploring the relationship between modification sites and biological activity. These models are categorized into post hoc and ante-hoc interpretations. Post hoc interpretability encompasses example-, attribution-, latent semantics-, and rule-based approaches. In contrast, ante hoc interpretable learning prioritizes interpretability as a fundamental objective of model design and training. This involves adopting a transparent model architecture or imposing interpretability constraints during training. When developing interpretable models, it is essential to design self-transparent neural networks that maintain model performance and efficiency while selecting appropriate interpretability evaluation metrics.

7.3. Multi-PTM and PTM Crosstalk Prediction

Protein sequences may contain multiple modification sites that play crucial roles in protein functionality and stability. Yan et al. [179] proposed a multi-PTM prediction method called the MIND-S. MIND-S applies a multi-label strategy to predict multiple PTM types and their sites. Multi-PTM prediction aims to systematically identify multiple types of post-translational modifications and their specific sites that coexist on a single protein, thereby elucidating the complex co-modification landscape of proteins. Unlike traditional single-PTM prediction approaches, this strategy not only focuses on the independent distribution of common modifications, such as phosphorylation and acetylation, but also emphasizes the analysis of spatial proximity, temporal dynamics, and potential functional relationships among different modifications. By integrating high-throughput mass spectrometry data with diverse sequence, structural, and evolutionary features, and leveraging advanced computational methods such as deep learning, multi-PTM prediction facilitates the construction of a more holistic functional regulatory map of proteins. This approach lays a crucial foundation for subsequent investigations into the synergistic or antagonistic effects between modifications, commonly referred to as PTM crosstalk.

PTM crosstalk plays a crucial role in organisms and can be categorized into crosstalk within the same protein (intra-protein) and crosstalk between different proteins (inter-protein). Currently, some studies have focused on identifying PTM crosstalk pairs [180,181,182]. However, research on PTM crosstalk mechanisms and interaction networks is limited. Research has shown that changes in certain modification sites may lead to disease [183]. Therefore, studying the relationship between modification sites and diseases is crucial for a deeper understanding of disease progression, intervention mechanisms, and the development of precise therapeutic drugs. However, there is currently limited research utilizing machine learning or deep learning methods to explore the relationship between modification sites and diseases, making this direction worthy of further exploration.

8. Conclusions

The accurate identification of PTM is crucial for various applications, including drug development, disease diagnosis, and the understanding of molecular processes. Although traditional biological experimental methods are accurate, they are resource-intensive. Machine learning can efficiently process large datasets. However, its prediction accuracy may be influenced by various factors. Machine learning methods include traditional methods that require feature extraction and deep learning. This manuscript reviews current PTM site predictors based on machine learning, including datasets, feature extraction, classifiers, and evaluation metrics. This manuscript summarizes the existing methods and explores future research directions. Data play an important role in machine learning. Thus, future research should explore ways to solve imbalanced data and PU problems. The use of few-shot and transfer learning addresses the data scarcity and model complexity. Multi-label learning is conducive to further exploring protein modifications. The association between modification sites and diseases is worth exploring. We hope that our review and analysis will assist research related to PTM.