Next Article in Journal
The Role of Curcumin in Prostate Cancer Cells and Derived Spheroids
Next Article in Special Issue
Radiomic Cancer Hallmarks to Identify High-Risk Patients in Non-Metastatic Colon Cancer
Previous Article in Journal
Validation of “Self-Evaluation of Communication Experiences after Laryngectomy” (SECEL) Questionnaire for Spanish-Speaking Laryngectomized Patients
Previous Article in Special Issue
Analysis of Cross-Combinations of Feature Selection and Machine-Learning Classification Methods Based on [18F]F-FDG PET/CT Radiomic Features for Metabolic Response Prediction of Metastatic Breast Cancer Lesions
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 14
Issue 14
10.3390/cancers14143349
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization

by
Hishan Tharmaseelan
1,
Alexander Hertel
1,
Shereen Rennebaum
1,
Dominik Nörenberg
1,
Verena Haselmann
2,
Stefan O. Schoenberg
1 and
Matthias F. Froelich
1,*
1
Department of Radiology and Nuclear Medicine, University Medical Center Mannheim, Medical Faculty Mannheim of the University of Heidelberg, 68167 Mannheim, Germany
2
Institute of Clinical Chemistry, University Medical Center Mannheim, Medical Faculty Mannheim of the University of Heidelberg, 68167 Mannheim, Germany
*
Author to whom correspondence should be addressed.
Cancers 2022, 14(14), 3349; https://doi.org/10.3390/cancers14143349
Submission received: 5 June 2022 / Revised: 7 July 2022 / Accepted: 8 July 2022 / Published: 9 July 2022
(This article belongs to the Collection Imaging Biomarker in Oncology)
Browse Figures
Review Reports Versions Notes

Abstract

:

Simple Summary

Modern, personalized therapy approaches are increasingly changing advanced cancer into a chronic disease. Compared to imaging, novel omics methodologies in molecular biology have already achieved an individual characterization of cancerous lesions. With quantitative imaging biomarkers, analyzed by radiomics or deep learning, an imaging-based assessment of tumoral biology can be brought into clinical practice. Combining these with other non-invasive methods, e.g., liquid profiling, could allow for more individual decision making regarding therapies and applications.

Abstract

Similar to the transformation towards personalized oncology treatment, emerging techniques for evaluating oncologic imaging are fostering a transition from traditional response assessment towards more comprehensive cancer characterization via imaging. This development can be seen as key to the achievement of truly personalized and optimized cancer diagnosis and treatment. This review gives a methodological introduction for clinicians interested in the potential of quantitative imaging biomarkers, treating of radiomics models, texture visualization, convolutional neural networks and automated segmentation, in particular. Based on an introduction to these methods, clinical evidence for the corresponding imaging biomarkers—(i) dignity and etiology assessment; (ii) tumoral heterogeneity; (iii) aggressiveness and response; and (iv) targeting for biopsy and therapy—is summarized. Further requirements for the clinical implementation of these imaging biomarkers and the synergistic potential of personalized molecular cancer diagnostics and liquid profiling are discussed.

1. Introduction

1.1. Achievements in Personalized Cancer Therapy

In modern medicine, cancer remains a tremendous therapeutic challenge. In 2018, it caused 1.4 million deaths and cost approximately EUR 199 billion in Europe [1]. Through better prevention, e.g., occupational safety, vaccination against oncogenic such as the human papillomavirus [2], and early detection measures such as colorectal cancer screening [3,4], the mortality of many cancers is decreasing [5]. Despite these improvements, it is still the second leading cause of global death for non-infectious diseases [6].
Modern cancer therapy primarily consists of multiple treatment modalities, including surgery, chemotherapy, radiation therapy, targeted antibodies, immunotherapy (including CAR-T-cells), and interventional radiology therapy. In general, treatment has changed significantly from mainly applying standardized surgery and chemotherapy protocols to personalized biological and immunotherapy protocols. This personalization of therapy has led to increased survival in cancer patients; for example, the anti-EGFR antibody Cetuximab used in the treatment of RAS-wildtype EGFR-expressing advanced colorectal cancer shows improved overall (hazard ratio: 0.77) and progression-free survival (hazard ratio: 0.68) compared to standard of care [7]. In terms of interventional radiology therapies, transarterial chemoembolization (TACE) and ablation therapies have enabled the targeting of specific lesions in clinical routine [8].

1.2. Current Status of Oncologic Imaging: Response Assessment

Since a broad range of therapies is available for solid tumors, individual characterization of therapy response is essential. The Response Evaluation Criteria in Solid Tumors (RECIST) in version 1.1 are currently widely used for response evaluation in solid tumors for clinical trials and beyond [9,10]. This very successful model was defined by the RECIST Working Group and initially intended for the standardization of imaging-guided therapy response evaluation in clinical studies.
The response of a patient is estimated by the definition of target- and non-target lesions and structured evaluation of growth patterns of tumor lesions. The criteria are defined by measurement on imaging and are widely used in clinical studies. The reproducibility of the RECIST criteria was shown to be limited in several diseases, such as prostate cancer, glioblastoma, and hepatocellular carcinoma [11]. The evaluation of response is often limited to tumor volume-shrinking cytostatic therapy. A solely size-based assessment is often not accurate in predicting survival [12], as modern non-cytostatic treatments that are used increasingly rely on the blockade of receptors to achieve their therapeutic effect, not only through tumor volume reduction. For example, inhibition of further growth may positively influence survival but may not be captured by the classic RECIST criteria [13]. As an alternative to RECIST, Choi criteria based on size and change in tumor attenuation in CT were developed to evaluate response in gastrointestinal stromal tumors that were treated with tyrosine kinase inhibitors [14]. In addition, they have been shown to be superior with respect to other oncologic entities and treatments, such as transarterial radioembolization in patients with hepatocellular carcinoma [15]. To evaluate these methods and support decision-making, more precise and objective criteria for response need to be developed.
For the improvement of the quantification of response, additional diameter-based criteria, such as early tumor shrinkage (ETS) and depth of response (DpR), have been proposed [16,17]. To address the drawbacks of diameter-based assessment, volumetric assessment of lesion size has been proposed as a more accurate alternative [18]. While this approach may help to improve the accuracy of solely lesion diameter-based measurement, the characterization of tumoral heterogeneity between lesions remains focused on changes in size. Specifically, an influence of initial lesion size and location can be noted [19]. Furthermore, an increased mean baseline CT-/Hounsfield attenuation of liver metastases could be shown to be associated with prolonged overall survival in metastatic colorectal cancer [20].

1.3. Current Status of Oncologic Imaging: Characterization of Lesions

Individual lesion analysis becomes progressively more feasible on the diagnostic side as modern methodologies, such as epigenetics, proteomics, transcriptomics, and genomics, provide additional information to traditional, biopsy-derived histological markers, such as grading of differentiation and immunohistochemistry staining [21]. In addition to the diagnostic and therapeutic advances at the benchside, the recent imaging advances at the bedside have enabled the targeting of individual lesions as an alternative or add-on to the application of pharmaceuticals.
To characterize individual lesions in clinical practice, often, semantic features that, for example, describe the shape, e.g., “spiculation”, the location, or the appearance/texture (“ground glass component”) are used. These markers have been shown to have a certain clinical relevance, but they do not cover the entire range of facets in tumoral biology. In comparison, quantitative imaging biomarkers can help gain objective and quantifiable information, such as radiomics or deep learning features. Radiomics has, for example, been shown to expose a dimension of data beyond a semantic visual characterization that can add valuable information to the predictive power of semantic features [22].
Until now, in contrast to the individual lesion characterization methodologies in molecular oncology and individual targeted therapies that have already been introduced to clinical research and practice, the imaging-based assessment of molecular characteristics has not been applied to translate these developments based on quantitative imaging biomarkers, such as aggressiveness or lesion mutational status, into optimized, personalized treatment. Therefore, quantitative imaging parameters should be deployed and designed with existing approaches already used in molecular oncology in mind in order to achieve a synergistic benefit for treatment planning.

1.4. Molecular Tumor Diagnostics for Personalized Therapy Stratification

Methodologies that enable molecular characterization by quantitative analyses are summarized as omics methodologies. Important examples of these techniques include genomics, transcriptomics, proteomics, and epigenomics. Recent advances in technologies such as whole-genome sequencing have enabled researchers to gain unprecedented insights into tumoral characteristics and behavior. The omics methodologies have been applied to a broad range of diseases. As some examples, survival in lung adenocarcinoma can be predicted by methylation signature [23], survival in patients with cancer of unknown primary can be improved with the use of whole-genome sequencing for therapy selection [24], and relapse in non-small lung cell cancer can be predicted by analyses of oncogenic mutations [25]. Via the combination of the different omics analyses, multi-omics analyses were established, making it possible to measure the different levels (e.g., mutational (genomics) and/or DNA methylation (epigenomics)) at which a pathway is affected. Using multi-omics analyses, for example, survival and drug response can be predicted in breast cancer by combining different molecular biomarkers, such as copy number variation or gene expression [26], and survival can be predicted in liver cancer [27]. Furthermore, the analysis of tumor mutational burden can be used to predict response to immunotherapies and thus can for therapeutic stratification [28,29]. Molecular genetic tumor profiling as a prerequisite for the administration of targeted therapeutics is defined as companion diagnostics and represents an integrated part of oncologic diagnostics.

2. Techniques for Lesion Analysis Using Quantitative Imaging Biomarkers

Starting from the deciphering of novel imaging markers using molecular oncology methodology, corresponding imaging biomarkers would help to solve challenges in oncologic imaging characterization, e.g., they can support the identification of aggressiveness and response. This can be employed as decision support for selecting lesions for locoregional therapy. In addition, it can support the identification of lesional etiology in cancers of unknown or multiple primaries. Furthermore, it could also be used as a measure of aggressiveness or in the situation of mixed response. Lesion characterization could be utilized for comparative analysis of primary and metastasis to identify driver mutations via imaging. Imaging biomarker analysis can be applied to a large scale of images—from computed tomography or MRI to ultrasound and nuclear imaging. Novel personalized oncologic imaging biomarkers in comparison with traditional imaging biomarkers are summarized in Figure 1.
To tackle the challenges of lesion-specific characterization, multiple methodologies of quantitative imaging biomarkers are feasible. The two most prominent for the identification of imaging biomarkers are classical radiomics features and deep learning.

2.1. Classical Radiomics Analysis Workflow and Texture Visualization

Radiomics is a term for the extraction of mostly non-human readable quantitative imaging data from radiological images [30]. The methodology has been a subject of clinical research for several years and is at the threshold of clinical application in certain cases. One of the main reasons for the incomplete translation of research findings in this field into practice is the partial reproducibility of radiomics features [31]. Radiomics features vary depending on different factors, such as ROI size [32], segmentation procedures [33], choice of scanner [34], scanning parameters (including tube current, slice thickness, and reconstruction algorithms [35]), and extraction software [36]. There is also a randomness factor in the analysis, as even coffee break test–retest analyses may not always agree [37]. To reduce the differences between the various extraction softwares and increase reproducibility, the Image Biomarker Standardization Initiative (IBSI) was founded [38]. In total, the IBSI defined 174 shape, first-order (histogram/intensity-based), and second-order (texture-based) features.
Although, methodologically, there are certain limitations, radiomics has been proven to be powerful in a diverse range of use cases in research. Often, radiomics is used in oncologic diseases, such as lung cancer [39], colorectal cancer [40], malignant lymphoma [41], prostate cancer [42], hepatocellular carcinoma [43], renal cell carcinoma [44], and breast cancer [45]. Apart from oncologic diseases, radiomics has been used in neurological diseases, such as Alzheimer’s [46], urologic diseases (e.g., kidney stones [47]), pulmonary diseases (e.g., idiopathic pulmonary fibrosis [48]), coronary plaques [49], and meniscus injuries [50].
The radiomics workflow consists of a step-by-step approach: first, regions of interest (ROI) need to be segmented automatically, semi-automatically, or manually. For automated segmentation, preferably, U-Net architectures are used [51]. Then, an extraction algorithm will be applied to calculate certain predefined quantitative features for this region of interest. A commonly used python package for feature extraction is pyradiomics [52]. Radiomics feature algorithms in pyradiomics are based on the IBSI definition containing shape, first order, and second-order features. In MRI studies, normalization is applied to the images to substitute the missing standardized scale. Different filters, such as Laplacian of Gaussian or wavelet filters, can be applied in the extraction process to accentuate distinctive image characteristics, such as edges [53]. As radiomics feature values can be highly redundant, feature selection methods, such as L1 (LASSO) regularization, Pearson correlation coefficient threshold, and random forest-based permutation feature importance calculation, can be applied to select a final feature set out of all features.
The resulting feature values can be used to train traditional machine learning algorithms, such as a random forest or XG Boost classifier [54,55], correlate them with clinical and genetic parameters, and statistically evaluate them by regression analyses or other classifiers. The radiomics workflow is summarized in Figure 2a.
Radiomics features can be visualized by breaking down the ROIs into smaller parts, providing a basis for a better understanding of tumor texture. For example, within a lesion, specific, particularly active areas can be identified and biopsied. Radiomics feature mapping could facilitate the clinical integration of radiomics information for improved characterization of cancer. Radiomics feature activation maps can also support the exploration of tumor biology in research and aid the delineation of specific tumoral regions that are important in disease progression [56].

2.2. Deep Learning Analysis Workflow

Deep learning architectures are characterized by analyzing large numbers of data, especially images, using neural networks with multiple layers. Due to better accessibility via open-source python packages for deep learning, such as Tensorflow and Pytorch, deep learning is increasingly used in biomedical research. From early technical developments of neuron-like circuits by McCullogh and Pitts in 1943 [57] to the implementation of backpropagation and multi-layered convolutional neural networks, many fundamental and groundbreaking developments have been made.
To train a model, unsupervised and (semi-) supervised training approaches can be used. In contrast to radiomics, deep learning has the advantage that it does not require prior segmentation, as the network automatically learns the ROIs/key areas. For supervised training approaches, the input data are labeled prior to being presented to the model. The so-called ground truth, on the basis of which the data are labeled, is ideally defined by the traditional gold standard for diagnosis (histology, genetic markers) or a consensus voting of multiple physicians. The input data for supervised approaches are mostly split into train, test, and validation sets, and performance is evaluated at the test set after the model has been trained using the training and validation set.
Compared to traditional radiomics-based machine learning models, convolutional neural networks have a more generalist approach and are not bound to specific predefined quantitative parameters. Therefore, they may generalize better in specific scenarios. In relation to the computational requirements of radiomics trained machine learning algorithms, deep learning on a large scale can only be performed on supercomputers with powerful GPUs. Therefore, they may generalize better in specific scenarios. A general problem in many research studies is training models in single-scanner, single-center settings. Algorithms need to be trained with heterogeneous datasets that reflect real-world settings as a condition for clinical applications [58]. In comparison to traditional machine learning, deep learning needs a vast number of data to build stable and generalizable models. Such quantities of data are often not available for rare diseases. In the case of a lack of images for training, various techniques can be applied. In particular, augmentation can be used, in which images are rotated, flipped, or otherwise manipulated and added, to increase the number of training data [59]. In addition, transfer learning, where pretrained models are trained on other datasets and fine-tuned with their own data, can be applied [60,61,62]. A typical workflow for the application of CNNs for medical imaging analysis is shown in Figure 2b.
An important limitation of all deep learning models is the risk of biased labeling of data that could, for example, lead to race- and gender-based discrimination [63]. This danger and limitation in artificial intelligence (AI) needs to be addressed in study planning and is being considered increasingly in the research on “ethical AI”, which is concerned with any kind of ethical issues raised by AI [64,65].
In the future, self-attention-based AI techniques (transformer networks), which are currently applied in natural language processing, may help enhance the performance of neural networks in image analysis even further [66,67].
Examples of the quantitative imaging biomarkers presented in this section are shown in Figure 3.

3. Application and Evidence for Novel Imaging Biomarkers for Improved Lesion Characterization and Prognosis

A broad range of applications of imaging biomarkers for lesion differentiation in various diseases is feasible. For example, radiomics has already been used to distinguish small hepatocellular carcinomas from focal nodular hyperplasia liver lesions [68], to classify the primary of spinal metastases [69], distinguish the primaries of metastases in lung lesions [70], distinguish benign from malignant lesions in the parotid gland [71], and identify subtypes of renal cell carcinoma [44].
In a complementary way, deep learning has already been used for the differential diagnosis of liver lesions [72,73], the detection of malignant lung nodules [74,75], the detection and classification of breast masses [76], and the assessment of lymph node metastases in women with breast cancer [77]. The methodology described before can be applied to multiple lesional imaging criteria.
Generally speaking, four levels of imaging biomarker usage for lesion differentiation and intralesional characterization can be delineated. First, the lesion’s dignity and etiology can be determined. Second, tumoral heterogeneity can be assessed by imaging biomarkers. Third, a radiogenomic characterization can help to identify lesion aggressiveness and response. Finally, imaging biomarkers can support targeting for biopsy and therapy. An overview of the criteria and methodology for imaging biomarkers is shown in Figure 4.

3.1. Lesion Dignity and Etiology Assessment

The differentiation of benign and malignant lesions and classification of them as primary or metastatic are critical clinical procedures in therapy planning. While visual criteria may guide lesion classification, the first evidence has been provided that novel imaging biomarkers can help in this regard and increase diagnostic accuracy.
In breast MRI, the application of a radiomics approach was able to perform lesion assessment and benign/malignant classification with similar accuracy to radiologists [78]. In identifying malignant lesions via mammography, radiomics classifiers could achieve an area under the curve of 0.90 in the test set [79]. Additionally, radiomics characteristics could be employed to distinguish benign and malignant peripheral nerve sheath tumors in neurofibromatosis type 1 with an area under the curve (AUC) of up to 0.94 [80].

3.2. Assessment of Tumoral Heterogeneity

When dealing with lesion characterization, the concept of tumoral heterogeneity, in particular, may be necessary to analyze. Heterogeneity can be subdivided into intra- and intertumoral heterogeneity. Intratumoral heterogeneity describes the heterogeneity inside a lesion, while intertumoral heterogeneity describes heterogeneity between different lesions.
In accordance with the concept of intertumoral heterogeneity, research studies have been able to show how the biology can differ from lesion to lesion. Interlesional molecular genetic heterogeneity has been detected in several cancers. In a disseminated tumor, in addition to the destructive growth of metastases into vital organs and vessels, there is also the issue of molecular genetic differentiation of metastatic lesions compared to the primary tumor, so that lesions with different degrees of therapy responsiveness and aggressiveness may coexist in the same patient at the same time. For example, phylogenetic analysis in ovarian cancer showed a high degree of heterogeneity over the course of time and in different lesions [81]. A multi-region analysis study by Gerlinger et al. was able to prove intratumoral heterogeneity and tumor evolution in primary renal carcinomas [82]. Heterogeneity has an effect on tumoral behavior, as it can change the degree of therapy response and lesion aggressiveness [83]. A quantitative assessment of heterogeneity could support decision making in lesion targeting for biopsy and locoregional interventional treatment. The assessment of interlesional heterogeneity based on imaging has not yet been studied comprehensively. First analyses, for example, by the utilization of radiomics parameters for unsupervised machine learning algorithms such as clustering, have been shown to be able to identify lesional subtypes and therefore provide a better understanding of imaging-based tumoral heterogeneity. For example, radiomics signatures of colorectal liver metastases can be clustered and subgroups of liver metastases correlated with clinical parameters can be identified [84]. PET analyses could also be used to identify tumoral heterogeneity by combining the information for multiple targets [85]. It is conceivable that quantifiable information about heterogeneity may be used as a theranostic tool to predict response and relapse.

3.3. Assessment of Aggressiveness and Response

The concept of tumor aggressiveness plays a vital role in therapy planning, as fast- growing and destructive tumors require a different intensity of therapy compared to slow-growing and less destructive tumors. The level of aggressiveness could be used to predict outcomes. Different works with volumetric assessments have shown in follicular lymphoma [86], colorectal cancer liver metastases [87], endometrial carcinomas [88], and peripheral T-cell carcinoma [89] that larger tumors tend to be more aggressive and have an influence on outcomes. However, volumetric assessment is of limited validity. Besides traditional analyses, it was demonstrated in patients with colorectal cancer that readily available biomarkers, such as CT attenuation/Hounsfield intensity values, could also be predictors of survival [20]. In line with these results, it was possible to use radiomics for aggressiveness prediction in papillary thyroid cancer using multiparametric MRI [90], in prostate cancer using bi-parametric MRI [91], and in diffuse lower-grade gliomas using diffusion- and perfusion-weighted sequences [92]. For prostate cancer, it was shown that differentiation and aggressiveness assessment using a radiomics-based model outperforms the traditional PI-RADS visual assessment. Chen et al. assessed the radiomics signature from a conventional and diffusion-weighted MRI. In their study, a radiomics trained logistic regression classifier (AUC = 0.985, 0.982, 0.999 (T2WI, ADC, T2WI, and ADC)) outperformed the PI-RADS V2 (AUC = 0.867) in diagnosing prostate cancer [93].
In modern therapy planning, the determination of individual genetic profiles is becoming increasingly important, if not almost essential. Linking mutation status to imaging phenotypes using quantitative imaging biomarkers can help to guide treatment decisions and improve survival prognosis. With computed tomography of lung adenocarcinoma, it is already possible to link mutational status (EGFR, KRAS, and ALK) with imaging-assessed characteristics, such as lymphangitis, pleural effusion, or lung metastases [94].
The discipline of linking genetics with imaging is called radiogenomics. Radiogenomic analyses have already been very successful in identifying various mutations [95]. Some examples of studies for the linking of radiomics and genetics are the detection of BRCA gene status in epithelial ovarian cancer [96], p53 and PD-L1 status in pancreatic cancer [97], the detection of p53 and IDH mutations in gliomas [98,99], the detection of EGFR status in brain metastases of lung adenocarcinoma [100], and KRAS status in rectal cancer [101]. Similar to radiogenomics, imaging-based deep learning and genetics have been linked, too. Thus, it was possible to detect IDH mutation status and PTEN mutation status in gliomas [102,103], KRAS status in patients with colorectal carcinoma [104], and EGFR status in lung adenocarcinoma [105]. Image-based recognition of genetic information can be highly relevant as a theranostic agent. For example, it has been demonstrated that EGFR mutations can change the response for EGFR tyrosine kinase inhibitors [106]. Thus, radiogenomics can serve the molecular classification of certain tumor subtypes, enabling rapid and precise diagnostic decisions. In case of disease progression or recurrence, radiogenomic approaches can identify aggressive and progredient lesions to guide therapeutic decisions in combination with molecular genetic tumor profiling either from biopsy or blood.
Furthermore, radiomics can be used to identify and quantify lesion imaging signatures and correlate them with response, aggressiveness, or growth patterns. First analyses have been able to identify subtypes in colorectal cancer liver metastases [84] and identify volumetric habitats in glioblastomas [107].

3.4. Targeting for Biopsy and Therapy

The studies presented here on tumoral heterogeneity and individual lesion biology suggest that biopsies of single lesions at a single time point during a patient’s journey cannot represent the full range of genetic diversity. Individual molecular biological analysis studies have shown how tumor biology differs over time and between different lesions [108]. Thus, smaller but more aggressive lesions might be missed and larger but less aggressive lesions biopsied instead. Furthermore, lesions could be fully missed if non-guided biopsies are performed. The usage of quantitative imaging biomarkers could improve targeting. This is notably important in the case of prostate carcinoma, where the spatial conditions mean that a solely MRI-guided biopsy does not always include clinically significant prostate cancer [109]. For example, combining prostate-specific antigen density and MRI for prostate biopsy planning allows the optimization of biopsy planning [110]. Interlenghi et al. were even able to predict the BI-RADS category for suspicious breast masses using ultrasound radiomics and training-modified random forest classifiers, support vector machines, and k-nearest neighbor classifiers [111]. Further developments could even lead to radiomics replacing biopsy and facilitate therapeutic decisions by predicting BI-RADS category by imaging.

4. Future Perspectives

The applications of quantitative imaging biomarkers presented in this paper are still in their infancy compared to other methods. Despite trends in research, for everyday clinical practice, deep learning or radiomics-based models are used sporadically, if at all. In the first place, the modern multi-omics technologies developed in molecular biology have made it possible to generate this new dimension of individual lesion characterization based on imaging. Without molecular biological multi-omics analyses, no ground truth can be defined and models cannot be trained properly. To enhance the progress in individual characterization, large datasets of multi-omics analyses need to be obtained to develop image-based markers. This could, for example, include a revival of research autopsies, which can help to understand tumor evolution [112]. Even though there are a large number of studies on specific questions using quantitative imaging biomarkers, there is a lack of more general methods that are, for example, scanner-independent.
In the future, a strong collaboration between laboratory medicine, pathology, and radiology will be essential to provide the best possible personalized therapy for patients with oncologic diseases. Models could be trained with a combination of imaging biomarkers and other non-invasively determined biomarkers, such as liquid profiling analyses. Liquid profiling describes the investigation of tumor genetics by blood analysis to determine circulating tumor DNA. Due to the lower hurdle of such a liquid biopsy compared to a traditional biopsy, genetic analyses can be performed at multiple time points [113]. This methodology has enormous potential and it could be employed as a personalized tumor marker, using patient- and tumor-specific alterations for tracking the genetic tumor evolution in blood. Importantly, the value as part of an integrative diagnostic approach in combination with radiomics to observe clonal heterogeneity in non-small cell lung cancer has been demonstrated recently [114].
Imaging biomarker analyses could be made part of clinical software (e.g., PACS) to provide integrated data analysis and risk stratification, prediction, and individual lesion characterization. By automatically incorporating them into diagnostic pathways, they could be used as a standardized element in therapy planning and clinical decision making in general.

5. Conclusions

Imaging biomarkers have great potential for lesion characterization and can help to foster the progress already made through molecular biological discoveries in a more low-threshold clinical setting. The potential applications, especially for lesion-specific assessment, are considerable, as shown by the many studies presented in this work. In order to improve quality of care and exploit the possibilities of personalized therapy, image-based assessment of tumor behavior is a promising approach for the future.

Author Contributions

Writing—original draft preparation, H.T., A.H. and S.R.; writing—review and editing, M.F.F., D.N., S.O.S. and V.H.; visualization, H.T. and M.F.F.; supervision, M.F.F. and S.O.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hofmarcher, T.; Lindgren, P.; Wilking, N.; Jönsson, B. The Cost of Cancer in Europe 2018. Eur. J. Cancer 2020, 129, 41–49. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Lei, J.; Ploner, A.; Elfström, K.M.; Wang, J.; Roth, A.; Fang, F.; Sundström, K.; Dillner, J.; Sparén, P. HPV Vaccination and the Risk of Invasive Cervical Cancer. N. Engl. J. Med. 2020, 383, 1340–1348. [Google Scholar] [CrossRef]
  3. Cardoso, R.; Zhu, A.; Guo, F.; Heisser, T.; Hoffmeister, M.; Brenner, H. Incidence and Mortality of Proximal and Distal Colorectal Cancer in Germany. Dtsch. Ärztebl. Int. 2021, 118, 281–287. [Google Scholar] [CrossRef] [PubMed]
  4. Brenner, H.; Stock, C.; Hoffmeister, M. Effect of Screening Sigmoidoscopy and Screening Colonoscopy on Colorectal Cancer Incidence and Mortality: Systematic Review and Meta-Analysis of Randomised Controlled Trials and Observational Studies. BMJ 2014, 348, g2467. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Torre, L.A.; Siegel, R.L.; Ward, E.M.; Jemal, A. Global Cancer Incidence and Mortality Rates and Trends—An Update. Cancer Epidemiol. Biomark. Prev. 2016, 25, 16–27. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Wang, H.; Naghavi, M.; Allen, C.; Barber, R.M.; Bhutta, Z.A.; Carter, A.; Casey, D.C.; Charlson, F.J.; Chen, A.Z.; Coates, M.M.; et al. Global, Regional, and National Life Expectancy, All-Cause Mortality, and Cause-Specific Mortality for 249 Causes of Death, 1980–2015: A Systematic Analysis for the Global Burden of Disease Study 2015. Lancet 2016, 388, 1459–1544. [Google Scholar] [CrossRef] [Green Version]
  7. Jonker, D.J.; O’Callaghan, C.J.; Karapetis, C.S.; Zalcberg, J.R.; Tu, D.; Au, H.-J.; Berry, S.R.; Krahn, M.; Price, T.; Simes, R.J.; et al. Cetuximab for the Treatment of Colorectal Cancer. N. Engl. J. Med. 2007, 357, 2040–2048. [Google Scholar] [CrossRef] [Green Version]
  8. Makary, M.S.; Khandpur, U.; Cloyd, J.M.; Mumtaz, K.; Dowell, J.D. Locoregional Therapy Approaches for Hepatocellular Carcinoma: Recent Advances and Management Strategies. Cancers 2020, 12, 1914. [Google Scholar] [CrossRef]
  9. Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New Response Evaluation Criteria in Solid Tumours: Revised RECIST Guideline (Version 1.1). Eur. J. Cancer 2009, 45, 228–247. [Google Scholar] [CrossRef]
  10. Schwartz, L.H.; Litière, S.; de Vries, E.; Ford, R.; Gwyther, S.; Mandrekar, S.; Shankar, L.; Bogaerts, J.; Chen, A.; Dancey, J.; et al. RECIST 1.1—Update and Clarification: From the RECIST Committee. Eur. J. Cancer 2016, 62, 132–137. [Google Scholar] [CrossRef] [Green Version]
  11. Ko, C.-C.; Yeh, L.-R.; Kuo, Y.-T.; Chen, J.-H. Imaging Biomarkers for Evaluating Tumor Response: RECIST and Beyond. Biomark. Res. 2021, 9, 52. [Google Scholar] [CrossRef] [PubMed]
  12. Birchard, K.R.; Hoang, J.K.; Herndon, J.E.; Patz, E.F. Early Changes in Tumor Size in Patients Treated for Advanced Stage Nonsmall Cell Lung Cancer Do Not Correlate with Survival. Cancer 2009, 115, 581–586. [Google Scholar] [CrossRef] [PubMed]
  13. Cousin, S.; Taieb, S.; Penel, N. A Paradigm Shift in Tumour Response Evaluation of Targeted Therapy: The Assessment of Novel Drugs in Exploratory Clinical Trials. Curr. Opin. Oncol. 2012, 24, 338–344. [Google Scholar] [CrossRef]
  14. Choi, H.; Charnsangavej, C.; Faria, S.C.; Macapinlac, H.A.; Burgess, M.A.; Patel, S.R.; Chen, L.L.; Podoloff, D.A.; Benjamin, R.S. Correlation of Computed Tomography and Positron Emission Tomography in Patients with Metastatic Gastrointestinal Stromal Tumor Treated at a Single Institution with Imatinib Mesylate: Proposal of New Computed Tomography Response Criteria. J. Clin. Oncol. 2007, 25, 1753–1759. [Google Scholar] [CrossRef]
  15. Weng, Z.; Ertle, J.; Zheng, S.; Lauenstein, T.; Mueller, S.; Bockisch, A.; Gerken, G.; Yang, D.; Schlaak, J.F. Choi Criteria Are Superior in Evaluating Tumor Response in Patients Treated with Transarterial Radioembolization for Hepatocellular Carcinoma. Oncol. Lett. 2013, 6, 1707–1712. [Google Scholar] [CrossRef] [Green Version]
  16. Heinemann, V.; Stintzing, S.; Modest, D.P.; Giessen-Jung, C.; Michl, M.; Mansmann, U.R. Early Tumour Shrinkage (ETS) and Depth of Response (DpR) in the Treatment of Patients with Metastatic Colorectal Cancer (MCRC). Eur. J. Cancer 2015, 51, 1927–1936. [Google Scholar] [CrossRef] [PubMed]
  17. Piessevaux, H.; Buyse, M.; Schlichting, M.; Van Cutsem, E.; Bokemeyer, C.; Heeger, S.; Tejpar, S. Use of Early Tumor Shrinkage to Predict Long-Term Outcome in Metastatic Colorectal Cancer Treated with Cetuximab. J. Clin. Oncol. 2013, 31, 3764–3775. [Google Scholar] [CrossRef]
  18. Winter, K.S.; Hofmann, F.O.; Thierfelder, K.M.; Holch, J.W.; Hesse, N.; Baumann, A.B.; Modest, D.P.; Stintzing, S.; Heinemann, V.; Ricke, J.; et al. Towards Volumetric Thresholds in RECIST 1.1: Therapeutic Response Assessment in Hepatic Metastases. Eur. Radiol. 2018, 28, 4839–4848. [Google Scholar] [CrossRef]
  19. Froelich, M.F.; Petersen, E.L.; Heinemann, V.; Nörenberg, D.; Hesse, N.; Gesenhues, A.B.; Modest, D.P.; Sommer, W.H.; Hofmann, F.O.; Stintzing, S.; et al. Impact of Size and Location of Metastases on Early Tumor Shrinkage and Depth of Response in Patients with Metastatic Colorectal Cancer: Subgroup Findings of the Randomized, Open-Label Phase 3 Trial FIRE-3/AIO KRK-0306. Clin. Colorectal Cancer 2020, 19, 291–300.e5. [Google Scholar] [CrossRef]
  20. Froelich, M.F.; Heinemann, V.; Sommer, W.H.; Holch, J.W.; Schoeppe, F.; Hesse, N.; Baumann, A.B.; Kunz, W.G.; Reiser, M.F.; Ricke, J.; et al. CT Attenuation of Liver Metastases before Targeted Therapy Is a Prognostic Factor of Overall Survival in Colorectal Cancer Patients. Results from the Randomised, Open-Label FIRE-3/AIO KRK0306 Trial. Eur. Radiol. 2018, 28, 5284–5292. [Google Scholar] [CrossRef]
  21. Nadler, A.; Cukier, M.; Rowsell, C.; Kamali, S.; Feinberg, Y.; Singh, S.; Law, C.H.L. Ki-67 Is a Reliable Pathological Grading Marker for Neuroendocrine Tumors. Virchows Arch. 2013, 462, 501–505. [Google Scholar] [CrossRef] [PubMed]
  22. Yip, S.S.F.; Liu, Y.; Parmar, C.; Li, Q.; Liu, S.; Qu, F.; Ye, Z.; Gillies, R.J.; Aerts, H.J.W.L. Associations between Radiologist-Defined Semantic and Automatically Computed Radiomic Features in Non-Small Cell Lung Cancer. Sci. Rep. 2017, 7, 3519. [Google Scholar] [CrossRef] [PubMed]
  23. Wang, X.; Zhou, B.; Xia, Y.; Zuo, J.; Liu, Y.; Bi, X.; Luo, X.; Zhang, C. A Methylation-Based Nomogram for Predicting Survival in Patients with Lung Adenocarcinoma. BMC Cancer 2021, 21, 801. [Google Scholar] [CrossRef]
  24. Hayashi, H.; Takiguchi, Y.; Minami, H.; Akiyoshi, K.; Segawa, Y.; Ueda, H.; Iwamoto, Y.; Kondoh, C.; Matsumoto, K.; Takahashi, S.; et al. Site-Specific and Targeted Therapy Based on Molecular Profiling by Next-Generation Sequencing for Cancer of Unknown Primary Site: A Nonrandomized Phase 2 Clinical Trial. JAMA Oncol. 2020, 6, 1931. [Google Scholar] [CrossRef] [PubMed]
  25. Wei, W.; Li, X.; Song, M.; Wang, C. Molecular Analysis of Oncogenic Mutations in Resected Margins by Next-Generation Sequencing Predicts Relapse in Non-Small Cell Lung Cancer Patients. OncoTargets Ther. 2020, 13, 9525–9531. [Google Scholar] [CrossRef]
  26. Malik, V.; Kalakoti, Y.; Sundar, D. Deep Learning Assisted Multi-Omics Integration for Survival and Drug-Response Prediction in Breast Cancer. BMC Genom. 2021, 22, 214. [Google Scholar] [CrossRef]
  27. Chaudhary, K.; Poirion, O.B.; Lu, L.; Garmire, L.X. Deep Learning–Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer. Clin. Cancer Res. 2018, 24, 1248–1259. [Google Scholar] [CrossRef] [Green Version]
  28. Snyder, A.; Makarov, V.; Merghoub, T.; Yuan, J.; Zaretsky, J.M.; Desrichard, A.; Walsh, L.A.; Postow, M.A.; Wong, P.; Ho, T.S.; et al. Genetic Basis for Clinical Response to CTLA-4 Blockade in Melanoma. N. Engl. J. Med. 2014, 371, 2189–2199. [Google Scholar] [CrossRef] [Green Version]
  29. Schumacher, T.N.; Schreiber, R.D. Neoantigens in Cancer Immunotherapy. Science 2015, 348, 69–74. [Google Scholar] [CrossRef] [Green Version]
  30. Gillies, R.J.; Kinahan, P.E.; Hricak, H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016, 278, 563–577. [Google Scholar] [CrossRef] [Green Version]
  31. Prayer, F.; Hofmanninger, J.; Weber, M.; Kifjak, D.; Willenpart, A.; Pan, J.; Röhrich, S.; Langs, G.; Prosch, H. Variability of Computed Tomography Radiomics Features of Fibrosing Interstitial Lung Disease: A Test-Retest Study. Methods 2021, 188, 98–104. [Google Scholar] [CrossRef] [PubMed]
  32. Jensen, L.J.; Kim, D.; Elgeti, T.; Steffen, I.G.; Hamm, B.; Nagel, S.N. Stability of Radiomic Features across Different Region of Interest Sizes—A CT and MR Phantom Study. Tomography 2021, 7, 238–252. [Google Scholar] [CrossRef] [PubMed]
  33. Wong, J.; Baine, M.; Wisnoskie, S.; Bennion, N.; Zheng, D.; Yu, L.; Dalal, V.; Hollingsworth, M.A.; Lin, C.; Zheng, D. Effects of Interobserver and Interdisciplinary Segmentation Variabilities on CT-Based Radiomics for Pancreatic Cancer. Sci. Rep. 2021, 11, 16328. [Google Scholar] [CrossRef] [PubMed]
  34. Lee, J.; Steinmann, A.; Ding, Y.; Lee, H.; Owens, C.; Wang, J.; Yang, J.; Followill, D.; Ger, R.; MacKin, D.; et al. Radiomics Feature Robustness as Measured Using an MRI Phantom. Sci. Rep. 2021, 11, 3973. [Google Scholar] [CrossRef]
  35. Lu, L.; Liang, Y.; Schwartz, L.H.; Zhao, B. Reliability of Radiomic Features Across Multiple Abdominal CT Image Acquisition Settings: A Pilot Study Using ACR CT Phantom. Tomography 2019, 5, 226–231. [Google Scholar] [CrossRef]
  36. Korte, J.C.; Cardenas, C.; Hardcastle, N.; Kron, T.; Wang, J.; Bahig, H.; Elgohari, B.; Ger, R.; Court, L.; Fuller, C.D.; et al. Radiomics Feature Stability of Open-Source Software Evaluated on Apparent Diffusion Coefficient Maps in Head and Neck Cancer. Sci. Rep. 2021, 11, 17633. [Google Scholar] [CrossRef]
  37. van Timmeren, J.E.; Leijenaar, R.T.H.; van Elmpt, W.; Wang, J.; Zhang, Z.; Dekker, A.; Lambin, P. Test–Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific? Tomography 2016, 2, 361–365. [Google Scholar] [CrossRef]
  38. Zwanenburg, A.; Vallières, M.; Abdalah, M.A.; Aerts, H.J.W.L.; Andrearczyk, V.; Apte, A.; Ashrafinia, S.; Bakas, S.; Beukinga, R.J.; Boellaard, R.; et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-Based Phenotyping. Radiology 2020, 295, 328–338. [Google Scholar] [CrossRef] [Green Version]
  39. Huang, Y.; Liu, Z.; He, L.; Chen, X.; Pan, D.; Ma, Z.; Liang, C.; Tian, J.; Liang, C. Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non—Small Cell Lung Cancer. Radiology 2016, 281, 947–957. [Google Scholar] [CrossRef]
  40. Shi, R.; Chen, W.; Yang, B.; Qu, J.; Cheng, Y.; Zhu, Z.; Gao, Y.; Wang, Q.; Liu, Y.; Li, Z.; et al. Prediction of KRAS, NRAS and BRAF Status in Colorectal Cancer Patients with Liver Metastasis Using a Deep Artificial Neural Network Based on Radiomics and Semantic Features. Am. J. Cancer Res. 2020, 10, 4513–4526. [Google Scholar]
  41. Enke, J.S.; Moltz, J.H.; D’Anastasi, M.; Kunz, W.G.; Schmidt, C.; Maurus, S.; Mühlberg, A.; Katzmann, A.; Sühling, M.; Hahn, H.; et al. Radiomics Features of the Spleen as Surrogates for CT-Based Lymphoma Diagnosis and Subtype Differentiation. Cancers 2022, 14, 713. [Google Scholar] [CrossRef] [PubMed]
  42. Woźnicki, P.; Westhoff, N.; Huber, T.; Riffel, P.; Froelich, M.F.; Gresser, E.; von Hardenberg, J.; Mühlberg, A.; Michel, M.S.; Schoenberg, S.O.; et al. Multiparametric MRI for Prostate Cancer Characterization: Combined Use of Radiomics Model with PI-RADS and Clinical Parameters. Cancers 2020, 12, 1767. [Google Scholar] [CrossRef] [PubMed]
  43. Shan, Q.; Hu, H.; Feng, S.; Peng, Z.; Chen, S.; Zhou, Q.; Li, X.; Xie, X.; Lu, M.; Wang, W.; et al. CT-Based Peritumoral Radiomics Signatures to Predict Early Recurrence in Hepatocellular Carcinoma after Curative Tumor Resection or Ablation. Cancer Imaging 2019, 19, 11. [Google Scholar] [CrossRef] [PubMed]
  44. Wang, W.; Cao, K.; Jin, S.; Zhu, X.; Ding, J.; Peng, W. Differentiation of Renal Cell Carcinoma Subtypes through MRI-Based Radiomics Analysis. Eur. Radiol. 2020, 30, 5738–5747. [Google Scholar] [CrossRef]
  45. Zheng, X.; Yao, Z.; Huang, Y.; Yu, Y.; Wang, Y.; Liu, Y.; Mao, R.; Li, F.; Xiao, Y.; Wang, Y.; et al. Deep Learning Radiomics Can Predict Axillary Lymph Node Status in Early-Stage Breast Cancer. Nat. Commun. 2020, 11, 1236. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Shu, Z.-Y.; Mao, D.-W.; Xu, Y.; Shao, Y.; Pang, P.-P.; Gong, X.-Y. Prediction of the Progression from Mild Cognitive Impairment to Alzheimer’s Disease Using a Radiomics-Integrated Model. Ther. Adv. Neurol. Disord. 2021, 14, 175628642110295. [Google Scholar] [CrossRef]
  47. De Perrot, T.; Hofmeister, J.; Burgermeister, S.; Martin, S.P.; Feutry, G.; Klein, J.; Montet, X. Differentiating Kidney Stones from Phleboliths in Unenhanced Low-Dose Computed Tomography Using Radiomics and Machine Learning. Eur. Radiol. 2019, 29, 4776–4782. [Google Scholar] [CrossRef]
  48. Liang, C.-H.; Liu, Y.-C.; Wan, Y.-L.; Yun, C.-H.; Wu, W.-J.; López-González, R.; Huang, W.-M. Quantification of Cancer-Developing Idiopathic Pulmonary Fibrosis Using Whole-Lung Texture Analysis of HRCT Images. Cancers 2021, 13, 5600. [Google Scholar] [CrossRef]
  49. Lin, A.; Kolossváry, M.; Cadet, S.; McElhinney, P.; Goeller, M.; Han, D.; Yuvaraj, J.; Nerlekar, N.; Slomka, P.J.; Marwan, M.; et al. Radiomics-Based Precision Phenotyping Identifies Unstable Coronary Plaques from Computed Tomography Angiography. JACC Cardiovasc. Imaging 2022, 15, 859–871. [Google Scholar] [CrossRef]
  50. Wang, Y.; Li, Y.; Huang, M.; Lai, Q.; Huang, J.; Chen, J. Feasibility of Constructing an Automatic Meniscus Injury Detection Model Based on Dual-Mode Magnetic Resonance Imaging (MRI) Radiomics of the Knee Joint. Comput. Math. Methods Med. 2022, 2022, 2155132. [Google Scholar] [CrossRef]
  51. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
  52. Van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.-C.; Pieper, S.; Aerts, H.J.W.L. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Depeursinge, A.; Andrearczyk, V.; Whybra, P.; van Griethuysen, J.; Müller, H.; Schaer, R.; Vallières, M.; Zwanenburg, A. Standardised Convolutional Filtering for Radiomics. arXiv 2021, arXiv:2006.05470. [Google Scholar]
  54. Parmar, C.; Grossmann, P.; Bussink, J.; Lambin, P.; Aerts, H.J.W.L. Machine Learning Methods for Quantitative Radiomic Biomarkers. Sci. Rep. 2015, 5, 13087. [Google Scholar] [CrossRef] [PubMed]
  55. Sun, P.; Wang, D.; Mok, V.C.; Shi, L. Comparison of Feature Selection Methods and Machine Learning Classifiers for Radiomics Analysis in Glioma Grading. IEEE Access 2019, 7, 102010–102020. [Google Scholar] [CrossRef]
  56. Vuong, D.; Tanadini-Lang, S.; Wu, Z.; Marks, R.; Unkelbach, J.; Hillinger, S.; Eboulet, E.I.; Thierstein, S.; Peters, S.; Pless, M.; et al. Radiomics Feature Activation Maps as a New Tool for Signature Interpretability. Front. Oncol. 2020, 10, 578895. [Google Scholar] [CrossRef]
  57. McCulloch, W.S.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5, 115–133. [Google Scholar] [CrossRef]
  58. Rajpurkar, P.; Chen, E.; Banerjee, O.; Topol, E.J. AI in Health and Medicine. Nat. Med. 2022, 28, 31–38. [Google Scholar] [CrossRef]
  59. Chlap, P.; Min, H.; Vandenberg, N.; Dowling, J.; Holloway, L.; Haworth, A. A Review of Medical Image Data Augmentation Techniques for Deep Learning Applications. J. Med. Imaging Radiat. Oncol. 2021, 65, 545–563. [Google Scholar] [CrossRef]
  60. Hosny, A.; Parmar, C.; Quackenbush, J.; Schwartz, L.H.; Aerts, H.J.W.L. Artificial Intelligence in Radiology. Nat. Rev. Cancer 2018, 18, 500–510. [Google Scholar] [CrossRef]
  61. McBee, M.P.; Awan, O.A.; Colucci, A.T.; Ghobadi, C.W.; Kadom, N.; Kansagra, A.P.; Tridandapani, S.; Auffermann, W.F. Deep Learning in Radiology. Acad. Radiol. 2018, 25, 1472–1480. [Google Scholar] [CrossRef] [Green Version]
  62. Esteva, A. A Guide to Deep Learning in Healthcare. Nat. Med. 2019, 25, 6. [Google Scholar] [CrossRef] [PubMed]
  63. Cirillo, D.; Catuara-Solarz, S.; Morey, C.; Guney, E.; Subirats, L.; Mellino, S.; Gigante, A.; Valencia, A.; Rementeria, M.J.; Chadha, A.S.; et al. Sex and Gender Differences and Biases in Artificial Intelligence for Biomedicine and Healthcare. NPJ Digit. Med. 2020, 3, 81. [Google Scholar] [CrossRef] [PubMed]
  64. Naik, N.; Hameed, B.M.Z.; Shetty, D.K.; Swain, D.; Shah, M.; Paul, R.; Aggarwal, K.; Ibrahim, S.; Patil, V.; Smriti, K.; et al. Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility? Front. Surg. 2022, 9, 862322. [Google Scholar] [CrossRef]
  65. Dankwa-Mullan, I.; Scheufele, E.L.; Matheny, M.E.; Quintana, Y.; Chapman, W.W.; Jackson, G.; South, B.R. A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle. J. Health Care Poor Underserved 2021, 32, 300–317. [Google Scholar] [CrossRef]
  66. Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jégou, H. Training Data-Efficient Image Transformers & Distillation through Attention 2021. In Proceedings of the 38th International Conference on Machine Learning, PMLR Virtual Conference, Online, 18–24 July 2021. [Google Scholar]
  67. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image Is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. arXiv 2021, arXiv:physics/2010.11929. Available online: http://arxiv.org/abs/physics/2010.11929 (accessed on 4 June 2022).
  68. Nie, P.; Yang, G.; Guo, J.; Chen, J.; Li, X.; Ji, Q.; Wu, J.; Cui, J.; Xu, W. A CT-Based Radiomics Nomogram for Differentiation of Focal Nodular Hyperplasia from Hepatocellular Carcinoma in the Non-Cirrhotic Liver. Cancer Imaging 2020, 20, 20. [Google Scholar] [CrossRef]
  69. Lang, N.; Zhang, Y.; Zhang, E.; Zhang, J.; Chow, D.; Chang, P.; Yu, H.J.; Yuan, H.; Su, M.-Y. Differentiation of Spinal Metastases Originated from Lung and Other Cancers Using Radiomics and Deep Learning Based on DCE-MRI. Magn. Reson. Imaging 2019, 64, 4–12. [Google Scholar] [CrossRef] [PubMed]
  70. Kirienko, M.; Cozzi, L.; Rossi, A.; Voulaz, E.; Antunovic, L.; Fogliata, A.; Chiti, A.; Sollini, M. Ability of FDG PET and CT Radiomics Features to Differentiate between Primary and Metastatic Lung Lesions. Eur. J. Nucl. Med. Mol. Imaging 2018, 45, 1649–1660. [Google Scholar] [CrossRef]
  71. Zheng, Y.; Li, J.; Liu, S.; Cui, J.; Zhan, J.; Pang, J.; Zhou, R.; Li, X.; Dong, C. MRI-Based Radiomics Nomogram for Differentiation of Benign and Malignant Lesions of the Parotid Gland. Eur. Radiol. 2021, 31, 4042–4052. [Google Scholar] [CrossRef]
  72. Hamm, C.A.; Wang, C.J.; Savic, L.J.; Ferrante, M.; Schobert, I.; Schlachter, T.; Lin, M.; Duncan, J.S.; Weinreb, J.C.; Chapiro, J.; et al. Deep Learning for Liver Tumor Diagnosis Part I: Development of a Convolutional Neural Network Classifier for Multi-Phasic MRI. Eur. Radiol. 2019, 29, 3338–3347. [Google Scholar] [CrossRef]
  73. Wang, C.J.; Hamm, C.A.; Savic, L.J.; Ferrante, M.; Schobert, I.; Schlachter, T.; Lin, M.; Weinreb, J.C.; Duncan, J.S.; Chapiro, J.; et al. Deep Learning for Liver Tumor Diagnosis Part II: Convolutional Neural Network Interpretation Using Radiologic Imaging Features. Eur. Radiol. 2019, 29, 3348–3357. [Google Scholar] [CrossRef] [PubMed]
  74. Venkadesh, K.V.; Setio, A.A.A.; Schreuder, A.; Scholten, E.T.; Chung, K.; Wille, M.M.W.; Saghir, Z.; van Ginneken, B.; Prokop, M.; Jacobs, C. Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT. Radiology 2021, 300, 438–447. [Google Scholar] [CrossRef] [PubMed]
  75. Ardila, D.; Kiraly, A.P.; Bharadwaj, S.; Choi, B.; Reicher, J.J.; Peng, L.; Tse, D.; Etemadi, M.; Ye, W.; Corrado, G.; et al. End-to-End Lung Cancer Screening with Three-Dimensional Deep Learning on Low-Dose Chest Computed Tomography. Nat. Med. 2019, 25, 954–961. [Google Scholar] [CrossRef]
  76. Al-masni, M.A.; Al-antari, M.A.; Park, J.-M.; Gi, G.; Kim, T.-Y.; Rivera, P.; Valarezo, E.; Choi, M.-T.; Han, S.-M.; Kim, T.-S. Simultaneous Detection and Classification of Breast Masses in Digital Mammograms via a Deep Learning YOLO-Based CAD System. Comput. Methods Programs Biomed. 2018, 157, 85–94. [Google Scholar] [CrossRef]
  77. Ehteshami Bejnordi, B.; Veta, M.; Johannes van Diest, P.; van Ginneken, B.; Karssemeijer, N.; Litjens, G.; van der Laak, J.A.W.M.; CAMELYON16 Consortium; Hermsen, M.; Manson, Q.F.; et al. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women with Breast Cancer. JAMA 2017, 318, 2199. [Google Scholar] [CrossRef]
  78. Daimiel Naranjo, I.; Gibbs, P.; Reiner, J.S.; Lo Gullo, R.; Thakur, S.B.; Jochelson, M.S.; Thakur, N.; Baltzer, P.A.T.; Helbich, T.H.; Pinker, K. Breast Lesion Classification with Multiparametric Breast MRI Using Radiomics and Machine Learning: A Comparison with Radiologists’ Performance. Cancers 2022, 14, 1743. [Google Scholar] [CrossRef]
  79. Wang, G. Radiomics Based on Digital Mammography Helps to Identify Mammographic Masses Suspicious for Cancer. Front. Oncol. 2022, 12, 10. [Google Scholar] [CrossRef] [PubMed]
  80. Ristow, I.; Madesta, F.; Well, L.; Shenas, F.; Wright, F.; Molwitz, I.; Farschtschi, S.; Bannas, P.; Adam, G.; Mautner, V.F.; et al. Evaluation of Magnetic Resonance Imaging-Based Radiomics Characteristics for Differentiation of Benign and Malignant Peripheral Nerve Sheath Tumors in Neurofibromatosis Type 1. Neuro-Oncology 2022, noac100. [Google Scholar] [CrossRef]
  81. Schwarz, R.F.; Ng, C.K.Y.; Cooke, S.L.; Newman, S.; Temple, J.; Piskorz, A.M.; Gale, D.; Sayal, K.; Murtaza, M.; Baldwin, P.J.; et al. Spatial and Temporal Heterogeneity in High-Grade Serous Ovarian Cancer: A Phylogenetic Analysis. PLoS Med. 2015, 12, e1001789. [Google Scholar] [CrossRef] [Green Version]
  82. Gerlinger, M.; Rowan, A.J.; Horswell, S.; Larkin, J.; Endesfelder, D.; Gronroos, E.; Martinez, P.; Matthews, N.; Stewart, A.; Tarpey, P.; et al. Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing. N. Engl. J. Med. 2012, 366, 883–892. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  83. Dagogo-Jack, I.; Shaw, A.T. Tumour Heterogeneity and Resistance to Cancer Therapies. Nat. Rev. Clin. Oncol. 2018, 15, 81–94. [Google Scholar] [CrossRef] [PubMed]
  84. Tharmaseelan, H.; Hertel, A.; Tollens, F.; Rink, J.; Woźnicki, P.; Haselmann, V.; Ayx, I.; Nörenberg, D.; Schoenberg, S.O.; Froelich, M.F. Identification of CT Imaging Phenotypes of Colorectal Liver Metastases from Radiomics Signatures—Towards Assessment of Interlesional Tumor Heterogeneity. Cancers 2022, 14, 1646. [Google Scholar] [CrossRef] [PubMed]
  85. Bailly, C.; Bodet-Milin, C.; Bourgeois, M.; Gouard, S.; Ansquer, C.; Barbaud, M.; Sébille, J.-C.; Chérel, M.; Kraeber-Bodéré, F.; Carlier, T. Exploring Tumor Heterogeneity Using PET Imaging: The Big Picture. Cancers 2019, 11, 1282. [Google Scholar] [CrossRef] [Green Version]
  86. Meignan, M.; Cottereau, A.S.; Versari, A.; Chartier, L.; Dupuis, J.; Boussetta, S.; Grassi, I.; Casasnovas, R.-O.; Haioun, C.; Tilly, H.; et al. Baseline Metabolic Tumor Volume Predicts Outcome in High–Tumor-Burden Follicular Lymphoma: A Pooled Analysis of Three Multicenter Studies. J. Clin. Oncol. 2016, 34, 3618–3626. [Google Scholar] [CrossRef]
  87. Mühlberg, A.; Holch, J.W.; Heinemann, V.; Huber, T.; Moltz, J.; Maurus, S.; Jäger, N.; Liu, L.; Froelich, M.F.; Katzmann, A.; et al. The Relevance of CT-Based Geometric and Radiomics Analysis of Whole Liver Tumor Burden to Predict Survival of Patients with Metastatic Colorectal Cancer. Eur. Radiol. 2021, 31, 834–846. [Google Scholar] [CrossRef] [PubMed]
  88. Ytre-Hauge, S.; Husby, J.A.; Magnussen, I.J.; Werner, H.M.J.; Salvesen, Ø.O.; Bjørge, L.; Trovik, J.; Stefansson, I.M.; Salvesen, H.B.; Haldorsen, I.S. Preoperative Tumor Size at MRI Predicts Deep Myometrial Invasion, Lymph Node Metastases, and Patient Outcome in Endometrial Carcinomas. Int. J. Gynecol. Cancer 2015, 25, 459–466. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  89. Cottereau, A.-S.; Hapdey, S.; Chartier, L.; Modzelewski, R.; Casasnovas, O.; Itti, E.; Tilly, H.; Vera, P.; Meignan, M.A.; Becker, S. Baseline Total Metabolic Tumor Volume Measured with Fixed or Different Adaptive Thresholding Methods Equally Predicts Outcome in Peripheral T Cell Lymphoma. J. Nucl. Med. 2017, 58, 276–281. [Google Scholar] [CrossRef] [Green Version]
  90. Dai, Z.; Wei, R.; Wang, H.; Hu, W.; Sun, X.; Zhu, J.; Li, H.; Ge, Y.; Song, B. Multimodality MRI-Based Radiomics for Aggressiveness Prediction in Papillary Thyroid Cancer. BMC Med. Imaging 2022, 22, 54. [Google Scholar] [CrossRef]
  91. Rodrigues, A.; Santinha, J.; Galvão, B.; Matos, C.; Couto, F.M.; Papanikolaou, N. Prediction of Prostate Cancer Disease Aggressiveness Using Bi-Parametric Mri Radiomics. Cancers 2021, 13, 6065. [Google Scholar] [CrossRef]
  92. Kim, M.; Jung, S.Y.; Park, J.E.; Jo, Y.; Park, S.Y.; Nam, S.J.; Kim, J.H.; Kim, H.S. Diffusion- and Perfusion-Weighted MRI Radiomics Model May Predict Isocitrate Dehydrogenase (IDH) Mutation and Tumor Aggressiveness in Diffuse Lower Grade Glioma. Eur. Radiol. 2020, 30, 2142–2151. [Google Scholar] [CrossRef]
  93. Chen, T.; Li, M.; Gu, Y.; Zhang, Y.; Yang, S.; Wei, C.; Wu, J.; Li, X.; Zhao, W.; Shen, J. Prostate Cancer Differentiation and Aggressiveness: Assessment with a Radiomic-Based Model vs. PI-RADS V2. J. Magn. Reson. Imaging 2019, 49, 875–884. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  94. Park, J.; Kobayashi, Y.; Urayama, K.Y.; Yamaura, H.; Yatabe, Y.; Hida, T. Imaging Characteristics of Driver Mutations in EGFR, KRAS, and ALK among Treatment-Naïve Patients with Advanced Lung Adenocarcinoma. PLoS ONE 2016, 11, e0161081. [Google Scholar] [CrossRef] [Green Version]
  95. Pinker, K.; Shitano, F.; Sala, E.; Do, R.K.; Young, R.J.; Wibmer, A.G.; Hricak, H.; Sutton, E.J.; Morris, E.A. Background, Current Role, and Potential Applications of Radiogenomics: Role and Applications of Radiogenomics. J. Magn. Reson. Imaging 2018, 47, 604–620. [Google Scholar] [CrossRef] [PubMed]
  96. Mingzhu, L.; Yaqiong, G.; Mengru, L.; Wei, W. Prediction of BRCA Gene Mutation Status in Epithelial Ovarian Cancer by Radiomics Models Based on 2D and 3D CT Images. BMC Med. Imaging 2021, 21, 180. [Google Scholar] [CrossRef]
  97. Iwatate, Y.; Hoshino, I.; Yokota, H.; Ishige, F.; Itami, M.; Mori, Y.; Chiba, S.; Arimitsu, H.; Yanagibashi, H.; Nagase, H.; et al. Radiogenomics for Predicting P53 Status, PD-L1 Expression, and Prognosis with Machine Learning in Pancreatic Cancer. Br. J. Cancer 2020, 123, 1253–1261. [Google Scholar] [CrossRef] [PubMed]
  98. Manikis, G.C.; Ioannidis, G.S.; Siakallis, L.; Nikiforaki, K.; Iv, M.; Vozlic, D.; Surlan-Popovic, K.; Wintermark, M.; Bisdas, S.; Marias, K. Multicenter DSC–MRI-Based Radiomics Predict IDH Mutation in Gliomas. Cancers 2021, 13, 3965. [Google Scholar] [CrossRef] [PubMed]
  99. Li, Y.; Qian, Z.; Xu, K.; Wang, K.; Fan, X.; Li, S.; Jiang, T.; Liu, X.; Wang, Y. MRI Features Predict P53 Status in Lower-Grade Gliomas via a Machine-Learning Approach. NeuroImage Clin. 2018, 17, 306–311. [Google Scholar] [CrossRef]
  100. Wang, G.; Wang, B.; Wang, Z.; Li, W.; Xiu, J.; Liu, Z.; Han, M. Radiomics Signature of Brain Metastasis: Prediction of EGFR Mutation Status. Eur. Radiol. 2021, 31, 4538–4547. [Google Scholar] [CrossRef]
  101. Cui, Y.; Liu, H.; Ren, J.; Du, X.; Xin, L.; Li, D.; Yang, X.; Wang, D. Development and Validation of a MRI-Based Radiomics Signature for Prediction of KRAS Mutation in Rectal Cancer. Eur. Radiol. 2020, 30, 1948–1958. [Google Scholar] [CrossRef]
  102. Liang, S.; Zhang, R.; Liang, D.; Song, T.; Ai, T.; Xia, C.; Xia, L.; Wang, Y. Multimodal 3D DenseNet for IDH Genotype Prediction in Gliomas. Genes 2018, 9, 382. [Google Scholar] [CrossRef] [Green Version]
  103. Chen, H.; Lin, F.; Zhang, J.; Lv, X.; Zhou, J.; Li, Z.-C.; Chen, Y. Deep Learning Radiomics to Predict PTEN Mutation Status from Magnetic Resonance Imaging in Patients with Glioma. Front. Oncol. 2021, 11, 734433. [Google Scholar] [CrossRef] [PubMed]
  104. He, K.; Liu, X.; Li, M.; Li, X.; Yang, H.; Zhang, H. Noninvasive KRAS Mutation Estimation in Colorectal Cancer Using a Deep Learning Method Based on CT Imaging. BMC Med. Imaging 2020, 20, 59. [Google Scholar] [CrossRef] [PubMed]
  105. Yin, G.; Wang, Z.; Song, Y.; Li, X.; Chen, Y.; Zhu, L.; Su, Q.; Dai, D.; Xu, W. Prediction of EGFR Mutation Status Based on 18F-FDG PET/CT Imaging Using Deep Learning-Based Model in Lung Adenocarcinoma. Front. Oncol. 2021, 11, 709137. [Google Scholar] [CrossRef] [PubMed]
  106. Nagano, T.; Tachihara, M.; Nishimura, Y. Mechanism of Resistance to Epidermal Growth Factor Receptor-Tyrosine Kinase Inhibitors and a Potential Treatment Strategy. Cells 2018, 7, 212. [Google Scholar] [CrossRef] [Green Version]
  107. Chiu, F.-Y.; Yen, Y. Efficient Radiomics-Based Classification of Multi-Parametric MR Images to Identify Volumetric Habitats and Signatures in Glioblastoma: A Machine Learning Approach. Cancers 2022, 14, 1475. [Google Scholar] [CrossRef]
  108. Iacobuzio-Donahue, C.A.; Michael, C.; Baez, P.; Kappagantula, R.; Hooper, J.E.; Hollman, T.J. Cancer Biology as Revealed by the Research Autopsy. Nat. Rev. Cancer 2019, 19, 686–697. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  109. Williams, C.; Ahdoot, M.; Daneshvar, M.A.; Hague, C.; Wilbur, A.R.; Gomella, P.T.; Shih, J.; Khondakar, N.; Yerram, N.; Mehralivand, S.; et al. Why Does Magnetic Resonance Imaging-Targeted Biopsy Miss Clinically Significant Cancer? J. Urol. 2022, 207, 95–107. [Google Scholar] [CrossRef] [PubMed]
  110. Falagario, U.G.; Jambor, I.; Lantz, A.; Ettala, O.; Stabile, A.; Taimen, P.; Aronen, H.J.; Knaapila, J.; Perez, I.M.; Gandaglia, G.; et al. Combined Use of Prostate-Specific Antigen Density and Magnetic Resonance Imaging for Prostate Biopsy Decision Planning: A Retrospective Multi-Institutional Study Using the Prostate Magnetic Resonance Imaging Outcome Database (PROMOD). Eur. Urol. Oncol. 2021, 4, 971–979. [Google Scholar] [CrossRef]
  111. Interlenghi, M.; Salvatore, C.; Magni, V.; Caldara, G.; Schiavon, E.; Cozzi, A.; Schiaffino, S.; Carbonaro, L.A.; Castiglioni, I.; Sardanelli, F. A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses. Diagnostics 2022, 12, 187. [Google Scholar] [CrossRef] [PubMed]
  112. Robb, T.J.; Tse, R.; Blenkiron, C. Reviving the Autopsy for Modern Cancer Evolution Research. Cancers 2021, 13, 409. [Google Scholar] [CrossRef]
  113. Haselmann, V.; Hedtke, M.; Neumaier, M. Liquid Profiling for Cancer Patient Stratification in Precision Medicine—Current Status and Challenges for Successful Implementation in Standard Care. Diagnostics 2022, 12, 748. [Google Scholar] [CrossRef] [PubMed]
  114. Cucchiara, F.; Del Re, M.; Valleggi, S.; Romei, C.; Petrini, I.; Lucchesi, M.; Crucitta, S.; Rofi, E.; De Liperi, A.; Chella, A.; et al. Integrating Liquid Biopsy and Radiomics to Monitor Clonal Heterogeneity of EGFR-Positive Non-Small Cell Lung Cancer. Front. Oncol. 2020, 10, 593831. [Google Scholar] [CrossRef] [PubMed]
Cancers 14 03349 g001 550
Figure 1. Comparison of traditional oncologic imaging approaches and quantitative imaging biomarkers for better lesion characterization toward personalized oncologic imaging.
Figure 1. Comparison of traditional oncologic imaging approaches and quantitative imaging biomarkers for better lesion characterization toward personalized oncologic imaging.
Cancers 14 03349 g001
Cancers 14 03349 g002 550
Figure 2. Analysis workflows for quantitative imaging biomarkers. (a) Workflow for radiomics analysis: Regions of Interest (ROIs) are segmented manually or automatically, following features of different categories (intensity-, texture-, or shape-based), which are extracted for each ROI. Extracted features are reduced and important features are outlined in the feature selection process. The final selection feature set can be used for classification. (b) Workflow for deep learning analysis: for supervised approaches the ground truth needs to be defined/the data need to be labeled. Following this, the neural networks can be trained, the learning process can be optimized by hyperparameter tuning, and the model validated on an external/unseen dataset.
Figure 2. Analysis workflows for quantitative imaging biomarkers. (a) Workflow for radiomics analysis: Regions of Interest (ROIs) are segmented manually or automatically, following features of different categories (intensity-, texture-, or shape-based), which are extracted for each ROI. Extracted features are reduced and important features are outlined in the feature selection process. The final selection feature set can be used for classification. (b) Workflow for deep learning analysis: for supervised approaches the ground truth needs to be defined/the data need to be labeled. Following this, the neural networks can be trained, the learning process can be optimized by hyperparameter tuning, and the model validated on an external/unseen dataset.
Cancers 14 03349 g002
Cancers 14 03349 g003 550
Figure 3. Overview of imaging biomarker methodology.
Figure 3. Overview of imaging biomarker methodology.
Cancers 14 03349 g003
Cancers 14 03349 g004 550
Figure 4. Criteria and methodology for novel imaging biomarkers. To solve various challenges, different methodologies can be used. This figure shows ideally applicable methods for (i) lesion dignity and etiology assessment, (ii) tumoral heterogeneity, (iii) aggressiveness and response, and (iv) targeting for biopsy and therapy.
Figure 4. Criteria and methodology for novel imaging biomarkers. To solve various challenges, different methodologies can be used. This figure shows ideally applicable methods for (i) lesion dignity and etiology assessment, (ii) tumoral heterogeneity, (iii) aggressiveness and response, and (iv) targeting for biopsy and therapy.
Cancers 14 03349 g004
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Tharmaseelan, H.; Hertel, A.; Rennebaum, S.; Nörenberg, D.; Haselmann, V.; Schoenberg, S.O.; Froelich, M.F. The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization. Cancers 2022, 14, 3349. https://doi.org/10.3390/cancers14143349

AMA Style

Tharmaseelan H, Hertel A, Rennebaum S, Nörenberg D, Haselmann V, Schoenberg SO, Froelich MF. The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization. Cancers. 2022; 14(14):3349. https://doi.org/10.3390/cancers14143349

Chicago/Turabian Style

Tharmaseelan, Hishan, Alexander Hertel, Shereen Rennebaum, Dominik Nörenberg, Verena Haselmann, Stefan O. Schoenberg, and Matthias F. Froelich. 2022. "The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization" Cancers 14, no. 14: 3349. https://doi.org/10.3390/cancers14143349

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop