Next Article in Journal
Development of a Diagnostic Model Focusing on Esophageal Dysmotility in Patients with Systemic Sclerosis
Next Article in Special Issue
A Systematic Review of Application Progress on Machine Learning-Based Natural Language Processing in Breast Cancer over the Past 5 Years
Previous Article in Journal
Circulating Biomarkers for Laboratory Diagnostics of Atherosclerosis—Literature Review
Previous Article in Special Issue
Breast Cancer Detection Using Automated Segmentation and Genetic Algorithms
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 12
Issue 12
10.3390/diagnostics12123140
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Quantitative Assessment of Breast-Tumor Stiffness Using Shear-Wave Elastography Histograms

by
Ismini Papageorgiou
1,2,*,
Nektarios A. Valous
3,4,
Stathis Hadjidemetriou
5,6,
Ulf Teichgräber
1 and
Ansgar Malich
2
1
Institute of Diagnostic and Interventional Radiology, Jena University Hospital—Friedrich Schiller University Jena, Am Klinikum 1, 07747 Jena, Germany
2
Institute of Radiology, Suedharz Hospital Nordhausen, Dr.-Robert-Koch-Str. 39, 99734 Nordhausen, Germany
3
Applied Tumor Immunity Clinical Cooperation Unit, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Im Neuenheimer Feld 460, 69120 Heidelberg, Germany
4
Center for Quantitative Analysis of Molecular and Cellular Biosystems (Bioquant), Heidelberg University, Im Neuenheimer Feld 267, 69120 Heidelberg, Germany
5
Applied Computer Science, Cyprus International Institute of Management, Akadimias Avenue 21, 2107 Nicosia, Cyprus
6
Department of Biological Sciences, University of Cyprus, 1678 Nicosia, Cyprus
*
Author to whom correspondence should be addressed.
Diagnostics 2022, 12(12), 3140; https://doi.org/10.3390/diagnostics12123140
Submission received: 31 October 2022 / Revised: 8 December 2022 / Accepted: 11 December 2022 / Published: 13 December 2022
(This article belongs to the Special Issue AI and Medical Imaging in Breast Disease)
Browse Figures
Review Reports Versions Notes

Abstract

:
Purpose: Shear-wave elastography (SWE) measures tissue elasticity using ultrasound waves. This study proposes a histogram-based SWE analysis to improve breast malignancy detection. Methods: N = 22/32 (patients/tumors) benign and n = 51/64 malignant breast tumors with histological ground truth. Colored SWE heatmaps were adjusted to a 0–180 kPa scale. Normalized, 250-binned RGB histograms were used as image descriptors based on skewness and area under curve (AUC). The histogram method was compared to conventional SWE metrics, such as (1) the qualitative 5-point scale classification and (2) average stiffness (SWEavg)/maximal tumor stiffness (SWEmax) within the tumor B-mode boundaries. Results: The SWEavg and SWEmax did not discriminate malignant lesions in this database, p > 0.05, rank sum test. RGB histograms, however, differed between malignant and benign tumors, p < 0.001, Kolmogorov–Smirnoff test. The AUC analysis of histograms revealed the reduction of soft-tissue components as a significant SWE biomarker (p = 0.03, rank sum). The diagnostic accuracy of the suggested method is still low (Se = 0.30 for Se = 0.90) and a subject for improvement in future studies. Conclusions: Histogram-based SWE quantitation improved the diagnostic accuracy for malignancy compared to conventional average SWE metrics. The sensitivity is a subject for improvement in future studies.

1. Introduction

Cancer tissue is “stiffer” than normal breast tissue. The stiffening process, a result of cell proliferation and extracellular matrix alterations, was correlated with cancer biology, perhaps with metastatic potential [1,2]. Shear-wave elastography (SWE™) is a sonographic method for the detection of tissue stiffness by measuring the attenuation of a shear wave delivered perpendicular to the tissue [3,4,5,6] (Figure 1a). Several SWE tissue applications [4,7] in the muscle–skeletal system [8] and liver [9,10,11], as well as innovative applications in vessel imaging [12], the small intestine in Crohn’s disease [13], thyroid nodules [14,15], the prostate [16] and breast ultrasound [17], ought to improve the diagnostic yield of B-mode. SWE implements a technology of two crossing ultrasound waves to gain an unbiased and user-independent estimate of tissue elasticity, which is considered advantageous compared to the mechanical pressure-based strain-elastography (SE) method [18]. The Guidelines and Recommendations of the World Federation of Ultrasound in Medicine and Biology (WFUMB) define the standard parameters for all clinically approved elastography applications, including those for SWE and SE in breast imaging [9]. SWE is a promising emerging asset to breast cancer screening, with a sensitivity (Se) and specificity (Spe) of ca. 96/85% [19]. SWE maps, though user-independent, are still vendor-specific and not directly comparable among devices—a hurdle for assembling large databases with high training quality for artificial intelligence (AI). This study used SWE images acquired with an AixPlorer® setup (SuperSonic Imagine™, Aix-en-Provence, France) in a single breast-cancer screening unit.
In contrast to qualitative metrics for strain elastography, known as the 5-tier Tsukuba–Ueno scale [20], SWE allows for a clear parametric readout of tissue stiffness in kPa. To effectively join function with anatomy, manufacturers project the kPa color-adjusted heatmap over the B-mode image to create a user-friendly joint image called an “elastogram” (Figure 1b). The SWE interface supports the custom definition of regions of interest (ROI) and calculates the average and maximum stiffness as characteristic tumor parameters (Figure 1b and Figure 2a). It is, however, a user´s interpretation as to whether the ROI encompasses or partially segments the tumor margins in B-mode. Hence, despite the clear parametric readout, two potential bias parameters emerge; (i) the user-defined ROI selection and (ii) the location of maximum attenuation of the shear wave. Previous reports show that the maximum shear wave attenuation sometimes occurs in the tumor’s surroundings [19], known as the “stiff rim” feature [21,22] (Figure 2a), which the strictly tumor-encompassing ROI definition might neglect. This study aimed to introduce an unbiased, quantitative, user-independent interpretation method of elastograms by extracting the red-green-blue (RGB) histograms. We conclude that the suggested histogram evaluation should provide improved, bias-free breast-tumor classification performance compared to the local average SWE metrics. In our future targets, we shall aim to define database reproducibility and standardization as essentials for the qualitative training of AI algorithms.

2. Materials and Methods

2.1. Patient Selection Criteria and Ground Truth

The study was retrospective for n = 73 female patients (n = 96 images) with histologically characterized breast tumors (n = 22/32 patients/unique lesions benign and n = 51/64 patients/unique lesions malignant). Patients were randomly recruited from a single-center breast-cancer database using randomization software (https://www.randomizer.org/, accessed 1 November 2022). An ultrasound-guided biopsy was the single inclusion criterion and served as the ground truth. No exclusion criteria were applied.

2.2. B-Mode and Shear-Wave Elastography Imaging

All patients were investigated with ultrasound in B-mode and SWE in a single session by the same user with more than 15 years of experience in elastography (AM). We used an Aixplorer® ultrasound setup (SuperSonic Imagine, Aix-en-Provence, France) with a SuperLinear™ probe SL 18-5 (5–18 MHz). The colored SWE spectrogram (known as “elastogram”) was the final diagnostic readout, displayed as an over-projection to the B-mode image in parallel screen mode (Figure 1b and Figure 2a). The 0–180 kPa heatmap color scale is a non-adjustable manufacturer´s standard feature, assigning low kPa values (high elasticity and low stiffness) to cool colors and high kPa values to warm colors (high tissue stiffness). A linear correlation is assumed as the baseline for further calculations (Figure 1b).

2.3. Image Analysis for Shear-Wave Elastography

The workflow for image analysis is graphically summarized in Figure 2. In detail, SWE images were analyzed by (a) a qualitative 5-point risk assessment scale, or the Tsukuba–Ueno scale, as suggested by Itoh et al. for strain elastography [20] and further implemented onto SWE by various groups [23,24,25] (Figure 2b), (b) the local average ROI-based metrics (Figure 2c), and (c) RGB histograms (Figure 2d).

2.3.1. Local Average SWE Metrics

SWE measures tissue stiffness in kPa. For the measurement, the SWE image is coupled to the B-mode image in a double-screen mode. The measurement region is defined with a circular ROI, which is manually positioned in the center of the target lesion as defined by the B-mode anatomical boundaries. The final readout is (1) the average tumor stiffness (SWEavg) and (2) maximal tumor stiffness (SWEmax) in kPa (Figure 2c). A reference ROI is placed in the healthy surrounding tissue (assigned as “ref”). The ratio of the average stiffness in the target ROI to the reference ROI (surrounding fat pad) is trademarked as the Q-box™ value (Figure 2c).

2.3.2. RGB Histogram

SWE heatmaps encompassing the target tumor and its surroundings were processed as described in (Figure 3). Heatmaps were orthogonally segmented to .jpeg data (Figure 3a,c) and imported to MATLAB 2017b (The MathWorks, Inc., Natick, MA, USA) using the inbuilt function “imread”. The histogram conversion assumes a linear correlation between (i) kPa and heatmap color code and (ii) during color-to-grayscale transformation. Grayscale components, such as black signal voids, were rejected. After R, G, and B channel extraction, each channel was augmented to fit a 256 × 256 binned matrix and exported to a 250-binned histogram using “imhist” (Figure 3b,d). Cumulative histograms for benign and malignant tumors were compared using the Kolmogorov–Smirnoff test in MATLAB (“kstest2”). Moreover, we quantitatively described the histograms by the area under curve (AUC) and interrogated the discriminative accuracy of the AUC by receiver operating curve (ROC) analysis.

2.4. Statistics and Software

Statistics and plots were performed in SigmaPlot (Systat Software, Inc., San Jose, CA, USA) and MATLAB 2021b (The MathWorks, Inc., Natick, CA, USA). Data distribution normality was assessed with a Shapiro–Wilk test. Parametric data were compared with a Kruskal–Wallis ANOVA on ranks with Dunn´s post-hoc test or Mann–Whitney rank sum test. Histograms were vectorized and compared with a Kolmogorov–Smirnoff test. ROC analysis was applied in SigmaPlot using an inbuilt macro. Graphical processing and halftones were supported by Inkscape (License name: GPL v2+, https://inkscape.org, accessed 1 October 2022).

3. Result

From the elastograms of n = 96 breast tumors (one image per tumor), we extracted RGB histograms with information from the tumor and its surroundings. Warm color (red) pixel values represent the stiff tumor components; blue pixel values stand for soft or fat-containing elements, whereas the green (“intermediate”) color is assigned to the tissue of intermediate stiffness (Figure 2a,d). Examples of RGB histogram extraction from a benign and malignant lesion are provided in Figure 3.

3.1. RGB Histograms Discriminate Malignant from Benign Tumors

Figure 4 illustrates the cumulative histograms for each channel (Figure 4a, warm colors; Figure 4b, intermediate colors; Figure 4c, cool colors) and tumor identity (gray line for benign, n = 32 and black line for malignant, n = 64). The Kolmogorov–Smirnoff statistical test showed a significant difference between malignant and benign lesions for all color channels, especially for the warm-color (p = 5.5 × 10−8) and cool-color (p = 1.36 × 10−4) components. The observed peak-broadening and rightward shift of the warm-color channel (Figure 4a), as well as the corresponding left-sided shift of the cool color channel (Figure 4c), graphically represent the increase in hard-tissue components (high-intensity bins, warm colors) and decrease in soft-tissue elements (low-intensity bins, cool colors) in cancer-including SWE image batches. We hypothesized that, besides the peak and average value descriptors, histograms include information about tissue anisotropy and texture that can enhance the discriminative capacity of SWE as a standalone tool [26,27].

3.2. The Reduction of Soft-Tissue Components Is a Potential Biomarker for Breast Malignancy

We further quantified RGB histograms by calculating the integral tissue signal for cool and warm colors, expressed as the AUC. To increase the discrimination capacity, the histograms of cool and warm colors were divided into “low-intensity bins” and “high-intensity bins.” From a whole histogram range of 1–250 bins, we rejected bins 0–14 and 201–250 as noise and defined the bin ranges 15–100 and 101–200 as “low” and “high-intensity bins, ” respectively. A rank sum comparison between benign and malignant lesions (Figure 4d–g) revealed a significant low-bin increase of cool colors in malignancy (Figure 4f); otherwise there were no significant differences (Figure 4d,e,g). Increase in low-intensity cool-colored bins is interpreted as a reduction of soft-tissue components in breast cancer compared to benign tumors. Interestingly, warm colors representing stiff tumor components showed no significant variation between cancer and benign tumors in this dataset (Figure 4d,e).
Hence, in this dataset, the reduction of soft-tissue components was more sensitive to discriminate malignancy than the increase in hard-tissue elements.

3.3. Receiver Operating Curves for RGB Histograms

Parametric testing of the AUC highlighted the low-intensity cool-colored bins as a potential cancer SWE biomarker with p = 0.03 at 95% CI (Figure 4f and Figure 5a). ROC analysis shows a low diagnostic accuracy, with sensitivity = 0.35 for specificity = 0.90. Increasing the sensitivity to 0.75 reduces the specificity to 0.30 (Figure 5b).
In contrast to the histogram analysis, local average SWE metrics in tumor-bound ROIs failed to reveal a statistically significant difference for malignancy in the same database (Figure 6a–d). Benign tumors were examined as a total and as dignity subgroups, fibroadenoma (FA), scar, in situ carcinoma (CIS), and adenosis (A). The average tumor stiffness (Figure 6a) and the maximum measured tumor stiffness (Figure 6b) showed a high variance (dots outside the box plots show the 75th to the 95th percentile) but were not significantly increased in cancerous lesions (histological ground truth) compared to lesions of benign nature, p > 0.05 Kruskal–Wallis ANOVA on ranks.
All in all, standalone analysis of SWE signal as implemented in this study discriminated between malignant and benign lesions (Figure 5) better than conventional descriptors (Figure 6), although still with a low diagnostic accuracy, which needs to be improved in future studies.
One of the technical drawbacks of the suggested RGB analysis method is the logistic demand and the need for integrated software with standard reference curves. The widely implemented Tsukuba 5-tier scale for the risk stratification of solid tumors in clinical practice requires no additional software. The Tsukuba scale tumor grading is only qualitative and empirical, based on the color appearance of the SWE image as graphically shown in Figure 7a. The color map is vendor-defined and cannot be thresholded by the user (unlike the doppler signal), therefore is a reliable and reproducible correlation between tissue stiffness in kPa and color representation. Still, it is inevitably subjected to user´s expectation based on the B-mode image. For this research dataset, the Tsukuba breast cancer (CA) score was statistically higher compared to the benign categories, p = 0.015 Kruskal–Wallis ANOVA on ranks (Figure 7). Nevertheless, the qualitative elastogram grading was not blinded to the B-mode features, which is, as commented above, a very likely source of bias.
Our results show that the quantitative heat map evaluation using RGB histograms could independently influence the diagnostic outcome.

4. Discussion

In this research project, we exploited the discriminative potential of the quantitative histogram analysis in breast SWE. In a dataset where the local average and peak quantitative values failed to classify malignancy (ANOVA p > 0.05, Figure 6), the split RGB histogram-based approach significantly improved the sensitivity of SWE for malignant versus benign lesions with p < 0.0001 at the 95% confidence interval. ROC analysis defined the reduction of soft-tissue components as a promising biomarker (Figure 5). Future studies should be designed to prove the current results, and improve the diagnostic accuracy by including clinical data along information with intra- and peritumoral regions such as in previous prominent research [28].

4.1. Discriminative Value of the Local SWE Average, State of the Art

Previous studies are mainly based on analyzing average and maximum SWE values from within the tumor boundaries. Au et al. [29], in a series of N = 123 masses, concluded that the elasticity ratio, i.e., the average strain ratio of the tumor to its surroundings (a Q-box equivalent), is the most sensitive discriminative factor for malignancy amongst strain descriptors. Yoon et al. [30] studied a series of N = 267 breast masses in order to highlight the value of the maximum SWE within tumor boundaries as the optimal discriminative criterion for malignancy. Yang et al. [31] used supersonic elastography in a study encompassing N = 226 subjects and showed a high discriminatory power of the mean and SD SWE values. In the latest peer-reviewed meta-analysis, Pillai et al. [32] included N = 19043 patients in a single receiver operating curve, proving the average SWE within tumor boundaries as the most sensitive discriminative parameter with an area under the curve (AUC) = 0.93.

4.2. The Role of Anisotropy in Tumor Classification

All above-cited studies focus on SWE measurements acquired within the tumor boundaries; the tumor surroundings serve as reference tissue for calculating an elasticity ratio [29]. However, amid previous reports on the high discriminative power of the average SWE, maximum SWE, or Q-box, in our medium-scale database of N = 96 breast masses, the standard SWE scores failed to discriminate for malignancy (Figure 6). Interestingly, the qualitative color map scale outperformed the SWE metrics and could differentiate between malignancy and benign tumors with a statistical significance illustrated in Figure 7. Evidence supporting the visual color map assessment of SWE elastograms has its foundations in numerous previous studies [23,24,30]. As noted above, the visual evaluation objectivity is jeopardized by B-mode-injected bias, which could explain the Ueno-scale superiority versus SWE values in this dataset. However, color maps include information on tumor anisotropy, which can add to their informational value. In a previous meta-analysis by Blank et al. [33] a series of N = 2989 patients revealed the importance of “stiffness gradient,” i.e., the slope between the maximum and minimum SWE tumor value, as an essential discriminative factor that outperformed standard measurements by 28% in malignancy classification. Slope gradient is a mesoscopic approach to quantify tumor anisotropy, further supported by evidence from the groups of Chen et al. [26] and Zhang et al. [27].

4.3. Histogram SWE Analysis, Experience from Previous Studies, and the Role of Deep Learning

In a previous study, Carlsen et al. reported equal discriminative efficacy for histograms and elastographic ratios [34] in a study paradigm implementing strain elastography (in this study, we implemented shear-wave elastography). Moreover, the study design of Carlsen et al. [34] focused on BIRADS-3 cases, whereas our research encompasses all BIRADS classes. Xue et al., in a more extensive study of N = 500, confirmed that the Ueno scale method performed equally to the SWE average values [35].
The approach of crossing the borders of qualitative color maps and extracting RGB histograms from the SWE data appear in previous literature, such as by Cheng et al. [36]. The authors used liver examinations to extract RGB histograms from color maps and merge them to organ “ultrasomics.” Remarkable achievements in breast-tumor classification were published by Zhang et al., Zhou et al., and Jiang et al. [37,38,39]. Independent researcher groups agree that convolutional neuronal network (CNN) training with cumulative B-mode and SWE data boosted the discriminative efficiency for malignancy. Part of the deep learning (DL) process implemented to improve the classification relies on the extraction of RGB histograms.
The essentiality of DL transparency, recently promoted under the term “explainable AI,” re-enforces the importance of knowing the independent effect of the intermediate processing steps. Notably, the suggested approach is free of hidden meta-data that may draw the attention of AI systems, such as numbers, letters, or other non-diagnostic image features. Such features can bias the results of sophisticated explainable AI (XAI) algorithms, meticulously summarized by the group of Van der Velden [40]. XAI training databases might benefit from data curation following the paradigm of this manuscript.

4.4. Effect of the ROI Size and Surrounding Tissue in SWE Interpretation

A final point that the authors would like to comment upon is the effect of ROI selection, size, and boundaries on the final SWE readout. Previous studies [41] have elucidated that the ROI size is an independent modulator of the SWE readout. The ROI-independent quantitative approach introduces objectivity in a traditional user-dependent method in ultrasonography and strips the final judgment from the inevitable B-mode-injected operator bias.
Previous research [42] implemented segmentation paradigms based on the B-mode intersection border, thus excluding the peritumoral shear-wave alterations. Lee et al., however, increased the accuracy of cancer detection by including the peritumoral tissue, thus elucidating the increased value of co-interpretation of tumoral and peritumoral patterns [43]. The “stiff-rim” sign in SWE describing a ring-like sink in elasticity in the peritumoral tissue was reported by the group of Zhou [22] and by Xu et al. [21]. Zhou et al. [38] introduced a segmentation method for preserving the peritumoral tissue that showed superior performance compared to its tumor intersection border predecessors. Using supersonic SWE, an alternative rim segmentation approach was suggested by Yu et al., implementing a hydraulic model for automatic tumor rim discrimination [31]. Though promising, the method is not yet commercialized, probably due to its high computational requirements. All in all, the increased discriminative capability of the RGB histograms relies, at least partly, on information from the tumoral and peritumoral milieu. In contrast, the average SWE metrics derive from strictly intratumoral selected ROIs.

4.5. Study Weaknesses

Our study was limited to a low-N, single-center, retrospective observational format. Regarding data interpretation, we would like to remark on the arbitrary use of the Tsukuba scale for SWE—a scale originally established for another elastography method (strain elastography) which may not apply to SWE with the same accuracy. However, SWE is a newly established elastography approach, and the Tsukuba–Ueno scale is the commonly used clinical standard, extrapolated from the classical strain elastography field. Moreover, our system is based on the assumption of a linear correlation between RGB tones and kPa values on the elastograms, which may inflict deviations from reality. Whether AixplorerTM uses a linear or retina-adapted color code is nevertheless a rigid feature, with possible influence on absolute values but not on the intergroup differences in our dataset. In future studies, we opt to address some underexplored points of this work, such as the effect of the lesion size and the definition of cut-off limits for the histogram implementation. Recruiting more patients from the central database and powering a retrospective dichotomized design might prove histograms as problem solvers in ambivalent tumor cases between surveillance and biopsy (BIRADS-3 class). Finally, an exciting point requiring a larger study population is the effect of the lesion size on the accuracy of SWE and the definition of a minimum necessary cut-off tumor size to guarantee elastogram reliability.

5. Conclusions and Clinical Significance

In this study, we applied quantitative shear-wave elastography using RGB histograms. The procedure is free of user bias, easily customizable, and showed improved discriminative ability between benign and malignant breast tumors compared to the standard manufacturer´s stiffness metrics. The RGB histogram evaluation could improve the standalone diagnostic accuracy of the SWE for breast tumors by 28% based on the ROC analysis (Youden index). Future studies should be designed to prove the current results, and improve the diagnostic accuracy by including clinical data along with information from intra- and peritumoral regions.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12123140/s1, Data_mat.

Author Contributions

I.P.: conceptualization, resources, writing original draft, and project administration; A.M. and U.T.: project administration and text editing; N.A.V. and S.H.: project administration, software, and text editing; A.M. and U.T.: supervision, validation, visualization, review, and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

All patient data derived from the breast-cancer database of a single center. Data were analyzed retrospectively and fully anonymized, following the ethical standards laid down in the 1964 Declaration of Helsinki and its amendments and the European Regulation 536/2014 and its latest addendum ICH GCP E6(R2)/2017, as well as with the guidelines of the Ethical Committee for clinical studies of the University affiliation of the corresponding investigator (IRB number 2019-1453-Daten).

Informed Consent Statement

Informed consent was acquired from all study participants, agreeing with the usage of anonymized data for general scientific purposes.

Data Availability Statement

Raw data are available, without restriction, in the Supplementary Materials.

Conflicts of Interest

The authors declare no competing financial interest.

Abbreviations

AIArtificial intelligence
ANOVAAnalysis of variance
AUCArea under curve
CACancer
CISCancer in situ
FAFibroadenoma
RGBRed-green-blue
ROCReceiver operating curve
ROIRegion of interest
SDStandard deviation
SEStrain elastography
SeSensitivity
SpeSpecificity
SWEShear-wave elastography
SWEavgAverage stiffness in kPa
SWEmaxMaximum stiffness in kPa
SWErefAverage stiffness of the reference region in kPa

References

  1. Baker, E.L.; Lu, J.; Yu, D.; Bonnecaze, R.T.; Zaman, M.H. Zaman, Cancer Cell Stiffness: Integrated Roles of Three-Dimensional Matrix Stiffness and Transforming Potential. Biophys. J. 2010, 99, 2048–2057. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Gkretsi, V.; Stylianopoulos, T. Cell Adhesion and Matrix Stiffness: Coordinating Cancer Cell Invasion and Metastasis. Front. Oncol. 2018, 8, 145. [Google Scholar] [CrossRef] [PubMed]
  3. Dietrich, C.F.; Barr, R.G.; Farrokh, A.; Dighe, M.; Hocke, M.; Jenssen, C.; Dong, Y.; Saftoiu, A.; Havre, R.F. Strain Elastography—How to Do It? Ultrasound Int. Open. 2017, 3, E137–E149. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Sigrist, R.M.S.; Liau, J.; Kaffas, A.E.; Chammas, M.C.; Willmann, J.K. Ultrasound Elastography: Review of Techniques and Clinical Applications. Theranostics 2017, 7, 1303–1329. [Google Scholar] [CrossRef]
  5. Gennisson, J.-L.; Deffieux, T.; Fink, M.; Tanter, M. Ultrasound elastography: Principles and techniques. Diagn. Interv. Imaging. 2013, 94, 487–495. [Google Scholar] [CrossRef]
  6. Jeong, W.K.; Lim, H.K.; Lee, H.-K.; Jo, J.M.; Kim, Y. Principles and clinical application of ultrasound elastography for diffuse liver disease. Ultrasonography (Seoul Korea) 2014, 33, 149–160. [Google Scholar] [CrossRef]
  7. Nitta, N.; Yamakawa, M.; Hachiya, H.; Shiina, T. A review of physical and engineering factors potentially affecting shear wave elastography. J. Med. Ultrason. 2021, 48, 403–414. [Google Scholar] [CrossRef]
  8. Prado-Costa, R.; Rebelo, J.; Monteiro-Barroso, J.; Preto, A.S. Ultrasound elastography: Compression elastography and shear-wave elastography in the assessment of tendon injury. Insights Into Imaging 2018, 9, 791–814. [Google Scholar] [CrossRef]
  9. Barr, R.G.; Nakashima, K.; Amy, D.; Cosgrove, D.; Farrokh, A.; Schafer, F.; Bamber, J.C.; Castera, L.; Choi, B.I.; Chou, Y.-H.; et al. WFUMB guidelines and recommendations for clinical use of ultrasound elastography: Part 2: Breast. Ultrasound Med. Biol. 2015, 41, 1148–1160. [Google Scholar] [CrossRef] [Green Version]
  10. Ferraioli, G. Review of Liver Elastography Guidelines. J. Ultrasound Med. 2019, 38, 9–14. [Google Scholar] [CrossRef]
  11. Kim, J.R.; Suh, C.H.; Yoon, H.M.; Lee, J.S.; Cho, Y.A.; Jung, A.Y. The diagnostic performance of shear-wave elastography for liver fibrosis in children and adolescents: A systematic review and diagnostic meta-analysis. Eur. Radiol. 2018, 28, 1175–1186. [Google Scholar] [CrossRef] [PubMed]
  12. Pruijssen, J.T.; de Korte, C.L.; Voss, I.; Hansen, H.H.G. Vascular Shear Wave Elastography in Atherosclerotic Arteries: A Systematic Review. Ultrasound Med. Biol. 2020, 46, 2145–2163. [Google Scholar] [CrossRef] [PubMed]
  13. Grażyńska, A.; Kufel, J.; Dudek, A.; Cebula, M. Shear Wave and Strain Elastography in Crohn’s Disease—A Systematic Review. Diagn. Basel Switz. 2021, 11, 1609. [Google Scholar] [CrossRef] [PubMed]
  14. Swan, K.Z.; Nielsen, V.E.; Bonnema, S.J. Evaluation of thyroid nodules by shear wave elastography: A review of current knowledge. J. Endocrinol. Investig. 2021, 44, 2043–2056. [Google Scholar] [CrossRef] [PubMed]
  15. Moraes, P.H.M.; Takahashi, M.S.; Vanderlei, F.A.B.; Schelini, M.V.; Chacon, D.A.; Tavares, M.R.; Chammas, M.C. Multiparametric Ultrasound Evaluation of the Thyroid: Elastography as a Key Tool in the Risk Prediction of Undetermined Nodules (Bethesda III and IV)—Histopathological Correlation. Ultrasound Med. Biol. 2021, 47, 1219–1226. [Google Scholar] [CrossRef]
  16. Anbarasan, T.; Wei, C.; Bamber, J.C.; Barr, R.G.; Nabi, G. Characterisation of Prostate Lesions Using Transrectal Shear Wave Elastography (SWE) Ultrasound Imaging: A Systematic Review. Cancers 2021, 13, 122. [Google Scholar] [CrossRef]
  17. Bian, J.; Zhang, J.; Hou, X. Diagnostic accuracy of ultrasound shear wave elastography combined with superb microvascular imaging for breast tumors: A protocol for systematic review and meta-analysis. Medicine (Baltimore) 2021, 100, e26262. [Google Scholar] [CrossRef]
  18. Carlsen, J.; Ewertsen, C.; Sletting, S.; Vejborg, I.; Schäfer, F.K.W.; Cosgrove, D.; Nielsen, M.B. Ultrasound Elastography in Breast Cancer Diagnosis. Ultraschall Med. Stuttg. Ger. 2015, 36, 550–565. [Google Scholar] [CrossRef]
  19. Chang, J.M.; Won, J.-K.; Lee, K.-B.; Park, I.A.; Yi, A.; Moon, W.K. Comparison of shear-wave and strain ultrasound elastography in the differentiation of benign and malignant breast lesions. AJR Am. J. Roentgenol. 2013, 201, W347–W356. [Google Scholar] [CrossRef]
  20. Itoh, A.; Ueno, E.; Tohno, E.; Kamma, H.; Takahashi, H.; Shiina, T.; Yamakawa, M.; Matsumura, T. Breast disease: Clinical application of US elastography for diagnosis. Radiology 2006, 239, 341–350. [Google Scholar] [CrossRef]
  21. Xu, Y.-J.; Gong, H.-L.; Hu, B.; Hu, B. Role of “Stiff Rim” sign obtained by shear wave elastography in diagnosis and guiding therapy of breast cancer. Int. J. Med. Sci. 2021, 18, 3615–3623. [Google Scholar] [CrossRef] [PubMed]
  22. Zhou, J.; Zhan, W.; Chang, C.; Zhang, X.; Jia, Y.; Dong, Y.; Zhou, C.; Sun, J.; Grant, E.G. Breast lesions: Evaluation with shear wave elastography, with special emphasis on the “stiff rim” sign. Radiology 2014, 272, 63–72. [Google Scholar] [CrossRef] [PubMed]
  23. Cong, R.; Li, J.; Guo, S. A new qualitative pattern classification of shear wave elastograghy for solid breast mass evaluation. Eur. J. Radiol. 2017, 87, 111–119. [Google Scholar] [CrossRef]
  24. Gweon, H.M.; Youk, J.H.; Son, E.J.; Kim, J.-A. Visually assessed colour overlay features in shear-wave elastography for breast masses: Quantification and diagnostic performance. Eur. Radiol. 2013, 23, 658–663. [Google Scholar] [CrossRef] [PubMed]
  25. Park, J.; Woo, O.H.; Shin, H.S.; Cho, K.R.; Seo, B.K.; Kang, E.Y. Diagnostic performance and color overlay pattern in shear wave elastography (SWE) for palpable breast mass. Eur. J. Radiol. 2015, 84, 1943–1948. [Google Scholar] [CrossRef]
  26. Chen, Y.-L.; Gao, Y.; Chang, C.; Wang, F.; Zeng, W.; Chen, J.-J. Ultrasound shear wave elastography of breast lesions: Correlation of anisotropy with clinical and histopathological findings. Cancer Imaging Off. Publ. Int. Cancer Imaging Soc. 2018, 18, 11. [Google Scholar] [CrossRef] [Green Version]
  27. Zhang, Q.; Xiao, Y.; Chen, S.; Wang, C.; Zheng, H. Quantification of elastic heterogeneity using contourlet-based texture analysis in shear-wave elastography for breast tumor classification. Ultrasound Med. Biol. 2015, 41, 588–600. [Google Scholar] [CrossRef]
  28. Bove, S.; Comes, M.C.; Lorusso, V.; Cristofaro, C.; Didonna, V.; Gatta, G.; Giotta, F.; la Forgia, D.; Latorre, A.; Pastena, M.I.; et al. A ultrasound-based radiomic approach to predict the nodal status in clinically negative breast cancer patients. Sci. Rep. 2022, 12, 7914. [Google Scholar] [CrossRef]
  29. Au, F.W.F.; Ghai, S.; Moshonov, H.; Kahn, H.; Brennan, C.; Dua, H.; Crystal, P. Crystal, Diagnostic performance of quantitative shear wave elastography in the evaluation of solid breast masses: Determination of the most discriminatory parameter. AJR Am. J. Roentgenol. 2014, 203, W328–W336. [Google Scholar] [CrossRef]
  30. Yoon, J.H.; Ko, K.H.; Jung, H.K.; Lee, J.T. Qualitative pattern classification of shear wave elastography for breast masses: How it correlates to quantitative measurements. Eur. J. Radiol. 2013, 82, 2199–2204. [Google Scholar] [CrossRef]
  31. Yu, Y.; Xiao, Y.; Cheng, J.; Chiu, B. Breast lesion classification based on supersonic shear-wave elastography and automated lesion segmentation from B-mode ultrasound images. Comput. Biol. Med. 2018, 93, 31–46. [Google Scholar] [CrossRef] [PubMed]
  32. Pillai, A.; Voruganti, T.; Barr, R.; Langdon, J. Diagnostic Accuracy of Shear-Wave Elastography for Breast Lesion Characterization in Women: A Systematic Review and Meta-Analysis. J. Am. Coll. Radiol. 2022, 19, 625–634.e0. [Google Scholar] [CrossRef] [PubMed]
  33. Blank, M.A.B.; Antaki, J.F. Breast Lesion Elastography Region of Interest Selection and Quantitative Heterogeneity: A Systematic Review and Meta-Analysis. Ultrasound Med. Biol. 2017, 43, 387–397. [Google Scholar] [CrossRef] [Green Version]
  34. Carlsen, J.F.; Ewertsen, C.; Sletting, S.; Talman, M.L.; Vejborg, I.; Bachmann Nielsen, M. Bachmann Nielsen, Strain histograms are equal to strain ratios in predicting malignancy in breast tumours. PLoS ONE 2017, 12, e0186230. [Google Scholar] [CrossRef] [Green Version]
  35. Xue, S.-S.; Zhao, Q.-L.; Ruan, L.-T.; Wang, F.-Q.; Zhou, C.; Sheng, W. Comparative analysis of the quantitative parameter method and elasticity color mode method for real-time shear wave elastography in the diagnosis of benign and malignant solid breast lesions. Tumori J. 2022, 108, 578–585. [Google Scholar] [CrossRef] [PubMed]
  36. Cheng, M.-Q.; Xian, M.-F.; Tian, W.-S.; Li, M.-D.; Hu, H.-T.; Li, W.; Zhang, J.-C.; Huang, Y.; Xie, X.-Y.; Lu, M.-D.; et al. RGB Three-Channel SWE-Based Ultrasomics Model: Improving the Efficiency in Differentiating Focal Liver Lesions. Front. Oncol. 2021, 11, 3772. [Google Scholar] [CrossRef] [PubMed]
  37. Zhang, X.; Liang, M.; Yang, Z.; Zheng, C.; Wu, J.; Ou, B.; Li, H.; Wu, X.; Luo, B.; Shen, J. Deep Learning-Based Radiomics of B-Mode Ultrasonography and Shear-Wave Elastography: Improved Performance in Breast Mass Classification. Front. Oncol. 2020, 10, 1621. [Google Scholar] [CrossRef]
  38. Zhou, Y.; Xu, J.; Liu, Q.; Li, C.; Liu, Z.; Wang, M.; Zheng, H.; Wang, S. A Radiomics Approach With CNN for Shear-Wave Elastography Breast Tumor Classification. IEEE Trans. Biomed. Eng. 2018, 65, 1935–1942. [Google Scholar] [CrossRef]
  39. Jiang, M.; Li, C.-L.; Chen, R.-X.; Tang, S.-C.; Lv, W.-Z.; Luo, X.-M.; Chuan, Z.-R.; Jin, C.-Y.; Liao, J.-T.; Cui, X.-W.; et al. Management of breast lesions seen on US images: Dual-model radiomics including shear-wave elastography may match performance of expert radiologists. Eur. J. Radiol. 2021, 141, 109781. [Google Scholar] [CrossRef]
  40. van der Velden, B.H.M.; Kuijf, H.J.; Gilhuijs, K.G.A.; Viergever, M.A. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med. Image Anal. 2022, 79, 102470. [Google Scholar] [CrossRef]
  41. Moon, J.H.; Hwang, J.-Y.; Park, J.S.; Koh, S.H.; Park, S.-Y. Impact of region of interest (ROI) size on the diagnostic performance of shear wave elastography in differentiating solid breast lesions. Acta Radiol. Stockh. Swed. 2018, 59, 657–663. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Huang, Y.-S.; Takada, E.; Konno, S.; Huang, C.-S.; Kuo, M.-H.; Chang, R.-F. Computer-Aided tumor diagnosis in 3-D breast elastography. Comput. Methods Programs Biomed. 2018, 153, 201–209. [Google Scholar] [CrossRef] [PubMed]
  43. Lee, S.; Jung, Y.; Bae, Y. Clinical application of a color map pattern on shear-wave elastography for invasive breast cancer. Surg. Oncol. 2016, 25, 44–48. [Google Scholar] [CrossRef] [PubMed]
Diagnostics 12 03140 g001 550
Figure 1. (a) Principles of shear-wave elastography. Shear-wave elastography (SWE) measures tissue density or stiffness in kPa through the mechanical distortion caused by an ultrasound wave. The transducer emits a perpendicular acoustic radiation force impulse (F), which causes a focal radial dislocation (Δl) and initiates a transverse wave, known as a shear wave. The propagation speed of the shear wave (Vs) is measured by a detection pulse emitted by the same transducer. The tissue density (ρ) is inversely proportional to the shear wave propagation speed square (ρ ~ 1/Vs2). (b) Sample image SWE (SuperSonic Imagine). The tissue stiffness of each pixel is measured in kPa and is color-coded in a spectrogram (“elastogram“) which is overlaid to the B-mode image in real time and can be projected in dual-mode to the B-mode image. As a manufacturer’s standard, high stiffness is warm-colored (red), and elastic tissue with low kPa values is cool-colored (blue).
Figure 1. (a) Principles of shear-wave elastography. Shear-wave elastography (SWE) measures tissue density or stiffness in kPa through the mechanical distortion caused by an ultrasound wave. The transducer emits a perpendicular acoustic radiation force impulse (F), which causes a focal radial dislocation (Δl) and initiates a transverse wave, known as a shear wave. The propagation speed of the shear wave (Vs) is measured by a detection pulse emitted by the same transducer. The tissue density (ρ) is inversely proportional to the shear wave propagation speed square (ρ ~ 1/Vs2). (b) Sample image SWE (SuperSonic Imagine). The tissue stiffness of each pixel is measured in kPa and is color-coded in a spectrogram (“elastogram“) which is overlaid to the B-mode image in real time and can be projected in dual-mode to the B-mode image. As a manufacturer’s standard, high stiffness is warm-colored (red), and elastic tissue with low kPa values is cool-colored (blue).
Diagnostics 12 03140 g001
Diagnostics 12 03140 g002 550
Figure 2. Workflow for image processing. (a) Sample image, SWE projected in dual mode with the B-mode parallel imaging. A region of interest (ROI) and a reference region (ref) are defined using the B-mode image to encase the primary lesion (ROI) and a representative reference sample of the surrounding fat tissue (ref). The color map provides pixel-based information on tissue elasticity, explained by the scale bar on the image’s left side and ROI-based statistics on the right (dotted box). Information on the ROI diameter and depth from the skin surface (depth) is also provided. (b) Qualitative lesion assessment based on the Tsukuba scale, increasing score from 1 to 5 reflects the probability for malignancy. (c) ROI statistics: average, minimum, maximum, and standard deviation (SD) of tissue elasticity in kPa. The Q-box RatioTM is the fraction of the average target ROI stiffness to the average stiffness of the reference region. (d) Segmentation of the whole SWE field for RGB analysis purposes. Image import, RGB channel splitting were performed in MATLAB as described in Materials and Methods section. Histogram differences were compared using the Kolmogorov–Smirnoff test. The diagnostic accuracy of the histogram area under curve (AUC) was interrogated with receiver operating curve (ROC) analysis. CA, cancer; SWE, shear-wave elastography.
Figure 2. Workflow for image processing. (a) Sample image, SWE projected in dual mode with the B-mode parallel imaging. A region of interest (ROI) and a reference region (ref) are defined using the B-mode image to encase the primary lesion (ROI) and a representative reference sample of the surrounding fat tissue (ref). The color map provides pixel-based information on tissue elasticity, explained by the scale bar on the image’s left side and ROI-based statistics on the right (dotted box). Information on the ROI diameter and depth from the skin surface (depth) is also provided. (b) Qualitative lesion assessment based on the Tsukuba scale, increasing score from 1 to 5 reflects the probability for malignancy. (c) ROI statistics: average, minimum, maximum, and standard deviation (SD) of tissue elasticity in kPa. The Q-box RatioTM is the fraction of the average target ROI stiffness to the average stiffness of the reference region. (d) Segmentation of the whole SWE field for RGB analysis purposes. Image import, RGB channel splitting were performed in MATLAB as described in Materials and Methods section. Histogram differences were compared using the Kolmogorov–Smirnoff test. The diagnostic accuracy of the histogram area under curve (AUC) was interrogated with receiver operating curve (ROC) analysis. CA, cancer; SWE, shear-wave elastography.
Diagnostics 12 03140 g002
Diagnostics 12 03140 g003 550
Figure 3. Sample elastogram and histograms. (a) Sample image of a benign breast lesion, (b) RGB histogram of the lesion (a), (c) sample image of a malignant breast lesion, and (d) RGB histogram of the lesion in (c). ROI, region of interest; ref, reference region (adjacent fat pad).
Figure 3. Sample elastogram and histograms. (a) Sample image of a benign breast lesion, (b) RGB histogram of the lesion (a), (c) sample image of a malignant breast lesion, and (d) RGB histogram of the lesion in (c). ROI, region of interest; ref, reference region (adjacent fat pad).
Diagnostics 12 03140 g003
Diagnostics 12 03140 g004 550
Figure 4. SWE histogram metrics. Cumulative histogram plots for the (a) warm colors, p = 5.5 × 10−8, Kolmogorov–Smirnoff test, (b) intermediate colors, p = 0.0258, Kolmogorov–Smirnoff test, and (c) cool colors, p = 1.36 × 10−4, Kolmogorov–Smirnoff test. (d–g) Area under curve (AUC) was calculated for the low-intensity bins (15–100) and the high-intensity bins (100–200). Normality was rejected with Shapiro–Wilk test. The AUC of benign to malignant tumor was compared with Mann–Whitney rank sum test. (f) Statistical significance was proven for the low-intensity bins of the cool colors, reflecting a reduction of soft-tissue components, p = 0.03, benign n = 32; cancer n = 64.
Figure 4. SWE histogram metrics. Cumulative histogram plots for the (a) warm colors, p = 5.5 × 10−8, Kolmogorov–Smirnoff test, (b) intermediate colors, p = 0.0258, Kolmogorov–Smirnoff test, and (c) cool colors, p = 1.36 × 10−4, Kolmogorov–Smirnoff test. (d–g) Area under curve (AUC) was calculated for the low-intensity bins (15–100) and the high-intensity bins (100–200). Normality was rejected with Shapiro–Wilk test. The AUC of benign to malignant tumor was compared with Mann–Whitney rank sum test. (f) Statistical significance was proven for the low-intensity bins of the cool colors, reflecting a reduction of soft-tissue components, p = 0.03, benign n = 32; cancer n = 64.
Diagnostics 12 03140 g004
Diagnostics 12 03140 g005 550
Figure 5. Receiver operating curves (ROC) of cool color histograms. (a) The best diagnostic accuracy was revealed for the decreased soft-tissue components (expressed as increased low intensity histogram bins) in malignant lesions, p = 0.03, rank sum test. (b) ROC shows a low diagnostic accuracy, Se = 0.3/Spe = 0.9, which is not sufficient as a standalone tool, but sheds light onto new image analysis aspects.
Figure 5. Receiver operating curves (ROC) of cool color histograms. (a) The best diagnostic accuracy was revealed for the decreased soft-tissue components (expressed as increased low intensity histogram bins) in malignant lesions, p = 0.03, rank sum test. (b) ROC shows a low diagnostic accuracy, Se = 0.3/Spe = 0.9, which is not sufficient as a standalone tool, but sheds light onto new image analysis aspects.
Diagnostics 12 03140 g005
Diagnostics 12 03140 g006 550
Figure 6. Average SWE metrics of the inbound tumor ROI. (a) Average stiffness from target ROI within the B-mode tumor boundaries, (b) average stiffness from reference surrounding fat tissue region, (c) maximum stiffness from target ROI, and (d) Q-box ratio for cancer (CA, n = 62 patients) and benign lesions (n = 32 patients). Box plots show the median and interquartile range, dots within the 5–95th percentile. FA, fibroadenoma; SCAR, scar; CIS, cancer in situ; A, adenosis. (a–d) p > 0.05, Kruskal–Wallis ANOVA on ranks.
Figure 6. Average SWE metrics of the inbound tumor ROI. (a) Average stiffness from target ROI within the B-mode tumor boundaries, (b) average stiffness from reference surrounding fat tissue region, (c) maximum stiffness from target ROI, and (d) Q-box ratio for cancer (CA, n = 62 patients) and benign lesions (n = 32 patients). Box plots show the median and interquartile range, dots within the 5–95th percentile. FA, fibroadenoma; SCAR, scar; CIS, cancer in situ; A, adenosis. (a–d) p > 0.05, Kruskal–Wallis ANOVA on ranks.
Diagnostics 12 03140 g006
Diagnostics 12 03140 g007 550
Figure 7. Qualitative elastography assessment. (a) Qualitative evaluation of the elastogram in a scale 1–5 with criteria similar to the Tsukuba–Ueno scale (b) Box plot showing the median and interquartile range, dots within the 5–95th percentile. Cancer (CA, n = 62 tumors) and benign lesions (n = 32 tumors), p = 0.015, Kruskal–Wallis ANOVA on ranks with Dunn’s post-hoc test. Note that the qualitative elastogram grading cannot be blinded to the B-mode features, which is a likely source of bias. FA, fibroadenoma; SCAR, scar; CIS, cancer in situ; A, adenosis.
Figure 7. Qualitative elastography assessment. (a) Qualitative evaluation of the elastogram in a scale 1–5 with criteria similar to the Tsukuba–Ueno scale (b) Box plot showing the median and interquartile range, dots within the 5–95th percentile. Cancer (CA, n = 62 tumors) and benign lesions (n = 32 tumors), p = 0.015, Kruskal–Wallis ANOVA on ranks with Dunn’s post-hoc test. Note that the qualitative elastogram grading cannot be blinded to the B-mode features, which is a likely source of bias. FA, fibroadenoma; SCAR, scar; CIS, cancer in situ; A, adenosis.
Diagnostics 12 03140 g007
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Papageorgiou, I.; Valous, N.A.; Hadjidemetriou, S.; Teichgräber, U.; Malich, A. Quantitative Assessment of Breast-Tumor Stiffness Using Shear-Wave Elastography Histograms. Diagnostics 2022, 12, 3140. https://doi.org/10.3390/diagnostics12123140

AMA Style

Papageorgiou I, Valous NA, Hadjidemetriou S, Teichgräber U, Malich A. Quantitative Assessment of Breast-Tumor Stiffness Using Shear-Wave Elastography Histograms. Diagnostics. 2022; 12(12):3140. https://doi.org/10.3390/diagnostics12123140

Chicago/Turabian Style

Papageorgiou, Ismini, Nektarios A. Valous, Stathis Hadjidemetriou, Ulf Teichgräber, and Ansgar Malich. 2022. "Quantitative Assessment of Breast-Tumor Stiffness Using Shear-Wave Elastography Histograms" Diagnostics 12, no. 12: 3140. https://doi.org/10.3390/diagnostics12123140

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop