Knee MRI Underestimates the Grade of Cartilage Lesions

: Purpose: This study was conducted in order to evaluate the clinical utility of MRI in detecting cartilage lesions and its dependence on anatomical location and lesion grade. Methods: A retrospective analysis of MRI reports and arthroscopic ﬁndings was performed on 190 consecutive patients treated in one orthopaedic department. MRI protocols were prepared by 18 radiologists from 10 different MRI centers with the use of 1.5 T magnets. The image protocols were selected by reading radiologists. Four hundred and ﬁfty-three chondral lesions in ﬁve anatomic locations were identiﬁed during this study and graded according to the ICRS classiﬁcation. Sensitivity, speciﬁcity, receiver operating characteristic (ROC), and Bangdiwala’s observer agreement charts were utilized to evaluate the diagnostic performance. Results: Only approximately 30% of MRI showed an adequate cartilage status in all anatomical locations. The sensitivity ranged from 92% in healthy cartilage to 5% in grade I lesions. The speciﬁcity differed also grossly depending on the lesion grade, reaching 96.5% in grade four lesions and 38% in healthy cartilage. The medial compartment Bangdiwala’s observer agreement charts show a gross underestimation of cartilage lesions, and the area under the curve (AUC) of ROC surpasses 0.7 only in the medial femoral condyle and patella-femoral joint. Overall, the medial compartment accuracy was signiﬁcantly higher than the lateral compartment. The MRI showed correspondence of its diagnostic performance with cartilage lesion severity. Conclusion: MRI underestimates the extent of cartilage injury and evaluation of cartilage defects based on MRI should be taken with caution by orthopaedic surgeons in planning surgery. Surgical planning on MRI should take cartilage lesions under consideration, even if no cartilage lesions are reported on the MRI.


Introduction
Chondral defects are a common cause of knee dysfunction found in over 60% of patients requiring arthroscopy [1]. It has become an important topic due to the poor healing capacity of the cartilage and the progressive nature of chondral defects irreversibly leading to osteoarthritis if untreated [2]. The complex mesh of collagen fibres enables the cartilage to withstand shearing forces, while water accumulated by proteoglycans is responsible for resisting compressive forces. The only widely accessible non-invasive modality appropriate for the detection of cartilage lesions is magnetic resonance imaging (MRI) [3]. However, the literature suggests that a pure morphologic evaluation of cartilage lacks sensitivity in detecting cartilage defects and provides insufficient information regarding exact cartilage grading [3]. Moreover, the sensitivity of the examination differs grossly in the literature, ranging from 45% up to 94% [4,5], with some authors even suggesting that the MRI might not be helpful in cases with a typical history and firm clinical findings [6]. In pursuit of a better cartilage evaluation, new imaging techniques based on MRI have been introduced in recent years. These techniques include sodium imaging, delayed gadoliniumenhanced MRI of cartilage (dGEMRIC), glycosaminioglycan specific chemical exchange saturation transfer (gagCEST), diffusion-weighted imaging (DWI), diffusion tensor imaging (DTI), and relaxometry measurements [7]. Such techniques enable the differentiation between a healthy and repaired cartilage [8], can correspond with glycosaminoglycans concentration [9][10][11], or enable the quantitative monitoring of macromolecules in early osteoarthritis [12]. Therefore, the techniques in question might be suitable for the detection and evaluation of early cartilage changes as well as an accurate cartilage evaluation. However, the techniques present certain limitations. First, the availability of specialized techniques is limited. Moreover, dGEMRIC requires gadolinium administration and potential side effects can be encountered [13]. Joint effusion can interfere with the gagCEST evaluation due to the presence of glycosaminoglycans in the synovial fluid [14].
Even after introducing a 3D MRI [15], which exhibits high sensitivity to grade III and IV lesions [16] and is more accessible than the compositional techniques, the 1.5 T MRI examination remains a workhorse for cartilage evaluation. Of special interest to the orthopaedic surgeons is the estimation of cartilage defect grade, size, and location, which strongly influences the decision of switching from a conservative to surgical treatment [17,18]. However, some authors suggest that the information obtained by MRI does not change scheduled arthroscopic surgeries and does not affect the overall treatment costs [19]. To date, there is a sparse number of articles presenting the diagnostic efficacy of MRI regarding the grade and lesion location in a typical clinical setting. A great number of papers published on the subject depend on the evaluation performed by a musculoskeletal radiologist, who is trained specifically for such diagnosis [20]. However, in many instances, the MRI is evaluated by a general radiologist, or often a referring orthopaedic surgeon that has no knowledge whether the radiologist specializes in musculoskeletal disorders. Moreover, clinical experience suggests that a routinely performed MRI may show significant differences in regard to the diagnostic accuracy of cartilage lesion depending on a lesion grade and anatomic location.
The aim of this study was to evaluate the sensitivity and specificity of MRI in detecting cartilage lesions and its dependence on anatomical location and lesion grade in the knee joint.

Materials and Methods
This retrospective cohort study was conducted on medical records of patients treated in the orthopaedic department of Leczna Hospital in Poland between September 2014 and February 2020. During this time, 190 consecutive knee arthroscopies with preoperatively obtained and stored MRI images and reports were performed. A bioethical committee at the Medical University in Lublin, with approval number KE-0254/262/2019, was obtained for the study. In addition, each patient signed a written consent concerning the use of medical data for clinical purposes without revealing personal data. Only examinations performed on 1.5 Tesla MRI with a full written report in the medical history were included in the study. MRI protocols were selected by radiology specialists based on an examination referral. In this study, we have not gained access to the referrals and had no influence on the MRI protocols or sequences used in the examination, which presents a common clinical setting. The MRI reports were prepared by 18 radiologists from 10 different MRI centres. There is no data concerning the experience or the field of interest of radiologists evaluating the images. The information of cartilage status was noted, and each articular cartilage deviation was noted in a written report. If no information on the cartilage defects was given, then the cartilage was regarded as a grade O as per the International Cartilage Repair Society (ICRS) [21,22]. A grade 0 for cartilage defects which are not mentioned was assumed due to the fact that an orthopaedic surgeon reading an MRI report will consider the lack of mentioning of cartilage defects as a healthy cartilage. This assumption was made in 57 patients. Arthroscopic surgery was a gold standard for the estimation of MRI diagnostic accuracy. All the surgeries were performed by three surgeons specializing in sports knee surgery with an experience of 5-15 years, with a standard 30-degree optics in a standardized eight-step knee evaluation. Typical antero-lateral and antero-medial portals with accessory portals (when indicated) were utilized for the surgery. Indications for the surgery were made by the treating surgeon, based on the medical history, along with clinical and radiological findings. During arthroscopy, chondral lesions were graded 0-4 with the use of the ICRS classification system [21,22]. Images presenting cartilage lesions are shown in Figures 1-4. Locations such as medial femoral condyle (MFC), lateral femoral condyle (LFC), medial tibial condyle (MTC), lateral tibial condyle (LTC), and patellofemoral joint (PFJ) were evaluated. All the tissues during the arthroscopy were visualised and examined with a probe. Images of all the intraoperative findings were taken during the surgery and stored in the hospital database. The data concerning the cartilage status during arthroscopy was obtained from operative notes.
Appl. Sci. 2021, 11, x FOR PEER REVIEW 3 of 17 sports knee surgery with an experience of 5-15 years, with a standard 30-degree optics in a standardized eight-step knee evaluation. Typical antero-lateral and antero-medial portals with accessory portals (when indicated) were utilized for the surgery. Indications for the surgery were made by the treating surgeon, based on the medical history, along with clinical and radiological findings. During arthroscopy, chondral lesions were graded 0-4 with the use of the ICRS classification system [21,22]. Images presenting cartilage lesions are shown in Figures 1-4. Locations such as medial femoral condyle (MFC), lateral femoral condyle (LFC), medial tibial condyle (MTC), lateral tibial condyle (LTC), and patellofemoral joint (PFJ) were evaluated. All the tissues during the arthroscopy were visualised and examined with a probe. Images of all the intraoperative findings were taken during the surgery and stored in the hospital database. The data concerning the cartilage status during arthroscopy was obtained from operative notes.

Statistical Analysis
The data were analysed by a statistics specialist with the use of Microsoft Excel and the IBM SPSS software. The overall capacity of MRI for detecting chondral lesions was estimated with the use of the receiver operating characteristic (ROC) curves. Additionally, the sensitivity and specificity for each anatomic location and the grade of lesion were counted, which is easy to incorporate in daily practice. These are values that describe the ability of a test to detect the trait being tested (sensitivity) or detect the absence of that trait (specificity). The sensitivity of a test is the ratio of true positives (TP) to the sum of true positives (TP) and false negatives (FN). Test specificity is the ratio of true negatives (TN) to the sum of true negatives (TN) and false positives (FP). The MRI diagnostic accuracy for each location was evaluated with the use of weighted Kappa [23] and visualized with Bangdiwala's observer agreement charts [24], where a graphical presentation of the results compliance is determined by the size of the field. A black colour indicates compliance, while a grey colour indicates one adjacent level of compliance. The direction of the observer bias is verified by examining the "path of the rectangles" and its deviation from the diagonal line [24][25][26]. A perfect congruence is when the black rectangles line up along the red line and they are black, not grey. Rectangles along the diagonal but partially grey signify a partial agreement, but if they move away from the diagonal, then there is a systematic error in the method that underestimates or overestimates the diagnosis.The Kappa statistics proposed by the B statistics are used to quantify the agreement between two observers, independently classifying the same n units into the same k category [27]. The Kappa Cohen takes values from 0 to 1. The closer to 1, the more consistent the two observer's assessments are, the closer to 0, the more divergent the marks are [28]. The most commonly used guidelines for Kappa are due to Landis and Koch [29]. The interpretation of the Kappa score according to Landis and Koch is as follows: 0-poor, 0 to 0.2 slight, 0.2

Statistical Analysis
The data were analysed by a statistics specialist with the use of Microsoft Excel and the IBM SPSS software. The overall capacity of MRI for detecting chondral lesions was estimated with the use of the receiver operating characteristic (ROC) curves. Additionally, the sensitivity and specificity for each anatomic location and the grade of lesion were counted, which is easy to incorporate in daily practice. These are values that describe the ability of a test to detect the trait being tested (sensitivity) or detect the absence of that trait (specificity). The sensitivity of a test is the ratio of true positives (TP) to the sum of true positives (TP) and false negatives (FN). Test specificity is the ratio of true negatives (TN) to the sum of true negatives (TN) and false positives (FP). The MRI diagnostic accuracy for each location was evaluated with the use of weighted Kappa [23] and visualized with Bangdiwala's observer agreement charts [24], where a graphical presentation of the results compliance is determined by the size of the field. A black colour indicates compliance, while a grey colour indicates one adjacent level of compliance. The direction of the observer bias is verified by examining the "path of the rectangles" and its deviation from the diagonal line [24][25][26]. A perfect congruence is when the black rectangles line up along the red line and they are black, not grey. Rectangles along the diagonal but partially grey signify a partial agreement, but if they move away from the diagonal, then there is a systematic error in the method that underestimates or overestimates the diagnosis.The Kappa statistics proposed by the B statistics are used to quantify the agreement between two observers, independently classifying the same n units into the same k category [27]. The Kappa Cohen takes values from 0 to 1. The closer to 1, the more consistent the two observer's assessments are, the closer to 0, the more divergent the marks are [28]. The most commonly used guidelines for Kappa are due to Landis and Koch [29]. The interpretation of the Kappa score according to Landis and Koch is as follows: 0-poor, 0 to 0.2 slight, 0.2 to 0.4 fair, 0.4 to 0.6 moderate, 0.6 to 0.8 substantial, and 0.8 to 1 almost perfect. The B-score interpretation was considered poor less than 0.25, fair 0.25 to 0.49, good 0.5 to 0.74, excellent 0.75 to 0.99, and perfect 1 [26].

Results
In our study, 74 females and 116 males were included with an average age of 41.3 (SD 15.3). The main indication for arthroscopic surgery were meniscal lesions (n = 88), followed by ACL tears (n = 75). Isolated chondral lesions without any concomitant intraarticular lesions were found in 27 patients. The mean time between the onset of symptoms to the MRI examination was 31 months (median 5.2). On average, patients waited nearly 4 months (3.9) for arthroscopic surgery after obtaining the MRI results. During arthroscopic surgery, chondral lesions were diagnosed in 139 knees, out of which 112 had concomitant injuries such as meniscal lesions or ACL tears. Grade II lesions were the most common in our study. Femoral condyles were affected more commonly by chondral lesions rather than by tibial condyles. Moreover, chondral lesions were most frequently found in the medial compartment. Overall, 453 chondral defects were encountered during arthroscopy. The lesions showed predilection for MFC (n = 118), followed by MTC (n = 91) and PFJ (n = 90). LFC and LTC were affected evenly and 77 lesions were found in each location. Chondral defects were multifocal and in over 90% (n = 106) of patients, multiple anatomical locations were involved. Only 7.2% (n = 33) of chondral lesions affected one location. The majority of isolated chondral lesions were located in MFC 54.5% (n = 18). Moreover, grade VI and III lesions were most prevalent on MFC, where nearly a half of grade IV defects was found 46.7% (n = 43), and one-third of grade III lesions 33% (n = 36). Grade IV chondral defects in the lateral compartment accounted only for 11% (n = 10) of all grade IV lesions. The results of the arthroscopic evaluation are shown in Table 1. Grade II lesions were encountered most frequently, followed by grade I lesions. Even though grade IV lesions were the rarest, they accounted for one-fifth of all the lesions. An adequate description of cartilage status on MRI was found only in 59 patients in all anatomical locations during arthroscopy. In 119 patients, at least one anatomical location was underestimated regarding the chondral lesion grade, and in 48 patients, at least one location was overestimated. In 36 patients, at least one anatomical location was overestimated and at least one was underestimated in regard to lesion severity. The MRI showed low sensitivity ranging from 24% to 42% depending on the anatomical location. The highest sensitivity was found in MFC and PFJ. The mean specificity of MRI in detecting chondral lesions ranged from 82% to 87.4%. Moreover, the specificity was highest in MFC and PFJ. A healthy cartilage was properly evaluated in over 60% regardless of the anatomical location with sensitivity over 90% and specificity of 39%. The sensitivity of MRI was increasing proportionally with an increasing grade of the cartilage lesion. Grade I lesions showed sensitivity only of 4.8%, and grade IV of 39%. The specificity of MRI in detecting a healthy cartilage was estimated as only 39%. If the cartilage lesion was observed, then the specificity of the MRI ranged from 96% to 97%. A low sensitivity was found for grade I lesions not exceeding 10% in any anatomical location. The sensitivity of 0% was found for grade I MFC lesions and for grade III LTC lesions. Overall, the MRI capacity for detecting lesions was evaluated with ROC curves. The MRI showed the greatest capacity for detecting chondral lesions in PFJ with an AUC of 0.737, followed by MFC. A lowest capacity was observed in LTC with AUC reaching only 0.578. A summary of the sensitivity, specificity, ROC evaluation, Bangdiwala's observer agreement charts, and Kappa values are shown in Tables 2-4 and Figures 5 and 7.

Discussion
The key finding of this study is the underestimation of cartilage lesion by MRI. Cartilage lesions are one of the most common findings during knee arthroscopy [1]. Moreover, it has a poor healing capacity and often requires surgical intervention. Therefore, an adequate preoperative assessment of the cartilage status is an important factor for surgical planning. Multiple factors can affect the cartilage, such as overuse, trauma, post-surgical gait changes, or rheumatoid diseases, joint inflammation, or malalignment [30][31][32][33]. In our study, cartilage lesions were found in the vast majority of patients who underwent knee arthroscopy, and if apparent, then in the majority of cases, the lesions were multifocal. Femoral condyles and the medial knee compartment were found to be most commonly affected. Unfortunately, there are no specific clinical tests for the evaluation of articular cartilage, and distinguishing between chondral or meniscal lesions is impossible during physical examination [34]. For this purpose, MRI becomes the only available diagnostic modality for a routine cartilage examination [35]. In our study, we have chosen arthroscopic visualization as a gold standard in grading cartilage lesions. The overall good inter-observer reliability [36] can be lowered by the lack of surgeon experience [37]. Therefore, for this study, only experienced surgeons that specialized in sports were selected.
Literature data show gross discrepancy regarding the sensitivity of MRI in the cartilage evaluation ranging from 45% to 94% [38]. However, most of the studies were performed on small patient groups. In our study, we have found 453 chondral lesions affecting different anatomical locations in 190 patients. The overall diagnostic accuracy in detecting chondral lesions was rather weak, with Kappa values lower than 0.6. Only healthy cartilages and grade IV lesions were evaluated with acceptable accuracy. Similar findings are reported by Figueroe et al. [4], showing that diagnostic accuracy increases with the chondral lesion severity (in a prospective study on 190 knees). This phenomenon is also found in 3 T MRI examinations [16], where sensitivity for grade I lesions reached only 8.8%. The diagnostic accuracy was also dependable on the anatomic location, with the highest accuracy in the patellofemoral joint and the medial compartment of the knee. New MRI-based compositional and functional techniques are being proposed in the literature. Techniques such as DWI have a potential to distinguish between the healthy and diseased cartilage and, moreover, have the potential to distinguish the healthy cartilage from a repaired one [8]. Methods of functional cartilage evaluation are also proposed in the literature, including serial T1 and T2 mapping under loading [39]. Techniques such as T1rho have the ability to assess the proteoglycan concentration in the cartilage [21,40]. These techniques seem to be the most promising due to providing insight into the histologic composition of the cartilage. In addition, they require no administration of contrast agents and present an excellent range of interobserver reliability and capacity to detect even mild cartilage changes [41]. These new imaging techniques may be of wide use in the future and may perform better in grading and evaluating cartilage defects than the standard morphologic MRI. However, at the moment, the availability of such modalities is limited and papers concerning actual diagnostic performance in vivo are limited, as well. These techniques require further investigation.
The medial compartment of the knee was most commonly affected with grade III and IV lesions. As shown in our study as well as previous studies, the diagnostic accuracy increases with the lesion grade, which can explain the higher overall accuracy in the medial compartment of the knee. The results found in PFJ can be explained by a great visualization of the cartilage on transverse views, where one can appreciate the lesions with little effort. In our study, Kappa values for any anatomical location did not surpass 0.6, which strongly shows a discrepancy between MRI and arthroscopy findings. These findings are clearly represented on Bangdiwala's observer agreement charts. The severity of the cartilage lesion was grossly underestimated, which can significantly affect the planned surgical procedure during arthroscopy.
Our study shows that even though MRI is the best available imaging modality for cartilage evaluation, it has significant weaknesses in regard to its specificity and sensitivity.
It has a limited diagnostic accuracy for grade I-III chondral lesions. MRI showed the lowest sensitivity on the lateral tibial condyle, regardless of the lesion grade. This finding can be attributed to a small number of grade III and IV lesions, which are most apparent on the MRI. It is known that chondral lesions progress over time [42,43]. Therefore, in our study, we have taken into consideration the time-lapse between the examination and surgery. However, no significant correlation between the time-lapse and the diagnostic accuracy was found for any anatomical location or grade of chondral lesions (p = 0.783). The ROC analysis was utilized to evaluate the diagnostic performance of MRI for the detection of any lesion, regardless of the lesion grade in specific anatomical locations. Only MFC and PFJ showed an acceptable diagnostic accuracy, with AUC exceeding 0.7. This shows that lesions in the lateral compartment of the knee can be omitted during an MRI examination. A systemic review by Quatman et al. [44] showed similar results. In our study, we have used only 1.5 T scanners for the examination. However, a study by Kijowski et al. [45] suggests that 3 T scanners show a similar diagnostic performance as the standard MRI protocol. Therefore, some authors seek indirect methods of knee cartilage status evaluation. Carotti et al. [46] proposed the identification of early osteoarthrosis which is not based on the cartilage status, but on visible bone marrow lesions and infrapatellar fat pad synovitis. Such an approach can be explained by the so-called satisfaction of the search [47] when the detection of abnormality can reduce the detectability of other intraarticular lesions.
The bias of this study consists of the lack of access to MRI referrals. This could influence the low sensitivity and the accuracy of MRI in our study when no attention in the referral was put on cartilage evaluation. As shown by Solivetti et al. [48], approximately 40% of MRI referrals may be totally inappropriate or uncertain. Moreover, some authors suggest that an MRI examination prior to the planned arthroscopic surgery does not change the treatment strategy [19]. The findings are supported by other researchers [49], however, if the MRI is ordered by an orthopaedic surgeon, it has a greater impact on the planned treatment [50] compared to a primary care physician. The accurate referral may incline the usage of cartilage dedicated sequences, which improves the overall diagnostic accuracy [51]. The experience and training of a radiologist performing the cartilage evaluation is also a factor which influences the diagnostic performance of the MRI [52]. In conclusion, the MRI is the best available modality for cartilage evaluation. However, it should be used with caution, and an underestimation of the cartilage defect severity should be taken into consideration when planning a surgical procedure. Chondral lesions identification is acceptable only in MFC and PFJ. Apart from healthy cartilage and grade IV lesions, the MRI results should be suspected of severity underestimation.

Limitations
The bias of this study consists of MRI referrals which were made by multiple orthopaedic surgeons. In addition, we had no access to the MRI referrals, which could have influenced the MRI acquisition, sequences, and finally interpretation of examination. Moreover, we had no information on the radiologists' experience in knee MRI, which could impact the overall diagnostic accuracy, if the experience was low. However, this setting represents the most common clinical presentation that an orthopaedic surgeon may encounter. Furthermore, a lack of blinding of the study could introduce a risk of bias. Even though the lack of information on the cartilage status is usually considered a healthy cartilage, in this study it could influence the overall results.

Conclusions
The MRI underestimates the extent of cartilage injury. In addition, the evaluation of cartilage defects based on the MRI should be taken with caution by orthopaedic surgeons planning a surgery. Surgical planning which is based on an MRI should take cartilage lesions under consideration, even if no cartilage lesions have been reported on the MRI scans.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to patients privacy.

Conflicts of Interest:
The authors declare no conflict of interest.