Next Article in Journal
Role of Endoscopic Ultrasonography-Guided Fine Needle Aspiration/Biopsy in the Diagnosis of Autoimmune Pancreatitis
Next Article in Special Issue
Complications of Endoscopic Ultrasound-Guided Fine Needle Aspiration: A Narrative Review
Previous Article in Journal
Impact of F-18 Fluorodeoxyglucose PET/CT and PET/MRI on Initial Staging and Changes in Management of Pancreatic Ductal Adenocarcinoma: A Systemic Review and Meta-Analysis
Previous Article in Special Issue
Morphometric Assessment of Confocal Laser Endomicroscopy for Pancreatic Ductal Adenocarcinoma, an Ex-Vivo Pilot Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 10
Issue 11
10.3390/diagnostics10110953
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Interobserver Reliability of Endoscopic Ultrasonography: Literature Review

by
Akira Yamamiya
,
Atsushi Irisawa
*,
Ken Kashima
,
Yasuhito Kunogi
,
Kazunori Nagashima
,
Takahito Minaguchi
,
Naoya Izawa
,
Akane Yamabe
,
Koki Hoshi
,
Keiichi Tominaga
,
Makoto Iijima
and
Kenichi Goda
Department of Gastroenterology, Dokkyo Medical University School of Medicine, 880 Kitakobayashi Mibu, Tochigi 321-0293, Japan
*
Author to whom correspondence should be addressed.
Diagnostics 2020, 10(11), 953; https://doi.org/10.3390/diagnostics10110953
Submission received: 23 October 2020 / Revised: 10 November 2020 / Accepted: 12 November 2020 / Published: 15 November 2020
(This article belongs to the Special Issue The Advanced Role of Diagnostic Endoscopic Ultrasonography)
Versions Notes

Abstract

Endoscopic ultrasonography (EUS) has been applied to the diagnosis of various digestive disorders. Although it has been widely accepted and its diagnostic value is high, the dependence of EUS diagnosis on image interpretation done by the endosonographer has persisted as an important difficulty. Consequently, high interobserver reliability (IOR) in EUS diagnosis is important to demonstrate the reliability of EUS diagnosis. We reviewed the literature on the IOR of EUS diagnosis for various diseases such as chronic pancreatitis, pancreatic solid/cystic mass, lymphadenopathy, and gastrointestinal and subepithelial lesions. The IOR of EUS diagnosis differs depending on the disease; moreover, EUS findings with high IOR and those with IOR that was not necessarily high were used as diagnostic criteria. Therefore, to further increase the value of EUS diagnosis, EUS diagnostic criteria with high diagnostic characteristics based on EUS findings with high IOR must be established.

1. Introduction

Endoscopic ultrasonography (EUS) was first reported in 1980 [1,2]. Since then, EUS has been applied to the diagnosis and treatment of pancreatic diseases [3,4,5,6,7]. Currently, EUS is used to diagnose diseases of various organs, including the pancreas. Generally, EUS is regarded as an effective modality because of its excellent image resolution, which allows the detection of small lesions and accurate assessment of staging in addition to the possibility of tissue sampling. Consequently, numerous reports of the diagnostic ability of EUS exist. Despite its benefits, EUS is not always perfect; limitations that hinder its accuracy are reportedly attributed to the following: (i) EUS imaging and diagnosis depend on the skill of the physicians, i.e., inter-operator variation; (ii) endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) is often necessary for differential diagnosis because distinguishing between neoplastic and non-neoplastic tissue is difficult with EUS imaging; and (iii) conditions such as biliary/pancreatic duct stenting and chronic pancreatitis can interfere with the detection and staging of small lesions [8,9,10,11,12,13]. Particularly, inter-operator variation presents major difficulties. Consequently, high inter-observer reliability (IOR) is important to improve the reliability of imaging by EUS.
Currently, numerous diagnostic criteria based on EUS for various diseases are available. In these diagnostic criteria, a high IOR of EUS diagnostic findings is crucially important to support its diagnostic value. Some studies of IOR of EUS diagnosis have been conducted for various gastrointestinal diseases. This review presents a summary of the literature regarding the IOR of EUS diagnosis and explains some of the “wonders and problems” of current EUS diagnosis. For this study, IOR is expressed as kappa (K) values, which are defined as follows: <0, no agreement; 0–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, high; and 0.81–1.00, almost perfect [14]. The literature from 1966 through 2019 was searched in PubMed using the keywords EUS and IOR.

2. Chronic Pancreatitis

EUS has been employed for the diagnosis of patients with chronic pancreatitis (CP) or suspected CP for over 20 years. Results of an earlier study suggested that CP was detectable by EUS even when findings of noninvasive imaging studies, such as computed tomography and magnetic resonance imaging, were negative [15]. Lees et al. demonstrated the usefulness of EUS morphology and histology after pancreatic resection in post-pancreatic resection specimens [16]. The results showed that CP was diagnosed histologically in six of the seven patients with CP findings on EUS. Other reports have described good correlation between EUS findings of CP and histological findings of CP, except in elderly patients [17,18]. Albashir et al. reported that the diagnostic performance of EUS based on histological findings was 84% for sensitivity and 100% for specificity [19].
Because of the well-known utility of EUS in the diagnosis of CP, this has been based on standard scoring of CP, defined by the International Working Group as standard criteria (SC) since these guidelines were published in 1998 [20] (Supplementary Table S1). The nine findings adopted were the following: hyperechoic foci, hyperechoic strands, lobularity, calcifications/stones, cysts, main pancreatic duct dilation, main pancreatic duct irregularity, hyperechoic main pancreatic duct (MPD) walls, and visible side branches. Subsequently, new diagnostic criteria for CP, i.e., the Rosemont Criteria (RC), were adopted in 2009 [21] (Supplementary Table S2).
The IOR of each finding has been reported in numerous studies (Table 1). The first report on IOR was by Wallence et al. in 2001 [22]. The K value of duct dilatation was 0.61 (high). That of lobularity was 0.51 (moderate), whereas that of the other features was <0.4 (slight and fair). Stevens et al. reported a similar result. Only the IOR of duct dilation was high (K = 0.61) [23]. Del Pozo et al. reported that the K values of lobularity, cysts, and MPD dilation were large (0.75, 0.66, and 075, respectively), whereas the IOR was only fair in lobularity with honeycombing (K = 0.31) [24]. Gardner et al. reported that the IOR is almost perfect in cysts (K = 1.00) and high in strands (K = 0.62) [25]; the other findings had low concordance rates: moderate in duct dilation, hyperechoic MPD wall, and lobularity (K = 0.53, 0.53, and 0.44, respectively) and fair in hyperechoic foci (K = 0.39). Lieb et al. reported that the IOR of stones and duct dilation were high (K = 0.78 and 0.77, respectively), whereas those of hyperechoic foci, strands, and side branch dilation were slight (K = 0.19, 0.07, and 0.11, respectively) [26]. A T study by Topazian et al. found low IOR in all cases: the IOR of cysts, which was the highest value (K = 0.48), was fair; those of the other findings were slight [27].
Final diagnosis of CP by EUS based on SC and RC has also been reported. The assessment of IOR of SC has been studied previously. It was found to be moderate at best. Wallence et al., Stevens et al., Del Pozo et al., and Kalmin et al. reported moderate IORs of the final diagnoses (K = 0.45, 0.54, 0.53, 0.50, respectively) [22,23,24,28]. The IOR of the final diagnosis in the study reported by Lieb et al. was fair (K = 0.39) [26].
Furthermore, several studies evaluated the IOR of the final diagnosis for both SC and RC. Del Pozo et al. reported that the IOR of the final diagnosis was moderate for both criteria (K = 0.53 for SC and K = 0.46 for RC) [24]. The study reported by Stevens et al. revealed the IOR of the final diagnosis as moderate for SC (K = 0.54) and high for RC (K = 0.65). However, the difference was not statistically significant (p = 0.12) [23]. Kalmin et al. reported that the IOR of the final diagnosis was moderate for SC (K = 0.50) and fair for RC (K = 0.27) [28]. These results suggest that the IOR did not seem to improve with the RC compared with the SC.
The examiners participating in most of the studies were experts. However, some studies have investigated differences in IOR according to years of experience with EUS. Wallence et al. reported the IOR of the final diagnosis as moderate for fellows and experts (K = 0.42 and 0.46, respectively) [22]. Stevens et al. reported no significant difference in IOR based on years of experience, irrespective of the criteria used (i.e., SC or RC) [16].
Although the IOR of the respective findings for CP varied, several reports have described that the IOR of the final diagnosis produced using both SC and RC is moderate. Moreover, the findings suggest that the IOR might not depend on the years of experience with EUS.

3. Solid Pancreatic Lesion

Imaging of a solid pancreatic lesion (SPL) is fundamentally important for treatment. SPLs include pancreatic adenocarcinoma, inflammatory mass in the context of CP, pancreatic neuroendocrine tumor, autoimmune pancreatitis, and other tumors. Among them, pancreatic adenocarcinoma has a particularly poor prognosis. For that reason, early detection is desirable. EUS plays a major role as a diagnostic modality: even small lesions of 5 mm can be visualized reliably by EUS. Moreover, contrast-enhanced harmonic endoscopic ultrasonography (CH-EUS) is an extremely useful test in the differentiation of SPL [29,30]. CH-EUS can assess tissue perfusion in real time without Doppler-related artifacts [30,31,32]. A recent study demonstrated that the microvascular pattern of SPL based on CH-EUS correlates closely with histological features [32,33]. Gong et al. reported that the sensitivity of CH-EUS for the differential diagnosis of pancreatic adenocarcinoma is 94%; the specificity was 89% [34].
Based on EUS, pancreatic adenocarcinomas are unevenly margined and well-defined. Small tumors present with a homogeneous hypoechoic appearance. However, as the tumor enlarges, a heterogeneous hyperechoic region develops in the center. Pancreatic duct dilation caudal to the mass and dilated surrounding branches accompanied by storage cysts are important indirect findings. CH-EUS often shows hypo-enhancement in the arteriovenous phase because of a desmoplastic reaction and an internal heterogeneous contrast pattern. Moreover, EUS images of neuroendocrine tumors (NETs) show a hypoechoic appearance with smooth edges, well-defined borders, and uniform interiors. In CH-EUS, it is of hyper-vascular-type. Inflammatory masses in the context of CP present an overall hypoechoic appearance. The boundary of the pancreatic parenchyma is somewhat obscured. They also often present with duct-penetrating signs.
Several studies have examined IOR of the diagnosis of SPL using CH-EUS (Table 2). Some studies have examined differences in IOR by years of experience with EUS. Kitano et al. reported that the combination of CH-EUS and EUS-FNA increased the sensitivity for the diagnosis of pancreatic adenocarcinoma to 100% [35]. After they evaluated the IOR of CH-EUS in 277 SPLs conducted by two experienced endo-sonographers, they reported a K value of 0.94 (almost perfect). Fusaroli et al. reported that the IOR of contrast uptake was moderate for all endo-sonographers (K = 0.56). Similar results were reported for experienced endo-sonographers (K = 0.56), and for non-experienced endo-sonographers (K = 0.55) [36]. Gincul et al. evaluated the IOR of CH-EUS by experienced endo-sonographers and non-experienced endo-sonographers. The overall IOR (K = 0.66) and the IOR of experienced (K = 0.65) and non-experienced (K = 0.76) endo-sonographers were high [37]. In this study, CH-EUS and EUS-FNA had high accuracy, with sensitivity of 95% and 96%, of 95% and 93% (p < 0.05). Bunganič et al. reported that the IOR of pancreatic adenocarcinoma diagnosis by EUS and CH-EUS was moderate (K = 0.45 and 0.60, respectively) [38]. The sensitivity and specificity of EUS and CH-EUS for pancreatic adenocarcinoma diagnosis were, respectively, 83.1% and 62.5% and 94.5% and 61.7%.
Palazzo et al. examined the IORs of the diagnoses of 81 cases of pancreatic NET (P-NET) [39]. The sensitivity and specificity of CH-EUS for the P-NET diagnosis were 96% and 82%, respectively. The IOR was high (K = 0.66) for all endo-sonographers, almost perfect (K = 0.83) for experienced endo-sonographers, and almost perfect (K = 0.82) for junior endo-sonographers.
Soares et al. evaluated the diagnostic ability of EUS-elastography (EUS-E) for SPLs [40]. Eleven endo-sonographers were divided into four groups: group A (long experience in EUS and EUS-E), group B (short experience in EUS and EUS-E), group C (long experience in EUS but no experience in EUS-E), and group D (no experience with EUS or EUS-E). The overall IOR was moderate (K = 0.42). The IOR of group A (K = 0.80) was significantly higher than that of groups B (K = 0.54), C (K = 0.54), and D (K = 0.28).
The IOR of SPL diagnosis by EUS was high. However, it might vary depending on the years of experience of endo-sonographers.

4. Pancreatic Cystic Lesion

Pancreatic cystic lesions include intraductal papillary mucinous neoplasm (IPMN), mucinous cystic neoplasm, and serous cystic neoplasm; all have a malignant potential [41,42,43]. EUS can provide morphological details of pancreatic cysts. However, a benign or malignant cyst is sometimes difficult to diagnose [44,45]. Earlier studies showed the diagnostic accuracy of EUS for pancreatic cystic lesions as 82–96% [46,47,48].
The first report of the IOR of pancreatic cystic lesion diagnosis was presented by Ahmad et al. in 2003 [49] (Table 3). Thirty-one cases with histologically classified pancreatic cystic lesions were reviewed independently by eight expert endo-sonographers. The IOR of diagnosing a benign or malignant cyst was fair (K = 0.24). Moreover, the IORs of the respective diagnostic finding were as follows: presence or absence of solid component, K = 0.43 (moderate); presence or absence of abnormal pancreatic duct, K = 0.29 (fair); debris, K = 0.21 (fair); septations, K = 0.30 (fair); presence or absence of margins, K = 0.01 (slight); and presence or absence of abnormal pancreatic parenchyma, K = 0.01 (slight). Fusaroli examined the IOR of pancreatic cystic lesion diagnosis by CH-EUS and found the IOR of contrast uptake to be moderate (K = 0.58) [36]. Gonzalez et al. examined the IOR of branch duct IPMN diagnosis by EUS and magnetic resonance cholangiopancreatography [50]. The IOR was moderate for PD = 5–9 mm (K = 0.45) and for abrupt change in PD (K = 0.529), fair for wall thickening (K = 0.259), and slight for non-enhanced mural nodule (K < 0) and lymphadenopathy (K < 0). In high-risk stigmata, the IOR was reported as slight for enhanced solid component (K = 0.12) and PD > 10 mm (K < 0). Jong et al. examined the IOR in three groups based on EUS experience [51]. The IOR of nodules was high in the expert group (K = 0.65) and fair in semi-expert and novice groups (K = 0.32 and 0.37, respectively). The IOR of the presence of solid components was moderate in the expert group (K = 0.52) and significantly higher than that in the other two groups (semi-experts, K = 0.09; and novices, K = 0.03).
Kim et al. reported the IOR of cyst communication with the duct as high (K = 0.69) and that of differentiating malignant from benign cysts as almost perfect (K = 0.92). The IOR of pancreatic cystic lesions is expected to be elevated with CH-EUS. Nonetheless, the challenge remains of eliminating the difference in IOR by years of experience with EUS.

5. Other Diseases

5.1. Lymphadenopathy

Differentiating benign from malignant lymphadenopathy is extremely important: EUS is appropriate for such evaluation [52,53,54]. Catalano et al. established the following EUS features as predictive of malignancy (in increasing order of importance): hypoechoic structures, sharply demarcated borders, rounded contours, and size >10 mm [52]. EUS-FNA is an efficient method for diagnosing lymphadenopathy, with recent reports describing sensitivity and specificity of 74–98% and 100%, respectively [55,56,57]. Although EUS-FNA has a high diagnostic value in lymphadenopathy, it is important to use only EUS to predict benign or malignant lesions.
Takasaki et al. evaluated the IOR of EUS features and proposed EUS diagnostic norms for lymphadenopathy based on the IOR [58]. The IOR was moderate for shape (K = 0.44) and fair for echogenicity, homogeneity, border, and hilum of the lymph node (K = 0.33, 0.34, 0.22, and 0.22, respectively). Moreover, they suggested the establishment of new EUS diagnostic criteria using shape, long axis (>20 mm), and short axis (>10 mm). Melo et al. studied the IOR of EUS diagnostic imaging for lymphadenopathy [59] and reported fair IOR for shape (K = 0.35) and moderate IOR for echogenicity and borders (K = 0.46 and 0.43, respectively). The overall IOR was high (K = 0.65).
Xia et al. evaluated the IOR of CH-EUS features for intra-abdominal lesions of undetermined origin. Results showed that about 80% of the eligible cases were in lymphadenopathy and that the IOR was almost perfect (K = 0.95). Furthermore, they reported that the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy with which CH-EUS differentiated malignant from benign lesions were 96.3, 100, 100, 94.1 and 97.6% [60].
Several studies of lymphadenopathy using EUS-E have been conducted. Based on the general EUS-E patterns, lymph nodes were classified as benign (e.g., green color) or malignant (e.g., blue color) and scored in the study of Giovannini et al. [61]. Janssen et al. reported that the accuracy range among the examiners was 82–88% for benign lymph nodes and 85–87% for malignant ones. The IOR was almost perfect (K = 0.84) [62]. Based on results of a study by Larsen et al., the IOR of distinguishing benign or malignant lymphadenopathy was reported as moderate (K = 0.59) [63] (Table 4).
The IOR of lymphadenopathy is expected to be elevated through the use of CH-EUS and EUS-E. Nevertheless, further development of diagnostic criteria to predict benign or malignant lesions is warranted.

5.2. Gallbladder Wall Thickening

Gallbladder (GB) wall thickening might be caused by a broad spectrum of pathologies, including GB carcinoma, chronic cholecystitis, and adeno-myomatosis. Differentiation between GB carcinoma, adeno-myomatosis, and GB inflammatory diseases presenting as wall thickening is an important clinical issue because misinterpretation of GB wall thickening might result in unnecessarily extended surgery in patients with GB inflammatory diseases or delayed treatment in patients with GB carcinoma. Because it allows observation of the laminar structure of the GB wall, EUS is useful for the diagnosis of GB thickening [64,65]. Recently, some studies showed that CH-EUS might be useful for the diagnosis of GB disease [66,67]. Imazu et al. evaluated the IOR of diagnosing GB wall thickening using EUS and CH-EUS [68]. Univariate analysis results indicate that GB wall thickening of >12 mm and disruption of the GB wall layer structure on EUS and inhomogeneous distribution pattern of contrast agent on CH-EUS are significantly associated with malignant GB wall thickening with a high odds ratio. The addition of contrast enhancement to conventional EUS was extremely useful for differential diagnosis of GB wall thickening. Overall sensitivity and specificity for diagnosing malignant GB wall thickening of EUS and CH-EUS were 83.3 vs. 89.6, 65 vs. 98%, respectively. The IOR of the differential diagnosis of malignant or nonmalignant GB wall thickening was moderate with EUS (K = 0.51) and high with CH-EUS (K = 0.77) (Table 4).
The IOR of GB wall thickening diagnosis by CH-EUS is high; nevertheless, examination of more cases is necessary to further evaluate the usefulness of the method.

5.3. Subepithelial Lesion

EUS is also useful for diagnosing subepithelial lesions (SELs). The adjunctive application of CH-EUS can augment the diagnostic work-up of SEL. Recently, sensitivity, specificity, and accuracy rates of 100%, 63%, and 83%, respectively, were reported for the prediction of malignancy of gastrointestinal SEL by CH-EUS [69]. Gress et al. demonstrated that the IOR of SEL was almost perfect for cystic lesions (K = 0.80) and extrinsic compressions (K = 0.94), high for lipoma (K = 0.65), moderate for leiomyoma and vascular lesions (K = 0.53 and 0.54, respectively), and fair for other submucosal lesions (K = 0.34). Overall agreement among observers was strong (K = 0.63) [70]. Furthermore, significant association was found between the total years of experience with EUS and the number of correct answers (p = 0.01). Moreover, Fusaroli et al. [36] determined the IOR of the parameters (i.e., uptake, pattern, washout) examined in SEL of three types by CH-EUS. The IOR was large for uptake (K = 0.638), slight for pattern (K = 0.183), and fair for washout (K = 0.394) (Table 4). The IOR of SEL is high and independent of years of experience, suggesting that EUS is useful for diagnosing SEL.

5.4. Intrapancreatic Accessory Spleen

Accessory spleen (AS) might be encountered as an intrapancreatic lesion on EUS. It can present as a diagnostic dilemma because it might resemble other pancreatic pathologies, such as pancreatic adenocarcinoma and neuroendocrine tumors on imaging. Intrapancreatic AS (IPAS) is usually an incidental finding that requires no treatment [71,72]. Nevertheless, IPAS must be distinguished from other pancreatic neoplasms to avoid unnecessary interventions. Rodriguez et al. showed that an AS tends to be an isoechoic or hypoechoic mass with well-defined, smooth borders on EUS [73]. Kim et al. reported that the IOR of determining whether or not the pancreatic lesion is IPAS is fair (K = 0.37) [74] (Table 4). They also reported that the sensitivity and specificity for IPAS were greater than 70%.
Few studies have examined the IOR of AS; nonetheless, it has been suggested that combining EUS with other modalities might increase the diagnostic capabilities for AS.

5.5. Mucosa-Associated Lymphoid Tissue (MALT) Lymphoma

Several studies have demonstrated the reliability of EUS for staging MALT lymphoma before and after therapy [75,76,77,78]. Particularly, as reported earlier, EUS has sensitivity of 89%, specificity of 97%, and overall accuracy of 95% for depth of invasion [79]. Fusaroli et al. reported the IOR of EUS staging of gastric MALT lymphoma before and after treatment [80]. The overall IOR of T-stage was fair before and after treatment (K = 0.38 and 0.37, respectively). That of N-stage was large before treatment and fair after treatment (K = 0.63 and 0.34, respectively) (Table 4). Although the IOR of EUS staging of gastric MALT lymphoma varies, EUS for gastric MALT lymphoma might determine the treatment effects that were not observable using macroscopic morphology.

5.6. Gastrointestinal Malignancy

EUS is used for staging of gastrointestinal malignancies. Yanai et al. reported the IOR of staging of submucosal invasion of superficial esophageal cancer (SEC) [81]. The IOR was fair (K = 0.46). It was suggested that EUS is useful for the evaluation of lymph node metastasis and might be useful in predicting the prognosis of SEC. Burtin et al. evaluated the IOR of staging of rectal cancer [82]. The IOR of uT1 tumors was fair (K = 0.40), uT2 tumors was slight (K = 0.20), and uT3 tumors was moderate (K = 0.58). In rectal cancer, EUS was also excellent for the evaluation of lymph node metastasis (K = 0.65). Roubein et al. was evaluated similarly for rectal cancer but stated only that the evaluation of lymph node metastases was a high IOR (K = 0.42) [83] (Table 4).
Reportedly, the IOR for staging of gastrointestinal malignancies cancers varies widely among observers. Further study of this point is required [84].

6. Future Perspectives

In recent years, as the development of artificial intelligence (AI) technology has progressed, it has been applied to the field of diagnosis by EUS [85,86,87]. Kuwahara et al. reported that the malignancy of IPMN can be diagnosed using AI with accuracy of 94%, which is higher than those obtained using conventional EUS features (40–60%) and endo-sonographer’s diagnosis (56%) [88]. Advances in AI are expected to be important for improving IOR-independent imaging capabilities in the field of EUS imaging.

7. Conclusions

EUS imaging has become a widely used approach for diagnosis of numerous diseases because of its high resolution and noninvasive nature. In addition, several EUS diagnostic criteria are extremely useful for diagnosing various diseases. However, the IOR of EUS diagnosis differs depending on the disease. Not only EUS findings with high IOR but also those with not necessarily high IOR are used as diagnostic criteria. Therefore, to further increase the value of EUS diagnosis, EUS diagnostic criteria with high diagnostic characteristics, based on EUS findings with high IOR, must be established.

Supplementary Materials

The following are available online at https://www.mdpi.com/2075-4418/10/11/953/s1: Table S1: Endoscopic ultrasound criteria for the diagnosis of chronic pancreatitis, Table S2: Classification of patients based on endoscopic criteria.

Author Contributions

A.Y. (Akira Yamamiya) and A.I. designed this manuscript, collected and analyzed the data, and drafted the manuscript; A.I. checked the manuscript and approved the final version; K.K., Y.K., K.N., T.M., N.I., A.Y. (Akane Yamabe), K.H., and K.T. collected the literature and created the tables; M.I. and K.G. supervised this manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest

References

  1. DiMango, E.P.; Buxton, J.L.; Regan, P.T.; Hattery, R.R.; Wilson, D.A.; Suarez, J.R.; Green, P.S. Ultrasonic endoscope. Lancet 1980, 315, 629–631. [Google Scholar] [CrossRef]
  2. Strohm, W.D.; Phillip, J.; Hagenmüller, F.; Classen, M. Ultrasonic tomography by means of an ultrasonic fiberendoscope. Endoscopy 1980, 12, 214–244. [Google Scholar] [CrossRef]
  3. Hisanaga, K.; Hisanaga, A.; Nagata, K.; Ichie, Y. High speed rotating scanner for transgastric sonography. AJR Am. J. Roentgenol. 1980, 135, 627–639. [Google Scholar] [CrossRef]
  4. DiMagno, E.P.; Regan, P.T.; Clain, J.E.; James, E.M.; Buxton, J.L. Human endoscopic ultrasonography. Gastroenterology 1982, 83, 824–829. [Google Scholar] [CrossRef]
  5. Khalid, A.; Nodit, L.; Zahid, M.; Bauer, K.; Brody, D.; Finkelstein, S.D.; McGrath, K.M. Endoscopic ultrasound fine needle aspirate DNA analysis to differentiate malignant and benign pancreatic masses. Am. J. Gastroenterol. 2006, 101, 2493–2500. [Google Scholar] [CrossRef]
  6. Gardner, T.B.; Levy, M.J. EUS diagnosis of chronic pancreatitis. Gastrointest. Endosc 2010, 71, 1280–1289. [Google Scholar] [CrossRef]
  7. Levy, M.J.; Wiersema, M.J. EUS-guided celiac plexus neurolysis and celiac plexus block. Gastrointest. Endosc. 2003, 57, 923–930. [Google Scholar] [CrossRef]
  8. Fusaroli, P.; Manta, R.; Fedeli, P.; Maltoni, S.; Grillo, A.; Giovannini, E.; Bucchi, L.; Caletti, G. The influence of endoscopic biliary stents on the accuracy of endoscopic ultrasound for pancreatic head cancer staging. Endoscopy 2007, 39, 813–817. [Google Scholar] [CrossRef] [PubMed]
  9. Bhutani, M.S.; Gress, F.G.; Giovannini, M.; Erickson, R.A.; Catalano, M.F.; Chak, A.; Deprez, P.H.; Faigel, D.O.; Nguyen, C.C. The No Endosonographic Detection of Tumor (NEST) study: A case series of pancreatic cancers missed on endoscopic ultrasonography. Endoscopy 2004, 36, 385–389. [Google Scholar] [CrossRef] [PubMed]
  10. Fritscher-Ravens, A.; Brand, L.; Knöfel, W.T.; Bobrowski, C.; Topalidis, T.; Thonke, F.; de Werth, A.; Soehendra, N. Comparison of endoscopic ultrasound-guided fine needle aspiration for focal pancreatic lesions in patients with normal parenchyma and chronic pancreatitis. Am. J. Gastroenterol. 2002, 97, 2768–2775. [Google Scholar] [CrossRef] [PubMed]
  11. Varadarajulu, S.; Tamhane, A.; Eloubeidi, M.A. Yield of EUS-guided FNA of pancreatic masses in the presence or the absence of chronic pancreatitis. Gastrointest. Endosc. 2005, 62, 728–736. [Google Scholar] [CrossRef] [PubMed]
  12. Krishna, N.B.; Mehra, M.; Reddy, A.V.; Agarwal, B. EUS/EUS-FNA for suspected pancreatic cancer: Influence of chronic pancreatitis and clinical presentation with or without obstructive jaundice on performance characteristics. Gastrointest. Endosc. 2009, 70, 70–79. [Google Scholar] [CrossRef] [PubMed]
  13. Eloubeidi, M.A.; Varadarajulu, S.; Desai, S.; Wilcox, C.M. Value of repeat endoscopic ultrasound-guided fine needle aspiration for suspected pancreatic cancer. J. Gastroenterol. Hepatol. 2008, 23, 567–570. [Google Scholar] [CrossRef]
  14. Landis, J.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef] [PubMed]
  15. Chong, A.K.; Hawes, R.H.; Hoffman, B.J.; Adams, D.B.; Lewin, D.N.; Romagnuolo, J. Diagnostic performance of EUS for chronic pancreatitis: A comparison with histopathology. Gastrointest. Endosc. 2007, 65, 808–814. [Google Scholar] [CrossRef] [PubMed]
  16. Lees, W.R. Endoscopic ultrasonography of chronic pancreatitis and pancreatic pseudocysts. Scand. J. Gastroenterol. 1986, 123, 130–134. [Google Scholar] [CrossRef]
  17. Bhutani, M.S.; Arantes, V.N.; Verma, D.; Moezzi, J.; Suryaprasad, S.; Kapadia, A.S.; Gopalswamy, N. Histopathologic correlation of endoscopic ultrasound findings of chronic pancreatitis in human autopsies. Pancreas 2009, 38, 820–824. [Google Scholar] [CrossRef]
  18. Varadarajulu, S.; Eltoum, I.; Tamhane, A.; Eloubeidi, M.A. Histopathologic correlates of noncalcific chronic pancreatitis by EUS: A prospective tissue characterization study. Gastrointest. Endosc. 2007, 66, 501–509. [Google Scholar] [CrossRef]
  19. Albashir, S.; Bronner, M.P.; Parsi, M.A.; Walsh, R.M.; Stevens, T. Endoscopic ultrasound, secretin endoscopic pancreatic function test, and histology: Correlation in chronic pancreatitis. Am. J. Gastroenterol. 2010, 105, 2498–2503. [Google Scholar] [CrossRef]
  20. The International Working Group for Minimal Standard Terminology in Gastrointestinal Endosonography. Minimal standard terminology in gastrointestinal endosonography. Dig. Endosc. 1998, 10, 159–184. [Google Scholar]
  21. Catalano, M.F.; Sahai, A.; Levy, M.; Romagnuolo, J.; Wiersema, M.; Brugge, W.; Freeman, M.; Yamao, K.; Canto, M.; Hernandez, L.V. EUS-based criteria for the diagnosis of chronic pancreatitis: The Rosemont classification. Gastrointestndosc 2009, 69, 1251–1261. [Google Scholar] [CrossRef] [PubMed]
  22. Wallace, M.B.; Hawes, R.H.; Durkalski, V.; Chak, A.; Mallery, S.; Catalano, M.F.; Wiersema, M.J.; Bhutani, M.S.; Ciaccia, D.; Kochman, M.L.; et al. The reliability of EUS for the diagnosis of chronic pancreatitis: Interobserver agreement among experienced endosonographers. Gastrointest. Endosc. 2001, 53, 294–299. [Google Scholar] [CrossRef]
  23. Stevens, T.; Lopez, R.; Adler, D.G.; Al-Haddad, M.A.; Conway, J.; Dewitt, J.M.; Forsmark, C.E.; Kahaleh, M.; Lee, L.S.; Levy, M.J.; et al. Multicenter comparison of the interobserver agreement of standard EUS scoring and Rosemont classification scoring for diagnosis of chronic pancreatitis. Gastrointest. Endosc. 2010, 71, 519–526. [Google Scholar] [CrossRef] [PubMed]
  24. Del Pozo, D.; Poves, E.; Tabernero, S.; Beceiro, I.; Moral, I.; Villafruela, M.; Sanz, C.; Borrego, G. Conventional versus Rosemont endoscopic ultrasound criteria for chronic pancreatitis: Interobserver agreement in same day back-to-back procedures. Pancreatology 2012, 12, 284–287. [Google Scholar] [CrossRef] [PubMed]
  25. Gardner, T.B.; Gordon, S.R. Interobserver agreement for pancreatic endoscopic ultrasonography determined by same day back-to-back examinations. J. Clin. Gastroenterol. 2011, 45, 542–545. [Google Scholar] [CrossRef]
  26. Lieb, J.G.; Palma, D.T.; Garvan, C.W.; Leblanc, J.K.; Romagnuolo, J.; Farrell, J.J.; Savides, T.J.; Eloubeidi, M.A.; Draganov, P.V.; Forsmark, C.E.; et al. Intraobserver agreement among endosonographers for endoscopic ultrasound features of chronic pancreatitis a blinded multicenter study. Pancreas 2011, 40, 177–180. [Google Scholar] [CrossRef]
  27. Topazian, M.; Enders, F.; Kimmey, M.; Brand, R.; Chak, A.; Clain, J.; Cunningham, J.; Eloubeidi, M.; Gerdes, H.; Gress, F.; et al. Interobserver agreement for EUS findings in familial pancreatic-cancer kindreds. Gastrointest. Endosc. 2007, 66, 62–67. [Google Scholar] [CrossRef]
  28. Kalmin, B.; Hoffman, B.; Hawes, R.; Romagnuolo, J. Conventional versus Rosemont endoscopic ultrasound criteria for chronic pancreatitis: Comparing interobserver reliability and intertest agreement. Can. J. Gastroenterol. 2011, 25, 261–264. [Google Scholar] [CrossRef]
  29. Kitano, M.; Kudo, M.; Maekawa, K.; Suetomi, Y.; Sakamoto, H.; Fukuta, N.; Nakaoka, R.; Kawasaki, T. Dynamic imaging of pancreatic diseases by contrast enhanced coded phase inversion harmonic ultrasonography. Gut 2004, 53, 854–859. [Google Scholar] [CrossRef]
  30. Fusaroli, P.; Saftoiu, A.; Mancino, M.G.; Caletti, G.; Eloubeidi, M.A. Techniques of image enhancement in EUS (with videos). Gastrointest. Endosc. 2011, 74, 645–655. [Google Scholar] [CrossRef]
  31. Kersting, S.; Konopke, R.; Kersting, F.; Volk, A.; Distler, M.; Bergert, H.; Saeger, H.D.; Grützmann, R.; Bunk, A. Quantitative perfusion analysis of transabdominal contrast-enhanced ultrasonography of pancreatic masses and carcinomas. Gastroenterology 2009, 137, 1903–1911. [Google Scholar] [CrossRef] [PubMed]
  32. Napoléon, B.; Alvarez-Sanchez, M.V.; Gincoul, R.; Pujol, B.; Lefort, C.; Lepilliez, V.; Labadie, M.; Souquet, J.C.; Queneau, P.E.; Scoazec, J.Y.; et al. Contrast-enhanced harmonic endoscopic ultrasound in solid lesions of the pancreas: Results of a pilot study. Endoscopy 2010, 42, 564–570. [Google Scholar] [CrossRef] [PubMed]
  33. Fusaroli, P.; Kypraios, D.; Caletti, G.; Eloubeidi, M.A. Pancreatico-biliary endoscopic ultrasound: A systematic review of the levels of evidence, performance and outcomes. World J. Gastroenterol. 2012, 18, 4243–4256. [Google Scholar] [CrossRef] [PubMed]
  34. Gong, T.T.; Hu, D.M.; Zhu, Q. Contrast-enhanced EUS for differential diagnosis of pancreatic mass lesions: A meta-analysis. Gastrointest. Endosc. 2012, 76, 301–309. [Google Scholar] [CrossRef]
  35. Kitano, M.; Kudo, M.; Yamao, K.; Takagi, T.; Sakamoto, H.; Komaki, T.; Kamata, K.; Imai, H.; Chiba, Y.; Okada, M.; et al. Characterization of small solid tumors in the pancreas: The value of contrast-enhanced harmonic endoscopic ultrasonography. Am. J. Gastroenterol. 2012, 107, 303–310. [Google Scholar] [CrossRef]
  36. Fusaroli, P.; Kypraios, D.; Mancino, M.G.; Spada, A.; Benini, M.C.; Bianchi, M.; Bocus, P.; De Angelis, C.; De Luca, L.; Fabbri, C.; et al. Interobserver agreement in contrast harmonic endoscopic ultrasound. J. Gastroenterol. Hepatol. 2012, 27, 1063–1069. [Google Scholar] [CrossRef]
  37. Gincul, R.; Palazzo, M.; Pujol, B.; Tubach, F.; Palazzo, L.; Lefort, C.; Fumex, F.; Lombard, A.; Ribeiro, D.; Fabre, M.; et al. Contrast-harmonic endoscopic ultrasound for the diagnosis of pancreatic adenocarcinoma: A prospective multicenter trial. Endoscopy 2014, 46, 373–379. [Google Scholar] [CrossRef]
  38. Bunganič, B.; Laclav, M.; Dvořáková, T.; Bradáč, O.; Traboulsi, E.; Suchánek, S.; Frič, P.; Zavoral, M. Accuracy of EUS and CEH EUS for the diagnosis of pancreatic tumours. Scand. J. Gastroenterol. 2018, 53, 1411–1417. [Google Scholar] [CrossRef]
  39. Palazzo, M.; Napoléon, B.; Gincul, R.; Pioche, M.; Pujol, B.; Lefort, C.; Fumex, F.; Hautefeuille, V.; Fabre, M.; Cros, J.; et al. Contrast harmonic EUS for the prediction of pancreatic neuroendocrine tumor aggressiveness (with videos). Gastrointest. Endosc. 2018, 87, 1481–1488. [Google Scholar] [CrossRef]
  40. Soares, J.B.; Garcia, J.; Goncalves, B.; Lindkvist, B.; Noia, J.; Bastos, P.; Caetano, A.C.L.; Ferreira, A.; Pimentel-Nunes, P.; Lopes, L.; et al. Interobserver agreement of EUS elastography in the evaluation of solid pancreatic lesions. Endosc. Int. Open 2015, 4, 244–249. [Google Scholar] [CrossRef]
  41. Fernandez-del, C.C.; Targarona, J.; Thayer, S.P.; Rattner, D.W.; Brugge, W.R.; Warshaw, A.L. Incidental pancreatic cysts: Clinicopathologic characteristics and comparison with symptomatic patients. Arch. Surg. 2003, 138, 427–430. [Google Scholar] [CrossRef]
  42. Kiely, J.M.; Nakeeb, A.; Komorowski, R.A.; Wilson, S.D.; Pitt, H.A. Cystic pancreatic neoplasms: Enucleate or resect? J. Gastrointest. Surg. 2003, 7, 890–897. [Google Scholar] [CrossRef] [PubMed]
  43. Talamini, M.A.; Pitt, H.A.; Hruban, R.H.; Boitnott, J.K.; Coleman, J.; Cameron, J.L. Spectrum of cystic tumors of the pancreas. Am. J. Surg. 1992, 163, 117–123. [Google Scholar] [CrossRef]
  44. Ahmad, N.A.; Kochman, M.L.; Lewis, J.D.; Ginsberg, G.G. Can EUS alone differentiate between malignant and benign cystic lesions of the pancreas? Am. J. Gastroenterol. 2001, 96, 3295–3300. [Google Scholar] [CrossRef] [PubMed]
  45. Hernandez, L.V.; Bhutani, M.S. Endoscopic ultrasound and pancreatic cysts: A sticky situation! Am. J. Gastroenterol. 2001, 96, 3229–3230. [Google Scholar] [CrossRef]
  46. Koito, K.; Namieno, T.; Nagakawa, T.; Shyonai, T.; Hirokawa, N.; Morita, K. Solitary cystic tumor of the pancreas: EUS-pathologic correlation. Gastrointest. Endosc. 1997, 45, 268–276. [Google Scholar] [CrossRef]
  47. Muller, M.; Meyenberger, C.; Bertschinger, P.; Schaer, R.; Marincek, B. Pancreatic tumors: Evaluation with endoscopic US, CT, and MR imaging. Radiology 1994, 190, 745–751. [Google Scholar] [CrossRef]
  48. Sahani, D.V.; Kadavigere, R.; Saokar, A.; Fernandez-del Castillo, C.; Brugge, W.R.; Hahn, P.F. Cystic pancreatic lesions: A simple imaging-based classification system for guiding management. Radiographics 2005, 25, 1471–1484. [Google Scholar] [CrossRef]
  49. Ahmad, N.A.; Kochman, M.L.; Brensinger, C.; Brugge, W.R.; Faigel, D.O.; Gress, F.G.; Kimmey, M.B.; Nickl, N.J.; Savides, T.J.; Wallace, M.B.; et al. Interobserver agreement among endosonographers for the diagnosis of neoplastic versus non-neoplastic pancreatic cystic lesions. Gastrointest. Endosc. 2003, 58, 59–64. [Google Scholar] [CrossRef]
  50. Gonzalez, L.U.; Keane, M.G.; Pereira, S.P.; Iglesias-García, J.; Dominguez-Muñoz, E.; Lariño-Noia, J. Agreement among magnetic resonance imaging magnetic resonance cholangiopancreatography (MRI-MRCP) and endoscopic ultrasound (EUS) in the evaluation of morphological features of branch duct intraductal papillary mucinous neoplasm (BD-IPMN). Pancreatology 2018, 18, 170–175. [Google Scholar] [CrossRef]
  51. De Jong, K.; Verlaan, T.; Dijkgraaf, M.G.; Poley, J.W.; Dullemen, H.; Bruno, M.J.; Fockens, P. Interobserver agreement for endosonography in the diagnosis of pancreatic cysts. Endoscopy 2011, 43, 579–584. [Google Scholar] [CrossRef] [PubMed]
  52. Catalano, M.F.; Sivak, M.V., Jr.; Rice, T.; Gragg, L.A.; Van Dam, J. Endosonographic features predictive of lymph node metastasis. Gastrointest. Endosc. 1994, 40, 442–446. [Google Scholar] [CrossRef]
  53. Faige, D.O. EUS in patients with benign and malignant lymphadenopathy. Gastrointest. Endosc. 2001, 53, 593–598. [Google Scholar] [CrossRef] [PubMed]
  54. Eloubeidi, M.A.; Wallace, M.B.; Reed, C.E.; Hadzijahic, N.; Lewin, D.N.; Van Velse, A.; Leveen, M.B.; Etemad, B.; Matsuda, K.; Patel, R.S.; et al. The utility of EUS and EUS-guided fine needle aspiration in detecting celiac lymph node metastasis in patients with esophageal cancer: A single-center experience. Gastrointest. Endosc. 2001, 54, 714–719. [Google Scholar] [CrossRef] [PubMed]
  55. Okasha, H.; Elkholy, S.; Sayed, M.; Salman, A.; Elsherif, Y.; El-Gemeie, E. Endoscopic ultrasound-guided fine-needle aspiration and cytology for differentiating benign from malignant lymph nodes. Arab. J. Gastroenterol. 2017, 18, 74–79. [Google Scholar] [CrossRef]
  56. Chin, Y.K.; Iglesias-Garcia, J.; de la Iglesia, D.; Larino-Noia, J.; Abdulkader-Nallib, I.; Lázare, H.; Olmedo, S.R.; Dominguez-Muñoz, J.E. Accuracy of endoscopic ultrasound-guided tissue acquisition in the evaluation of lymph nodes enlargement in the absence of on-site pathologist. World J. Gastroenterol. 2017, 23, 5755–5763. [Google Scholar] [CrossRef]
  57. Chen, V.K.; Eloubeidi, M.A. Endoscopic ultrasound-guided fine needle aspiration is superior to lymph node echofeatures: A prospective evaluation of mediastinal and peri-intestinal lymphadenopathy. Am. J. Gastroenterol. 2004, 99, 628–633. [Google Scholar] [CrossRef]
  58. Takasaki, Y.; Irisawa, A.; Shibukawa, G.; Sato, A.; Abe, Y.; Yamabe, A.; Arakawa, N.; Maki, T.; Yoshida, Y.; Igarashi, R.; et al. New endoscopic ultrasonography criteria for malignant lymphadenopathy based on interrater agreement. PLoS ONE 2019, 14, e0212427. [Google Scholar] [CrossRef]
  59. Du Melo, A.W.; Panjala, C.; Crespo, S.; Diehl, N.N.; Woodward, T.A.; Raimondo, M.; Wallace, M.B. Interobserver agreement on the endosonographic features of lymph nodes in aerodigestive malignancies. Dig. Dis. Sci. 2011, 56, 3204–3208. [Google Scholar] [CrossRef]
  60. Xia, Y.; Kitano, M.; Kudo, M.; Imai, H.; Kamata, K.; Sakamoto, H.; Komaki, T. Characterization of intra-abdominal lesions of undetermined origin by contrast-enhanced harmonic EUS (with videos). Gastrointest. Endosc. 2010, 72, 637–642. [Google Scholar] [CrossRef]
  61. Giovannini, M.; Hookey, L.C.; Bories, E.; Pesenti, C.; Monges, G.; Delpero, J.R. Endoscopic ultrasound elastography: The first step towards virtual biopsy? Preliminary results in 49 patients. Endoscopy 2006, 38, 344–348. [Google Scholar] [CrossRef] [PubMed]
  62. Janssen, J.; Dietrich, C.F.; Will, U.; Greiner, L. Endosonographic elastography in the diagnosis of mediastinal lymph nodes. Endoscopy 2007, 39, 952–957. [Google Scholar] [CrossRef] [PubMed]
  63. Larsen, M.H.; Fristrup, C.W.; Mortensen, M.B. Intra- and interobserver agreement of endoscopic sonoelastography in the evaluation of lymph nodes. Ultraschall Med. 2011, 32, E45–E50. [Google Scholar] [CrossRef] [PubMed]
  64. Muguruma, N.; Okamura, S.; Ichikawa, S.; Tsujigami, K.; Suzuki, M.; Tadatsu, M.; Kusaka, Y.; Okita, Y.; Yano, M.; Ito, S. Endoscopic ultrasonography in the diagnosis of gallbladder wall lesions in patients with gallstones. J. Clin. Ultrasound. 2001, 29, 395–400. [Google Scholar] [CrossRef]
  65. Kim, H.J.; Park, J.H.; Park, D.I.; Cho, Y.K.; Sohn, C.I.; Jeon, W.K.; Kim, B.I.; Choi, S.H. Clinical usefulness of endoscopic ultrasonography in the differential diagnosis of gallbladder wall thickening. Dig. Dis. Sci. 2012, 57, 508–515. [Google Scholar] [CrossRef]
  66. Choi, J.H.; Seo, D.W.; Choi, J.H.; Park, D.H.; Lee, S.S.; Lee, S.K.; Kim, M.H. Utility of contrast-enhanced harmonic EUS in the diagnosis of malignant gallbladder polyps (with videos). Gastrointest. Endosc. 2013, 78, 484–493. [Google Scholar] [CrossRef]
  67. Park, C.H.; Chung, M.J.; Oh, T.G.; Park, J.Y.; Bang, S.; Park, S.W.; Kim, H.; Hwang, H.K.; Lee, W.J.; Song, S.Y. Differential diagnosis between gallbladder adenomas and cholesterol polyps on contrast-enhanced harmonic endoscopic ultrasonography. Surg. Endosc. 2013, 27, 1414–1421. [Google Scholar] [CrossRef]
  68. Imazu, H.; Mori, N.; Kanazawa, K.; Chiba, M.; Toyoizumi, H.; Torisu, Y.; Koyama, S.; Hino, S.; Ang, T.L.; Tajiri, H. Contrast-enhanced harmonic endoscopic ultrasonography in the differential diagnosis of gallbladder wall thickening. Dig. Dis. Sci. 2014, 59, 1909–1916. [Google Scholar] [CrossRef]
  69. Sakamoto, H.; Kitano, M.; Matsui, S.; Kamata, K.; Komaki, T.; Imai, H.; Dote, K.; Kudo, M. Estimation of malignant potential of GI stromal tumors by contrast-enhanced harmonic EUS. Gastrointest. Endosc. 2011, 73, 227–237. [Google Scholar] [CrossRef]
  70. Gress, F.; Schmitt, C.; Savides, T.; Faigel, D.O.; Catalano, M.; Wassef, W.; Roubein, L.; Nickl, N.; Ciaccia, D.; Bhutani, M.; et al. Interobserver agreement for EUS in the evaluation and diagnosis of submucosal masses. Gastrointest. Endosc. 2001, 53, 71–76. [Google Scholar] [CrossRef]
  71. Guo, W.; Han, W.; Liu, J.; Jin, L.; Li, J.S.; Zhang, Z.T.; Wang, Y. Intrapancreatic accessory spleen: A case report and review of the literature. World J. Gastroenterol. 2009, 15, 1141–1143. [Google Scholar] [CrossRef] [PubMed]
  72. Marques, S.; Bispo, M.; Noia, L. Intrapancreatic accessory spleen: A diagnosis not to forget! Case Rep. Gastroenterol. 2016, 10, 749–754. [Google Scholar] [CrossRef] [PubMed]
  73. Rodriguez, E.; Netto, G.; Li, Q.K. Intrapancreatic accessory spleen: A case report and review of literature. Diagn. Cytopathol. 2013, 41, 466–469. [Google Scholar] [CrossRef] [PubMed]
  74. Kim, G.E.; Morris, J.D.; Anand, N.; DePalma, F.; Greenwald, B.D.; Kim, R.E.; Laczek, J.; Lee, W.J.; Papadopoulas, I.; Uradomo, L.; et al. Recognizing intrapancreatic accessory spleen via EUS: Interobserver variability. Endosc. Ultrasound. 2019, 8, 392–397. [Google Scholar] [CrossRef]
  75. Caletti, G.C.; Ferrari, A.; Bocus, P.; Togliani, T.; Scalorbi, C.; Barbara, L. Endoscopic ultrasonography in gastric lymphoma. Schweiz Med. Wochenschr. 1996, 126, 819–825. [Google Scholar]
  76. Suekane, H.; Iida, M.; Yao, T.; Matsumoto, T.; Masuda, Y.; Fujishima, M. Endoscopic ultrasonography in primary gastric lymphoma: Correlation with endoscopic and histologic findings. Gastrointest. Endosc. 1993, 39, 139–145. [Google Scholar] [CrossRef]
  77. Palazzo, L.; Roseau, G.; Ruskone-Fourmestraux, A.; Rougier, P.; Chaussade, S.; Rambaud, J.C.; Couturier, D.; Paolaggi, J.A. Endoscopic ultrasonography in the local staging of primary gastric lymphoma. Endoscopy 1993, 25, 502–508. [Google Scholar] [CrossRef]
  78. Van Dam, J. The role of endoscopic ultrasonography in monitoring treatment: Response to chemotherapy in lymphoma. Endoscopy 1994, 26, 772–773. [Google Scholar] [CrossRef]
  79. Caletti, G.; Ferrari, A.; Brocchi, E.; Barbara, L. Accuracy of endoscopic ultrasonography in the diagnosis and staging of gastric cancer and lymphoma. Surgery 1993, 113, 14–27. [Google Scholar]
  80. Fusaroli, P.; Buscarini, E.; Peyre, S.; Federici, T.; Parente, F.; Angelis, C.D.; Bonanno, G.; Meroni, E.; Napolitano, V.; Pisani, A.; et al. Interobserver agreement in staging gastric malt lymphoma by EUS. Gastrointest. Endosc. 2002, 55, 662–668. [Google Scholar] [CrossRef]
  81. Yanai, H.; Harada, T.; Okamoto, T.; Hirano, A.; Takeo, N.; Yoshida, T.; Okita, K.; Kawano, T. Prognostic value and interobserver agreement of endoscopic ultrasonography for superficial squamous cell carcinoma of the esophagus: A prospective study. Int. J. Gastrointest. Cancer 2003, 34, 1–8. [Google Scholar] [CrossRef]
  82. Burtin, P.; Rabot, A.F.; Heresbach, D.; Carpentier, S.; Rousselet, M.C.; Le Berre, N.; Boyer, J. Interobserver agreement in the staging of rectal cancer using endoscopic ultrasonography. Endoscopy 1997, 29, 620–625. [Google Scholar] [CrossRef] [PubMed]
  83. Roubein, L.D.; Lynch, P.; Glober, G.; Sinicrope, F.A. Interobserver variability in endoscopic ultrasonography: A prospective evaluation. Gastrointest. Endosc. 1996, 44, 573–577. [Google Scholar] [CrossRef]
  84. Meining, A.; Rösch, T.; Wolf, A.; Lorenz, R.; Allescher, H.D.; Kauer, W.; Dittler, H.J. High interobserver variability in endosonographic staging of upper gastrointestinal cancers. Z. Gastroenterol. 2003, 41, 391–394. [Google Scholar] [CrossRef] [PubMed]
  85. Kuwahara, T.; Hara, K.; Mizuno, N.; Haba, S.; Okuno, N.; Koda, H.; Miyano, A.; Fumihara, D. Current status of artificial intelligence analysis for endoscopic ultrasonography. Dig. Endosc. 2020. Online ahead of print. [Google Scholar] [CrossRef]
  86. Marya, N.B.; Powers, P.D.; Chari, S.T.; Gleeson, F.C.; Leggett, C.L.; Abu Dayyeh, B.K.; Chandrasekhara, V.; Iyer, P.G.; Majumder, S.; Pearson, R.K.; et al. Utilisation of artificial intelligence for the development of an EUS-convolutional neural network model trained to enhance the diagnosis of autoimmune pancreatitis. Gut 2020. Online ahead of print. [Google Scholar] [CrossRef]
  87. Minoda, Y.; Ihara, E.; Komori, K.; Ogino, H.; Otsuka, Y.; Chinen, T.; Tsuda, Y.; Ando, K.; Yamamoto, H.; Ogawa, Y. Efficacy of endoscopic ultrasound with artificial intelligence for the diagnosis of gastrointestinal stromal tumors. J. Gastroenterol. 2020. Online ahead of print. [Google Scholar] [CrossRef]
  88. Kuwahara, T.; Hara, K.; Mizuno, N.; Okuno, N.; Matsumoto, S.; Obata, M.; Kurita, Y.; Koda, H.; Toriyama, K.; Onishi, S.; et al. Usefulness of Deep Learning Analysis for the Diagnosis of Malignancy in Intraductal Papillary Mucinous Neoplasms of the Pancreas. Clin. Transl. Gastroenterol. 2019, 10, 1–8. [Google Scholar] [CrossRef]
Table 1. Interobserver reliability of endoscopic ultrasonography (EUS) criteria in chronic pancreatitis.
Table 1. Interobserver reliability of endoscopic ultrasonography (EUS) criteria in chronic pancreatitis.
EUS FindingsTopazian (2007)
n = 20		Wallence (2001)
n = 33		Stevens (2010)
n = 50		Lieb (2011)
n = 16		Gardner (2011)
n = 44		Kalmin (2011)
n = 36		Del Poza (2011)
n = 69
Hyperechoic foci0.120.290.210.190.39NC0.48
Strands0.140.310.290.070.62NC0.55
Lobularity0.230.510.160.530.44NC0.75
Cysts0.480.320.35NC1NC0.66
Stones0.030.380.360.78NCNCNC
Duct dilationNC0.610.610.770.53NC0.75
Duct irregularity0.20.290.50.60NCNCNC
Hyperechoic MPD margin0.20.360.330.340.53NCNC
Dilated side branch0.090.180.460.11NCNCNC
Rosemont classificationNCNC0.65NCNC0.270.46
NC; not calculated.
Table 2. Interobserver reliability of contrast-enhanced harmonic endoscopic ultrasonography (CH-EUS) criteria in solid pancreatic lesion.
Table 2. Interobserver reliability of contrast-enhanced harmonic endoscopic ultrasonography (CH-EUS) criteria in solid pancreatic lesion.
AuthorYearDiseaseDegree of ExperienceK-Value
Kitano et al.
(n = 277)		2012AdenocarcinomaExperienced0.56
Fusaroli et al.
(n = 40)		2012SPLOverall
Experienced
Non-experienced		0.56
0.56
0.55
Gincul et al.
(n = 100)		2014SPLOverall
Experienced
Non-experienced		0.66
0.76
0.65
Bunganic et al.
(n = 116)		2018SPLOverall0.60
Palazzo et al.
(n = 81)		2018P-NETOverall
Experienced
Non-experienced		0.66
0.82
0.83
SPL, solid pancreatic lesion; P-NET, pancreatic neuroendocrine tumor.
Table 3. Interobserver reliability of EUS criteria for solid pancreatic lesion and pancreatic cystic lesion.
Table 3. Interobserver reliability of EUS criteria for solid pancreatic lesion and pancreatic cystic lesion.
EUS FindingsAhmad (2003)
n = 31		Fusaroli (2012)
n = 40		Jong (2011)
By Expert
n = 22		Jong (2011)
By Semi-Expert
n = 22		Jong (2011)
By Novice
n = 22
Contrast-enhanced(-)(+)(−)(−)(−)
Solid component0.43NCNCNCNC
Irregular pancreatic duct0.29NCNCNCNC
Septation0.30NCNCNCNC
Irregular margins0.01NC0.650.320.37
Abnormal pancreatic parenchyma0.01NCNCNCNC
Final diagnosis0.240.580.430.090.30
NC, not calculated.
Table 4. Interobserver reliability of EUS criteria in other diseases.
Table 4. Interobserver reliability of EUS criteria in other diseases.
DiseaseYearAuthornModalityK-Value
Lymphadenopathy2019Takasaki et al.68EUS0.65
2010Xia et al.43CH-EUS0.95
2007Jassen et al.50EUS-E0.84
2011Larsen et al.61EUS-E0.59
Gallbladder Wall Thickening2014Imazu et al.36EUS0.51
---CH-EUS0.77
Subepithelial Lesion2001Gress et al.20CH-EUS0.63
2012Fusaroli et al.40CH-EUSuptake 0.64,
pattern 0.18
washout 0.39
Intrapancreatic Accessory2019Kim et al.12EUS0.37
MALT Lymphoma2002Fusaroli et al.54EUST-stage before and after treatment = 0.38 and 0.37
N-stage before treatment and fair after treatment = 0.63 and 0.34
Staging of superficial esophageal cancer2003Yanai et al.31EUS0.31
Staging of rectal cancer1997Burtin et al.27EUST2 0.20
T3 0.58
MALT, Mucosa-associated Lymphoid Tissue.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yamamiya, A.; Irisawa, A.; Kashima, K.; Kunogi, Y.; Nagashima, K.; Minaguchi, T.; Izawa, N.; Yamabe, A.; Hoshi, K.; Tominaga, K.; et al. Interobserver Reliability of Endoscopic Ultrasonography: Literature Review. Diagnostics 2020, 10, 953. https://doi.org/10.3390/diagnostics10110953

AMA Style

Yamamiya A, Irisawa A, Kashima K, Kunogi Y, Nagashima K, Minaguchi T, Izawa N, Yamabe A, Hoshi K, Tominaga K, et al. Interobserver Reliability of Endoscopic Ultrasonography: Literature Review. Diagnostics. 2020; 10(11):953. https://doi.org/10.3390/diagnostics10110953

Chicago/Turabian Style

Yamamiya, Akira, Atsushi Irisawa, Ken Kashima, Yasuhito Kunogi, Kazunori Nagashima, Takahito Minaguchi, Naoya Izawa, Akane Yamabe, Koki Hoshi, Keiichi Tominaga, and et al. 2020. "Interobserver Reliability of Endoscopic Ultrasonography: Literature Review" Diagnostics 10, no. 11: 953. https://doi.org/10.3390/diagnostics10110953

APA Style

Yamamiya, A., Irisawa, A., Kashima, K., Kunogi, Y., Nagashima, K., Minaguchi, T., Izawa, N., Yamabe, A., Hoshi, K., Tominaga, K., Iijima, M., & Goda, K. (2020). Interobserver Reliability of Endoscopic Ultrasonography: Literature Review. Diagnostics, 10(11), 953. https://doi.org/10.3390/diagnostics10110953

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop