IOTA Three-Step Strategy for Classifying Adnexal Masses: A Systematic Review and Meta-Analysis

Simple Summary

Adnexal masses remain a significant clinical problem. Ultrasound is considered the first-line diagnostic technique. Several approaches have been proposed. IOTA group proposed a three-step strategy in 2012. Our meta-analysis aimed to evaluate the diagnostic performance of this strategy. Seven studies were selected, comprising a total of 5772 women. The pooled sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of the three-step strategy for adnexal mass classification were 94%, 94%, 17.0, and 0.07, respectively. We conclude that the three-step strategy has a good diagnostic performance.

Abstract

Background: Our goal was to assess the diagnostic performance of the IOTA 3-step strategy for discriminating benign from malignant adnexal masses. Methods: Systematic review and meta-analysis design. A systematic search across three databases (Medline [PubMed], SCOPUS, and Web of Science) was conducted to identify primary studies reporting on the use of the IOTA three-step strategy from January 2012 to July 2024. Prospective cohort studies utilizing the three-step strategy, with histologic diagnosis or conservative management confirming spontaneous resolution or persistence in cases of benign-appearing masses for at least one year of follow-up, were used as the reference standard. Studies unrelated to the topic, those not addressing the IOTA three-step strategy, studies focusing on other prediction models, letters to the editor, commentaries, narrative reviews, consensus documents, and studies lacking data for constructing a 2 × 2 table were excluded. Quantitative synthesis was done, calculating the pooled sensitivity, specificity, and positive and negative likelihood ratios. Qualitative synthesis was done using QUADAS-2. Results: A total of 448 citations were initially identified, with 7 studies meeting inclusion criteria, comprising 5722 patients. The mean prevalence of ovarian malignancy was 28%. The quality of the studies was considered good. IOTA 3-step strategy showed a pooled sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of the three-step strategy for adnexal mass classification were 94% (95% CI = 91–95%), 94% (95% CI = 91–97%), 17.0 (95% CI = 10–28.8), and 0.07 (95% CI = 0.05–0.1), respectively. Heterogeneity for sensitivity was moderate, and for specificity it was high. Conclusions: We conclude that the three-step strategy has good diagnostic performance, reducing the need for expert examiner evaluation.

1. Introduction

Adnexal masses pose a common clinical challenge in gynecology, with diagnoses ranging from benign conditions to malignant ovarian tumors. Accurate preoperative assessment is crucial for guiding treatment, minimizing unnecessary surgeries, and ensuring timely intervention for malignancies [1]. Although subjective evaluation by expert examiners is considered the best approach, limited availability of experts has prompted the development of alternative methods to address this widespread issue. In 2000, a group of expert sonologists formed the International Ovarian Tumor Analysis (IOTA) group, creating a consensus on defining the morphological features of ovarian masses and proposing a standardized examination technique to establish a reproducible approach across different countries [2]. This standardization facilitated a common diagnostic language internationally, yet the scarcity of expert examiners remained a challenge.

To support non-expert examiners, the IOTA group developed several mathematical models or classification systems for assessing the risk of malignancy. In 2005, they introduced two logistic regression models with 12 (LR1) and 6 (LR2) clinical and sonographic variables, respectively [3]. In 2010, IOTA demonstrated that both logistic models were equally effective for discriminating benign from malignant adnexal masses [4]. In 2008, they proposed the so-called “simple rules” approach, a classification system that could differentiate benign from malignant adnexal masses in approximately 75% of cases based on well-defined benign or malignant morphologic features [5]. Subsequently, in 2012, the IOTA group introduced a three-step strategy designed for non-expert examiners to reach a diagnosis in most cases: the first step used simple descriptors to immediately identify common benign adnexal masses, while the second step applied the simple rules for distinguishing between benign and malignant masses when the initial descriptors were inconclusive. If both steps were insufficient, the third step involved referral to an expert for further evaluation [6]. Finally, the IOTA group reported the ADNEX model in 2014 [7], which is a dichotomous (benign versus malignant), but also polytomous model for identifying different types of malignant lesions, such as borderline tumors, primary ovarian cancers, and metastatic tumors to the ovary. This ADNEX model has also been stated as a method for assigning risk groups when using the so-called O-RADS classification [8].

Several meta-analyses of primary studies addressing the diagnostic performance of LR1, LR2, simple rules, ADNEX model, and O-RADS classification have been reported [9,10,11,12,13,14,15,16,17,18,19,20,21].

However, to date, there is no meta-analysis reported assessing the diagnostic performance of the IOTA three-step strategy. This stepwise approach offers a practical solution for non-expert examiners, allowing them to differentiate between most benign and malignant cases effectively, leaving inconclusive cases to experts. In fact, several prospective primary studies have validated the utility of this strategy in clinical practice.

This meta-analysis aims to synthesize the current literature on the three-step strategy in distinguishing benign from malignant adnexal masses and to provide robust evidence on the diagnostic accuracy and limitations of this approach in routine clinical settings.

2. Materials and Methods

2.1. Protocol and Registration

We performed this meta-analysis according to PRISMA recommendations (http://www.prisma-statement.org (accessed on 2 July 2024) and the SEDATE guidelines [22]. We did not register the protocol. However, we defined the inclusion and exclusion criteria for studies to be selected, as well as how data extraction and quality assessment should be performed, before starting the study.

2.2. Data Sources and Searches

Three electronic databases, namely PubMed/Medline, Scopus, and Web of Science, were searched. The period of search was stated from January 2012 (the year of first publication of the IOTA 3-step strategy) until June 2024. We used the following terms: “adnexal masses”, “ultrasound“, and/or “IOTA”. No language limit was set.

2.3. Study Selection and Data Collection

The first task was to delete all duplicate articles. After deleting duplicates, the titles and abstracts of the retained citations were read to exclude irrelevant articles. This was done by three authors.

Then, four authors selected full-text articles to identify potentially eligible studies applying the following three inclusion criteria:

• Prospective cohort study including patients diagnosed as having at least one adnexal mass classified using the three-step strategy after transvaginal/transabdominal ultrasound assessment.
• Report of histologic diagnosis of the adnexal mass after surgical removal or conservative management, demonstrating spontaneous resolution or persistence in cases of benign appearing masses after 12 months of follow-up scan as the reference standard.
• Presence of data reported that would allow constructing a 2 × 2 table to estimate true positive, true negative, false positive, and false negative cases for the three steps strategy.

Studies not related to the topic under review, studies not using three-step strategy, letters to the editor, commentaries, doctoral thesis, narrative reviews, consensus documents, and studies where no data were available for constructing a 2 × 2 table (no data about true positive, true negative, false positive, and false negative cases were available), were excluded

2.4. Risk of Bias in Individual Studies

Quality assessment of studies included was conducted using the QUADAS-2 tool [23]. This tool comprises four items: patient selection, index test, reference standard, and flow and timing. For each item, the risk of bias and concerns about applicability (the latter not applying to the domain of flow and timing) were assessed and rated as low, high, or unclear risk. Quality assessment was used to evaluate the overall quality of the studies and investigate potential sources of heterogeneity. Three authors independently performed this assessment. When a disagreement appeared, this was solved by discussion and consensus between the authors.

Quality assessment was based on data reported in the selected studies. For example, study’s design, inclusion and exclusion criteria, how the of the index test was performed and interpreted, which was the reference standard used (for this domain, in case of conservative management of the mass, at least 1 year of follow-up was considered as appropriate to identify true negative or false negative cases), and description of the time elapsed from index test assessment to the reference standard result for the flow and timing domain (surgery > 120 days after diagnosis was considered as high-risk). Unclear risk was stated when the corresponding information for each domain was not reported in the study.

2.5. Statistical Analysis

Three authors extracted or derived information related to the diagnostic performance of the three-step strategy (Figure 1), namely the number of true positives, true negatives, false positives, and false negatives reported in each study. This strategy consists of applying first the simple descriptors by a non-expert sonographer (first step) (

Table 1. Description of IOTA Simple Descriptors.

Benign Descriptors	Malignant Descriptors
Unilocular tumor with ground glass echogenicity in premenopausal age.	Tumor with ascites and at least moderate color Doppler blood flow in postmenopausal women.
Unilocular tumor with mixed echogenicity and acoustic shadows in premenopausal age. Unilocular tumor anechoic tumor with regular walls and largest diameter of less than 100 mm.	Clinical: Women aged > 50 years and Laboratory: Serum CA-125 > 100.

). If the mass could not be classified as benign or malignant according to simple descriptors, the same examiner applied simple rules to classify the mass (second step) (

Table 2. IOTA Simple Rules.

Benign Features	Malignant Features
B1 Unilocular cyst B2 Solid components < 7 mm B3 Acoustic shadows B4 Smooth multilocular cyst < 100 mm B5 Color flow score 1	M1 Irregular solid tumor M2 Ascites M3 At least 4 papillary projections M4 Irregular multilocular solid > 100 mm M5 Color flow score 4

). Finally, if the mass could not be classified using simple rules, an expert examiner evaluated the mass and classified it as benign, malignant, or uncertain according to their subjective impression (step three). In the case of an uncertain classification, the mass was managed as a malignant tumor, and the woman was referred to the gynecological oncology division in most of the studies.

We used the random effects model to estimate the pooled sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio (LR−). Likelihood ratios were used to characterize the clinical utility of a test and to estimate the post-test probability of disease. Using the mean prevalence of ovarian malignancy (pre-test probability), post-test probabilities were calculated using the positive and negative likelihood ratios and plotted on Fagan’s nomogram.

Heterogeneity for sensitivity and specificity was assessed using Cochran’s Q statistic and the I² index [24]. A p-value < 0.1 indicates heterogeneity. I2 values of 25%, 50%, and 75% would be considered to indicate low, moderate, and high heterogeneity, respectively.

Forest plots of the sensitivity and specificity of all studies were plotted. Meta-regression was used to assess covariates that could explain this heterogeneity, if heterogeneity existed. The covariates analyzed were sample size, malignancy prevalence, country, and number of centers (one versus more than one). Summary receiver operating characteristic (sROC) curves were plotted to illustrate the relationship between sensitivity and specificity. Lastly, publication bias was assessed using Deeks’ method [25].

All analyses were performed using MIDAS and METANDI commands in STATA version 12.0 for Windows (Stata Corporation, College Station, TX, USA). A p-value < 0.05 was considered statistically significant.

A subgroup analysis was performed, excluding data from two studies where expert examiners conducted the first and second steps, to specifically assess sensitivity and specificity among non-expert examiners performing these initial steps.

3. Results

3.1. Search Results

The electronic research identified 448 citations (123 articles in PubMed/MEDLINE, 190 in Web of Science, 132 in Scopus, none in The Cochrane Library). Two hundred and fifteen citations were duplicate records, which were excluded, and 233 citations remained. After reading titles and abstracts, 203 citations were excluded. The full text of the remaining 30 articles was examined.

Out of these 30 papers, twenty-three papers were excluded for the following reasons: being editorials or theses, not following the SD as recommended, no specifications about the author or year of publication, and in one of them, the adnexal masses were only endometriomas. No paper was excluded for the absence of data related to true positive, true negative, false positive, and false negative cases. Seven papers were ultimately included in the qualitative and quantitative synthesis [6,26,27,28,29,30,31]. A flowchart summarizing the literature search is shown in Figure 2.

3.2. Characteristics of Included Studies

The characteristics of the seven selected studies are shown in

Table 3. Main Characteristics of the Studies Included in the Meta-Analysis.

Author	Year	Country	Study Design	N	N Malignancy	N Centres	Index Test	N Observers	Reference Test	Flow and Timing ***
Ameye et al. [6]	2012	Belgium *	Prospective	1938	951	21	IOTA 3-step	Multiple **	Histology	<120 days
Sayasneh et al. [26]	2013	UK	Prospective	301	92	3	IOTA 3-step	36	Histology	<120 days
Testa et al. [27]	2014	Italy *	Prospective	2403	980	18	IOTA 3-step	Multiple **	Histology	<120 days
Peces-Rama et al. [28]	2015	Spain	Prospective	81	7	1	IOTA 3-step	3	Histology or FU	<120 days
Alcázar et al. [29]	2016	Spain	Prospective	666	53	2	IOTA 3-step	9	Histology or FU	<120 days
Hidalgo et al. [30]	2019	Spain	Prospective	283	62	2	IOTA 3-step	6	Histology or FU	<60 days
Grover et al. [31]	2020	India	Prospective	100	57	1	IOTA 3-step	2	Histology	N.A.

* Several countries participated. ** Not stated how many observers participated. NA: data not available. FU: Follow-up. *** This is the time elapsed between the US and surgery, and does not apply to those women managed conservatively.

[6,26,27,28,29,30,31].

Seven studies were published between 2012 and 2020; they included a total of 5772 patients with at least one adnexal mass. Among these 5772 masses, 1794 were malignant (studied with histology after surgery was performed). After all, the mean prevalence of ovarian malignancy was 28%, ranging from 8% to 57%,

All studies were prospective observational studies, most of them multicentric, except for one study, which was developed only in one single hospital [28].

The ultrasound examination was carried out via transvaginal in all women. The exam was performed by non-expert observers in one study [26], as in the other two studies, all the masses were analyzed by experts only [6,27]. In the other 4 studies, both expert and non-expert examiners performed the first two steps. Surgical findings were used as the reference standard in all 7 studies, but 3 of them also included follow-up of benign masses for one year [28,29,30]. The total number of women who underwent surgery was 5350, and 422 women were managed conservatively with follow-up.

All studies except one of them [26], explained the 3-step system as follows: first step using SD, if second step is required, then the SR are used, finally if the mass cannot be classified with the 2 steps before mentioned, then a expert examiner determined whether the mass was classified as benign or malignant. As stated above, one study only uses nonexpert examiners, so the third step was classified by the same nonexpert examiners as a malignant mass directly.

3.3. Qualitative Synthesis

All studies were considered to have a low risk of bias in the domain of patient selection (

Table 4. Qualitative Assessment of Studies Included According to QUADAS-2 Criteria.

Study	Patient Selection	Index Test	Reference Test	Flow and Timing
Ameye [6]	Low	Low	Low	Low
Sayasneh [26]	Low	High	Low	Low
Testa [27]	Low	Low	Low	Low
Peces-Rama [28]	Low	Low	Low	Low
Alcázar [29]	Low	Low	Low	Low
Hidalgo [30]	Low	Low	Low	Low
Grover [31]	Low	Low	Low	Unclear

).

Concerning bias for the index test, all studies except one were deemed to have a low risk of bias. The study, considered as having a high risk of bias, did not report clearly who the examiner (expert or non-expert) was who performed the third step [26]. Regarding the reference standard domain, all studies were considered as having a low risk of bias.

Finally, regarding the flow and timing domain, the risk of bias was low for all studies, but only one did not report this information [31].

In terms of applicability, all studies were assessed as low risk for all domains assessed.

3.4. Quantitative Synthesis

The pooled sensitivity and specificity of the IOTA three-step strategy for the preoperative characterization of benign and malignant ovarian tumors were 94.0% (95% confidence interval [CI], 91.0% to 95.0%) and 94.0% (95% CI, 91.0% to 97.0%), respectively. The LR+ and LR− were 17.0 (95% CI, 10.0 to 28.8) and 0.07 (95% CI, 0.05 to 0.10), respectively. The DOR was 248 (95% CI, 120 to 511).

High heterogeneity was observed for sensitivity (I² = 82.2; Cochran Q = 33.8; p < 0.001) along with high heterogeneity for specificity (I² = 94.6; Cochran Q = 110.7; p < 0.001 (Figure 3).

Summary receiver operating characteristic curves illustrating the diagnostic performance of the IOTA three-step strategy are presented in Figure 4. The area under the curve was 0.95 (95% CI: 0.93–0.97).

We observed that when IOTA three-step strategy resulted in suspicion of malignancy the probability of malignancy significantly increased the pretest probability from 28% to 87%. Conversely, if IOTA three-step strategy renders a result of benignity, the risk of malignancy significantly reduced the pretest probability from 28% to 3% (Figure 5).

Publication bias was observed (Figure 6).

In the sensitivity analysis, including only five studies in which the first two steps of IOTA three-step strategy were performed by non-expert examiners, we found that the pooled sensitivity and specificity of the IOTA three-step strategy for the preoperative characterization of benign and malignant ovarian tumors were 95.0% (95% confidence interval [CI], 91.0% to 97.0%) and 96.0% (95% CI, 92.0% to 98.0%), respectively. No heterogeneity was observed for sensitivity (I² = 0; Cochran Q = 3.1; p = 0.53) and moderate heterogeneity was observed for specificity (I² = 71.6; Cochran Q = 14.1; p = 0.001 (Figure 7).

4. Discussion

4.1. Summary of Evidence

In this study, we conducted a systematic review and meta-analysis of the three-step strategy for sonographic assessment of adnexal masses. We identified seven prospective studies with data on over 5772 patients, including both pre- and postmenopausal women. Across these studies, there were 1794 malignant tumors, with a mean prevalence of 28% (range: 8–57%).

Our analysis showed pooled sensitivity and specificity of the three-step strategy at 94% each. All studies applied the IOTA group’s 2012 three-step strategy, though examiner expertise levels varied in the first and second steps. In five of the seven studies, non-expert examiners performed the initial simple descriptors (step one) and simple rules (step two) assessments [28,29,30,31]. Notably, in a subgroup analysis of these five studies, diagnostic performance remained high, with sensitivity and specificity reaching 95% and 96%, respectively, and there was no heterogeneity.

4.2. Limitations and Strengths

The main strength of this study is that to the best of our knowledge it is the first meta-analysis studying the three steps strategy. It included only prospective clinical trials, obtaining a large number of patients. The methodology was correct, considering that the quality of the studies was high.

Among the main limitations is that the number of studies is low. Also, heterogeneity was moderate for sensitivity and high for specificity. For these reasons, we do consider that our results should be taken with caution. On the other hand, all but one study was performed in Europe, particularly in Spain. This fact could raise a question about the generalizability of the results we have observed.

4.3. Interpretation of Results

Our data indicate that the three-step strategy provides very high sensitivity and specificity for classifying adnexal masses, with good quality evidence and moderate heterogeneity for sensitivity, though specificity varied widely. Most studies demonstrated high diagnostic performance regardless of whether non-expert or expert examiners conducted the first and second steps.

However, the three-step strategy appears to be falling out of use, with the latest study published in 2020 [31]. This fact is to be noted because the diagnostic performance of this strategy is similar to or even better than that of other approaches proposed by IOTA. In fact, meta-analyses about IOTA simple rules observed that the pooled sensitivity and specificity were 91–96% and 80–92% [9,10,11,12,20]. For the LR2 model, these figures were 93% and 84% [11]. For the ADNEX model, the pooled sensitivity and specificity were 91–97% and 77–84% [14,15,16], and for the O-RADS classification, the pooled sensitivity and specificity were 95–97% and 77–89% [17,18,19,20,21]. In other words, the three-step strategy should not be discarded as a strategy for discriminating benign from malignant masses. Furthermore, this strategy significantly reduces the number of cases that need to be referred to an expert examiner, potentially improving the workflow of women diagnosed as having an adnexal mass and facilitating early management.

Despite this, in 2021, a consensus statement from the European Society of Gynecologic Oncology (ESGO), the European Society of Gynecologic Endoscopy (ESGE), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), and the IOTA group recommended the direct use of the ADNEX model or an expert examination after ruling out normal ovaries and physiological changes [32].

More recently, in 2023, the IOTA group proposed a two-step strategy, using simple descriptors for initial assessment to identify common benign masses and the ADNEX model as a second step if needed. This new approach completes the assessment without requiring an expert examiner [33].

Despite this trend in the literature, subjective assessment by an expert examiner remains the gold standard for diagnosing adnexal masses. We believe any effective strategy should ideally include this type of evaluation, as in the three-step strategy. This approach also emphasizes the importance of training expert sonologists in gynecology within clinical centers. It is noteworthy that the first two steps have shown good reproducibility, both for the benign easy descriptors (first step) [34] and for the simple rules (second step) [35,36]. It could be questioned whether the three-step strategy is a reproducible approach, but evaluating a multi-step process involving different examiners would be inherently very challenging. Although this aspect has not been specifically studied, the forest plots from the included studies demonstrate remarkably similar reported sensitivity and specificity. This consistency suggests, to some extent, that the approach may have good reproducibility.

From a clinical perspective, the three-step strategy remains a rational and practical alternative for assessing adnexal masses, particularly when expert examination is not immediately available. This strategy enables an efficient approach by initially identifying common benign masses through simple descriptors, utilizing the simple rules model in a second step for further assessment, and involving an expert only in inconclusive cases.

Interestingly, a sensitivity analysis was performed to observe the percentage of adnexal masses that were classified with the first two steps, which would indicate how many of these masses would be classifiable by non-experts. It was observed that 82.8% of the masses were classified by non-experts. This result is interesting given that most masses can be classified by non-expert sonographers using SD and SR.

4.4. Future Research Agenda

It would be of interest to compare the three-step versus the two-step strategies in the future and determine whether there are differences in the diagnostic performance in clinical scenarios where non-expert examiners are the first line of work-up. Further studies are needed to evaluate cases where non-expert examiners perform the first two steps, as this scenario would be particularly suited to the three-step strategy. Additionally, it would be valuable to investigate whether the second step of the three-step strategy performs better using the simple rules as originally proposed or the ADNEX model.

5. Conclusions

In conclusion, our data indicate that the three-step strategy performs well in distinguishing between benign and malignant adnexal masses. However, further studies are needed that specifically evaluate the first and second steps when used by non-expert examiners to determine if this approach should become the standard for ultrasound evaluation of adnexal masses in cases where expert examiners are not involved in the initial assessment.