IOTA Simple Rules in Differentiating between Benign and Malignant Adnexal Masses by Non-expert Examiners

Objective: To evaluate the diagnostic performance of IOTA simple rules in predicting malignant adnexal tumors by non-expert examiners. Materials and Methods: Five obstetric/gynecologic residents, who had never performed gynecologic ultrasound examination by themselves before, were trained for IOTA simple rules by an experienced examiner. One trained resident performed ultrasound examinations including IOTA simple rules on 100 women, who were scheduled for surgery due to ovarian masses, within 24 hours of surgery. The gold standard diagnosis was based on pathological or operative findings. The five-trained residents performed IOTA simple rules on 30 patients for evaluation of inter-observer variability. Results: A total of 100 patients underwent ultrasound examination for the IOTA simple rules. Of them, IOTA simple rules could be applied in 94 (94%) masses including 71 (71.0%) benign masses and 29 (29.0%) malignant masses. The diagnostic performance of IOTA simple rules showed sensitivity of 89.3% (95%CI, 77.8%; 100.7%), specificity 83.3% (95%CI, 74.3%; 92.3%). Inter-observer variability was analyzed using Cohen’s kappa coefficient. Kappa indices of the four pairs of raters are 0.713-0.884 (0.722, 0.827, 0.713, and 0.884). Conclusions: IOTA simple rules have high diagnostic performance in discriminating adnexal masses even when are applied by non-expert sonographers, though a training course may be required. Nevertheless, they should be further tested by a greater number of general practitioners before widely use.


Introduction
It has been well accepted that preoperative differentiating between benign and malignant ovarian masses is important for plan of management since they need different interventions. For example, functional ovarian cysts usually need only expectant management, or endometriotic cysts may need laparoscopic conservative surgery, while malignant ovarian tumors need oncologists or well-planned consultation of highly-skilled surgeons or referral to a tertiary care center. Sonographic evaluation with several systems is the most commonly used to discriminate these two types of adnexal masses (Tongsong et al., 2007;Tongsong et al., 2009;Guerriero et al., 2011;Hafeez et al., 2013), especially when combined with other certain ultrasound characteristics, indicative of benignity (B-features) or malignancy (M-features), developed for widely use by general practitioners. IOTA simple rules have been reported to have a sensitivity of 92% and specificity of 96% (Timmerman et al., 2010). Our preliminary experience on IOTA simple rules also showed high accuracy (Tantipalakorn et al., 2014), when applied in our population. However, in most previous studies, IOTA simple rules have been performed by highlyskilled examiners and never been tested by inexperienced hands. Before widely used by general practitioners, IOTA simple rules should be evaluated for its effectiveness when performed by non-expert examiners. To determine whether IOTA simple rules could be applied among general practitioners or not, we therefore conducted this study aimed to evaluate their diagnostic performance in predicting malignant adnexal tumors by non-expert examiners.

Materials and Methods
This study was conducted on gynecologic patients scheduled for surgery because of an adnexal mass either detected by pelvic examination or previous ultrasound examination, at Department of Obstetrics and Gynecology, Chiang Mai University, between March 2014 and December 2014. The study was undertaken with ethical approval by the institute review board. The patients were counseled and invited to join the project with written informed consent. The patients with already known diagnoses of adnexal masses and patients undergoing surgery beyond 24 hours after ultrasound examination were excluded.
Phase I, five residents, who had never performed gynecologic ultrasound examination by themselves but had been exposed to gynecologic sonographic images or video clips before, were trained for IOTA simple rules by the experienced examiner (TT), using 50 teaching video clips for two weeks and then hands-on practice under supervision for 20 cases.
Phase II, all patients underwent ultrasound examination within 24 hours of surgery by the first author. All examinations were done with either transabdominal or transvaginal approach as suitable, using real-time 5-7.5 MHz transvaginal or 2.5-5 MHz transabdominal curvilinear transducer connected to a machine Hitachi-Aloka model ProSound37 (Hitachi Aloka Medical Ltd, Inc., Tokyo, Japan). On ultrasound examination, evaluation of mass morphology and vascularization was done using 2D real-time ultrasound and color flow mapping, and the results were prospectively recorded in the research forms. The descriptions of the masses were based on IOTA simple rules as defined elsewhere (Timmerman et al., 2008), to specify malignant (M) or benign (B) features, as presented in Table 1 and Figure 1. If one or more B-rules applied in the absence of an M-rule, the mass was predicted as benign. If one or more M-rules applied in the absence of a B-rule, the mass was predicted as malignant. If both M-rules and B-rules applied or no rule applied, the mass was classified as inconclusive.
The gold standard diagnosis was based on histopathological examinations or intra-operative diagnosis maded by the operators in some cases of benign disorders with no pathological specimens. All of adnexal masses were sonographically classified into 2 groups, as benign and malignant group. The masses with pathological diagnosis of low malignant potential tumors were classified as malignant group.
During the study periods, ultrasound examinations of 30 cases were performed by all of the five residents and recorded the results separately for evaluation of interobserver variability. Statistical analysis: Inter-observer variability in sonographic categorization between the five examiners was assessed for agreement using Cohen's kappa coefficient. Diagnostic performance of IOTA simple rules were calculated and presented as sensitivity, specificity, positive predictive values and negative predictive values, as well as 95% confidence interval. The statistical analysis was performed using IBM SPSS version 21.0 (IBM SPSS Statistics for Windows, Released 2012. Armonk, NY: IBM Corp).

Results
During one year of the study period, after one-month training program with 20 cases of hands-on practice, 100 adnexal masses from 100 patients initially diagnosed as ovarian tumors were enrolled in the study and underwent ultrasound examinations within twenty-four hours of operation. All of them were satisfactorily performed either via transabdominal or transvaginal ultrasound approach. The mean (+SD) age of the women was 44.21+12.9   19-75 years). Forty-six cases (46.0%) were nulliparous. Most women (73 women, 73.0%) were in reproductive age, and 27 (27.0%) were in postmenopausal period. Most of them (71%) had no contraception, whereas 11% had hormonal contraception and 10% had undergone tubal sterilization. Additionally, only one patient was on hormone replacement therapy.

Discussion
Preoperative differentiating between benign and malignant ovarian masses is essential for plan of management, especially in areas of unavailability of gynecologic oncologists. IOTA simple rules enable general gynecologists or practitioners to categorize which patients should be referred to the tertiary centers where oncologists are available or they should manage by themselves. Though, IOTA simple rules have been tested for effectiveness by several studies (Di Legge et al., 2012;Nunes et al., 2012;Ruiz de Gauna et al., 2014;Timmerman et al., 2008;Timmerman et al., 2010), including external vaildiation (Alcazar et al., 2013;Sayasneh et al., 2013a;Sayasneh et al., 2013b;Tantipalakorn et al., 2014), none of them evaluate the effectiveness among non-expert sonographers. Therefore we conducted this study to see whether or not they could be applied by general practitioners. This study indicates that IOTA simple rules could be effectively performed by non-expert examiners with high sensitivity and specificity. Thus, the simple rules seem to be attractive and practical because they need no high expertise, and are user-friendly in differentiating an adnexal mass as benign or malignant.
Compared to our previous study in which ultrasound examinations were performed by an experienced sonographer, the effectiveness of the IOTA rules were comparable. The sensitivity and specificity in this study were 89.3% and 83.3%, respectively, compared to 82.9% and 95.3%, respectively in our previous one (Tantipalakorn et al., 2014). The findings suggested that the simple IOTA rules likely be reproducible when applied by  general practitioners or even non-expert sonographers, though they certainly need a training course or practice under supervision for an appropriate time. Surprisingly, inconclusive results were found in only 6% of the cases in this study, better than those observed in the previous studies. The reasons of the discrepancy are still unclear. The weakness of this study included the followings. 1) Ultrasound examinations of the residents in this study were in early learning curves of their practice. The results might have not been perfectly reliable. Nevertheless, the results of relatively high effectiveness in differentiating benign and malignant masses suggest that interpretation of the B-rules and M-rules be simple and could be successfully trained in no time. 2) Another possible weakness was that only adnexal masses undergoing surgery were enrolled. This was mandatory since pathological examinations or operative findings were required, as a gold standard for determining diagnostic performance. However, this bias was unlikely to affect the conclusion because masses not requiring operation tended to be less complex and easier to be categorized on ultrasound examination. 3) Only one examiner (DT) performed ultrasound examinations. Nevertheless, Cohen's kappa coefficient indicated good inter-rater agreement among five non-expert examiners, supporting reproducibility in widely use by qualified or trained general practitioners, like the five residents in this study.
In conclusion, IOTA simple rules have high diagnostic performance in discriminating adnexal masses even when are applied by non-expert sonographers, though a training course may be required. Additionally, our findings, as external validation, suggest that IOTA simple rules be reproducible. Together with its simplicity to learn and practice, thus the IOTA rules are probably suitable for widely use, by any qualified ultrasound practitioner. Nevertheless, they should be further tested by a greater number general practitioners before widely use.