• Korean Journal of Radiology
• /
• v.25 no.7
• /
• pp.656-661
• /
• 2024

Evaluating the performance of a binary diagnostic test, including artificial intelligence classification algorithms, involves measuring sensitivity, specificity, positive predictive value, and negative predictive value. Particularly when comparing the performance of two diagnostic tests applied on the same set of patients, these metrics are crucial for identifying the more accurate test. However, comparing predictive values presents statistical challenges because their denominators depend on the test outcomes, unlike the comparison of sensitivities and specificities. This paper reviews existing methods for comparing predictive values and proposes using the permutation test. The permutation test is an intuitive, non-parametric method suitable for datasets with small sample sizes. We demonstrate each method using a dataset from MRI and combined modality of mammography and ultrasound in diagnosing breast cancer.

• Korean Journal of Adult Nursing
• /
• v.27 no.1
• /
• pp.29-38
• /
• 2015

Purpose: The purpose of the study was to validate fall risk assessment scales among hospitalized adult patients in South Korea using the electronic medical records by comparing sensitivity, specificity, positive predictive values, and negative predictive values of Morse Fall Scale (MFS), Bobath Memorial Hospital Fall Risk Assessment Scale (BMFRAS), and Johns Hopkins Hospital Fall Risk Assessment tool (JHFRAT). Methods: A total of 120 patients who experienced fall episodes during their hospitalization from June 2010 to December 2013 was categorized into the fall group. Another 120 patients, who didn't experience fall episodes with age, sex, clinical departments, and the type of wards matched with the fall group, were categorized to the comparison group. Data were analyzed for the comparisons of sensitivity, specificity, positive and negative predictive values, and the area under the curve of the three tools. Results: MFS at a cut-off score of 48 had .806 for ROC curves, 76.7% for sensitivity, 77.5% for specificity, 77.3% for positive predictive value, and 76.9% for negative predictive value, which were the highest values among the three fall assessment scales. Conclusion: The MFS with the highest score and the highest discrimination was evaluated to be suitable and reasonable for predicting falls of inpatients in med-surg units of university hospitals.

• Journal of the Korean Association of Oral and Maxillofacial Surgeons
• /
• v.33 no.4
• /
• pp.367-374
• /
• 2007

Purpose: The diagnostic relevancies and characteristics and of clinical methods in the diagnosis of internal derangement(ID) were tested by comparing the results of them with those of magnetic resonance imaging(MRI). Methods: 75 patients(150 temporomandibular joints; TMJs), who were suspected to have ID by clinical diagnoses, were included. Clinical diagnoses including mouth opening pathway and TMJ sound were conducted and MRI takings were done. Accuracies, sensitivities, specificities, positive predictive values, and negative predictive values of clinical diagnosis, mouth opening pathway, and TMJ sound were calculated by comparing with diagnoses with MRIs. Results: Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of clinical diagnosis were 59.3%, 83%, 49%, 81%, and 51%. They were 59%, 82%, 25%, 73%, and 35% for mouth opening pathways. Although deviation was somewhat accurate for representing disc displacement with reduction(ADDWR), other discrepancies on opening pathways were not clinically relevant. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of clicking sounds were 85%, 49%, 78%, 85%, and 37%. TMJs with crepitus were only three. But all TMJs with crepitus were diagnosed to have disc displacement without reduction(ADDWOR). Conclusion: When compared with diagnoses with MRIs, clinical diagnoses for ID were not so accurate. But they were suitable for screening tests for ID. Opening pathways and TMJ sounds were not so relevant in the diagnoses of IDs and so it was concluded that considerations for other factors must be included in the diagnoses of IDs.

• Pediatric Gastroenterology, Hepatology & Nutrition
• /
• v.22 no.4
• /
• pp.350-357
• /
• 2019

Purpose: This study aimed to evaluate a possible association between the anti-tissue transglutaminase antibody (anti-tTG) titer and stage of duodenal mucosal damage and assess a possible cut-off value of anti-tTG at which celiac disease (CD) may be diagnosed in children in conjunction with clinical judgment. Methods: This observational study was conducted at a gastroenterology clinic in a tertiary hospital from April 2012 to May 2013. Seventy children between 6-months and 18-years-old with suspected CD underwent celiac serology and duodenal biopsy. Statistical analyses were done using SPSS 16. Diagnostic test values were determined for comparing the anti-tTG titer with duodenal biopsy. An analysis of variance and Tukey-Kramer tests were performed for comparing the means between groups. A receiver operating characteristics curve was plotted to determine various cut-off values of anti-tTG. Results: The mean antibody titer increased with severity of Marsh staging (p<0.001). An immunoglobulin (Ig) A-tTG value at 115 AU/mL had 76% sensitivity and 100% specificity with a 100% positive predictive value (PPV) and 17% negative predictive value (NPV) for diagnosis of CD (p<0.001, 95% confidence interval [CI], 0.75-1). Conclusion: There is an association between the anti-tTG titer and stage of duodenal mucosal injury in children with CD. An anti-tTG value of 115 AU/mL (6.4 times the upper normal limit) had 76% sensitivity, 100% specificity, with a 100% PPV, and 17% NPV for diagnosing CD (95% CI, 0.75-1). This cut-off may be used in combination with clinical judgment to diagnose CD.

• The Korean Journal of Cytopathology
• /
• v.18 no.2
• /
• pp.119-125
• /
• 2007

Urine cytology is an important screening tool for urinary tract neoplasms. Liquid-based preparation methods, such as $ThinPrep^{(R)}$, have been introduced for non-gynecological samples. We aimed to assess the diagnostic accuracy of liquid-based preparations in urine cytology by comparing the results of the conventional Cytospin preparation method for the same samples. A total of 236 cases subject to urine cytology were enrolled in this study from January 2005 to December 2005. All cases were subjected to cystoscopy and if a malignancy was suspected, a biopsy was performed. Urine cytology slides were made using the $ThinPrep^{(R)}$ preparation method and the conventional Cytospin and/or direct smear method from the individual samples. The results of urine cytology were compared with the final cystoscopic or histological diagnoses. We analyzed the sensitivity, specificity, positive predictive value, negative predictive value and accuracy of both cytology preparation methods. A total of 236 slides made using the liquid based method were satisfactory for slide quality, whereas 5 slides (2.1%) prepared by conventional methods were unsatisfactory because of air-drying, a thick smear, or a bloody or inflammatory background. The $ThinPrep^{(R)}$ method showed 53.1% sensitivity, 92.6% specificity, a 92,6% positive predictive value, a 94.1% negative predictive value and 85,6% accuracy, while the conventional method showed 51% sensitivity, 98.4% specificity, a 92.6% positive predictive value, a 98.4% negative predictive value and 88,6% accuracy. Although the diagnostic values were equivalent between the use of the two methods, the quality of the cytology slides and the time consumed during the microscopic examination for a diagnosis were superior for the $ThinPrep^{(R)}$ method than for the conventional method. In conclusion, our limited studies have shown that the use of the liquid based preparation method is beneficial to improve the quality of slides and reduce the duration for a microscopic examination, but did not show better sensitivity, accuracy and predictive values.

• Structural Engineering and Mechanics
• /
• v.84 no.2
• /
• pp.143-154
• /
• 2022

Corrosion of steel reinforcement is considered as the main cause of concrete structures deterioration, especially those under humid environmental conditions. Hence, fiber reinforced polymer (FRP) bars are being increasingly used as a replacement for conventional steel owing to their non-corrodible characteristics. However, predicting the shear strength of beams reinforced with FRP bars still challenging due to the lack of robust shear theory. Thus, this paper aims to develop an explicit data driven based model to predict the shear strength of FRP reinforced beams using multi-objective evolutionary polynomial regression analysis (MOGA-EPR) as data driven models learn the behavior from the input data without the need to employee a theory that aid the derivation, and thus they have an enhanced accuracy. This study also evaluates the accuracy of predictive models of shear strength of FRP reinforced concrete beams employed by different design codes by calculating and comparing the values of the mean absolute error (MAE), root mean square error (RMSE), mean (𝜇), standard deviation of the mean (𝜎), coefficient of determination (R²), and percentage of prediction within error range of ±20% (a20-index). Experimental database has been developed and employed in the model learning, validation, and accuracy examination. The statistical analysis illustrated the robustness of the developed model with MAE, RMSE, 𝜇, 𝜎, R², and a20-index of 14.6, 20.8, 1.05, 0.27, 0.85, and 0.61, respectively for training data and 10.4, 14.1, 0.98, 0.25, 0.94, and 0.60, respectively for validation data. Furthermore, the developed model achieved much better predictions than the standard predictive models as it scored lower MAE, RMSE, and 𝜎, and higher R² and a20-index. The new model can be used in future with confidence in optimized designs as its accuracy is higher than standard predictive models.

• Geomechanics and Engineering
• /
• v.22 no.5
• /
• pp.397-414
• /
• 2020

In this paper, practical predictive models for soil shear strength parameters are proposed. As cohesion and internal friction angle are of essential shear strength parameters in any geotechnical studies, we try to predict them via artificial neural network (ANN) and neuro-imperialism approaches. The proposed models was based on the result of a series of consolidated undrained triaxial tests were conducted on reinforced sandy soil. The experimental program surveys the increase in internal friction angle of sandy soil due to addition of polypropylene fibers with different lengths and percentages. According to the result of the experimental study, the most important parameters impact on internal friction angle i.e., fiber percentage, fiber length, deviator stress, and pore water pressure were selected as predictive model inputs. The inputs were used to construct several ANN and neuro-imperialism models and a series of statistical indices were calculated to evaluate the prediction accuracy of the developed models. Both simulation results and the values of computed indices confirm that the newly-proposed neuro-imperialism model performs noticeably better comparing to the proposed ANN model. While neuro-imperialism model has training and test error values of 0.068 and 0.094, respectively, ANN model give error values of 0.083 for training sets and 0.26 for testing sets. Therefore, the neuro-imperialism can provide a new applicable model to effectively predict the internal friction angle of fiber-reinforced sandy soil.

• Fire Science and Engineering
• /
• v.24 no.5
• /
• pp.60-67
• /
• 2010

Fire model shall be verified and validated to reliably predict the consequences of fires within its limitations. Generally the verification and validation procedures are conducted by comparison with experimental test data. This study aims to evaluate predictive capabilities of FDS in the pool fire with two rooms and the sensitivity between input parameters such as heat release rate and ventilation rate and the output values like temperature, concentration, and heat flux. The predictive capabilities of FDS will be evaluated by comparing FDS simulation results with PRISME experimental data which result from the international fire test project. The sensitivity analysis will be conducted to decide which one of input parameters affects outcomes by comparison of FDS results with ${\pm}$ 10% changes of input parameter. From this study, the FDS predictive capabilities are within 20% error range. Heat release rate as input parameter affects most of outcomes and flow rate only has relation with concentration of oxygen and combustion products.

• International Journal of Highway Engineering
• /
• v.16 no.6
• /
• pp.105-113
• /
• 2014

PURPOSES: The objective of this paper is to select the confidential intervals by utilizing the second moment reliability index(Hasofer and Lind; 1974) related to the number of load applications to failure which explains the fatigue failure and rut depth that it indicates the permanent deformation. By using Finite Element Method (FEM) Program, we can easily confirm the rut depth and number of load repetitions without Pavement Design Procedures for generally designing pavement depths. METHODS : In this study, the predictive models for the rut depth and the number of load repetitions to fatigue failure were used for determining the second moment reliability index (${\beta}$). From the case study results using KICTPAVE, the results of the rut depth and the number of load repetitions to fatigue failure were deducted by calculating the empirical predictive equations. Also, the confidential intervals for rut depth and number of load repetitions were selected from the results of the predictive models. To determine the second moment reliability index, the spreadsheet method using Excel's Solver was used. RESULTS : From the case studies about pavement conditions, the results of stress, displacement and strain were different with depth conditions of layers and layer properties. In the clay soil conditions, the values of strain and stresses in the directly loaded sections are relatively greater than other conditions. It indicates that the second moment reliability index is small and confidential intervals for rut depth and the number of load applications are narrow when we apply the clay soil conditions comparing to the applications of other soil conditions. CONCLUSIONS : According to the results of the second moment reliability index and the confidential intervals, the minimum and maximum values of reliability index indicate approximately 1.79 at Case 9 and 2.19 at Case 22. The broadest widths of confidential intervals for rut depth and the number of load repetitions are respectively occurred in Case 9 and Case 7.

• Korean Journal of Clinical Laboratory Science
• /
• v.36 no.2
• /
• pp.144-152
• /
• 2004

With technological advances in automatic hematology analyzers, primary and screening differential counts of white blood cells (WBC) are done with automatic hematology analyzers. They are using different measurement and analysis principles, so differences in WBC differentials and WBC morphology flag exist. This study was carried out to analyze WBC differential counts and WBC morphology flags comparing them with the manual method. Patient EDTA samples in Vacutainer requested for WBC differentials were analyzed with XE-2100. And those samples with suspect flags messages index over 100 were selected and were analyzed with ADVIA-120. Peripheral blood smear film was subsequently made. Three investigators counted 200 cells each (600 cells) in 111 Wright-Giemsa stained blood films. Between two automatic hematology analyzers, neutrophil, lymphocyte, eosinophil, and monocyte showed good correlations, but basophil had moderate correlation. Among automatic hematology analyzers and manual count, neutrophil, lymphocyte, and eosinophil had good correlations, but monocyte had moderate correlation. XE-2100 had higher monocyte, which was due to atypical lymphocyte and myeloblast. LUC in ADVIA-120 was not due to monocyte in XE-2100. Morphology flagging rates were 146.9% in XE-2100 and was 93.2% in ADVIA-120. Positive predictive values of morphology flag were 58.2% in XE-2100 and 54.4% in ADVIA-120. Flags such as atypical lymphocyte, immature granulocyte, and left shift had higher predictive values and those such as N-RBC, platelets clump, and blast had lower ones. Between automatic hematology analyzers, WBC differentials showed good correlations. Predictive values for morphology flags can be variable with changing criteria. Reviewing criteria for WBC differentials and morphology flags should be established in each laboratory with regards to size of laboratory and patients it serves.