• Title/Summary/Keyword: Diagnostic validation

Search Result 168, Processing Time 0.025 seconds

Automatic Detection and Classification of Rib Fractures on Thoracic CT Using Convolutional Neural Network: Accuracy and Feasibility

  • Qing-Qing Zhou;Jiashuo Wang;Wen Tang;Zhang-Chun Hu;Zi-Yi Xia;Xue-Song Li;Rongguo Zhang;Xindao Yin;Bing Zhang;Hong Zhang
    • Korean Journal of Radiology
    • /
    • v.21 no.7
    • /
    • pp.869-879
    • /
    • 2020
  • Objective: To evaluate the performance of a convolutional neural network (CNN) model that can automatically detect and classify rib fractures, and output structured reports from computed tomography (CT) images. Materials and Methods: This study included 1079 patients (median age, 55 years; men, 718) from three hospitals, between January 2011 and January 2019, who were divided into a monocentric training set (n = 876; median age, 55 years; men, 582), five multicenter/multiparameter validation sets (n = 173; median age, 59 years; men, 118) with different slice thicknesses and image pixels, and a normal control set (n = 30; median age, 53 years; men, 18). Three classifications (fresh, healing, and old fracture) combined with fracture location (corresponding CT layers) were detected automatically and delivered in a structured report. Precision, recall, and F1-score were selected as metrics to measure the optimum CNN model. Detection/diagnosis time, precision, and sensitivity were employed to compare the diagnostic efficiency of the structured report and that of experienced radiologists. Results: A total of 25054 annotations (fresh fracture, 10089; healing fracture, 10922; old fracture, 4043) were labelled for training (18584) and validation (6470). The detection efficiency was higher for fresh fractures and healing fractures than for old fractures (F1-scores, 0.849, 0.856, 0.770, respectively, p = 0.023 for each), and the robustness of the model was good in the five multicenter/multiparameter validation sets (all mean F1-scores > 0.8 except validation set 5 [512 x 512 pixels; F1-score = 0.757]). The precision of the five radiologists improved from 80.3% to 91.1%, and the sensitivity increased from 62.4% to 86.3% with artificial intelligence-assisted diagnosis. On average, the diagnosis time of the radiologists was reduced by 73.9 seconds. Conclusion: Our CNN model for automatic rib fracture detection could assist radiologists in improving diagnostic efficiency, reducing diagnosis time and radiologists' workload.

Machine Learning-Based Prediction of COVID-19 Severity and Progression to Critical Illness Using CT Imaging and Clinical Data

  • Subhanik Purkayastha;Yanhe Xiao;Zhicheng Jiao;Rujapa Thepumnoeysuk;Kasey Halsey;Jing Wu;Thi My Linh Tran;Ben Hsieh;Ji Whae Choi;Dongcui Wang;Martin Vallieres;Robin Wang;Scott Collins;Xue Feng;Michael Feldman;Paul J. Zhang;Michael Atalay;Ronnie Sebro;Li Yang;Yong Fan;Wei-hua Liao;Harrison X. Bai
    • Korean Journal of Radiology
    • /
    • v.22 no.7
    • /
    • pp.1213-1224
    • /
    • 2021
  • Objective: To develop a machine learning (ML) pipeline based on radiomics to predict Coronavirus Disease 2019 (COVID-19) severity and the future deterioration to critical illness using CT and clinical variables. Materials and Methods: Clinical data were collected from 981 patients from a multi-institutional international cohort with real-time polymerase chain reaction-confirmed COVID-19. Radiomics features were extracted from chest CT of the patients. The data of the cohort were randomly divided into training, validation, and test sets using a 7:1:2 ratio. A ML pipeline consisting of a model to predict severity and time-to-event model to predict progression to critical illness were trained on radiomics features and clinical variables. The receiver operating characteristic area under the curve (ROC-AUC), concordance index (C-index), and time-dependent ROC-AUC were calculated to determine model performance, which was compared with consensus CT severity scores obtained by visual interpretation by radiologists. Results: Among 981 patients with confirmed COVID-19, 274 patients developed critical illness. Radiomics features and clinical variables resulted in the best performance for the prediction of disease severity with a highest test ROC-AUC of 0.76 compared with 0.70 (0.76 vs. 0.70, p = 0.023) for visual CT severity score and clinical variables. The progression prediction model achieved a test C-index of 0.868 when it was based on the combination of CT radiomics and clinical variables compared with 0.767 when based on CT radiomics features alone (p < 0.001), 0.847 when based on clinical variables alone (p = 0.110), and 0.860 when based on the combination of visual CT severity scores and clinical variables (p = 0.549). Furthermore, the model based on the combination of CT radiomics and clinical variables achieved time-dependent ROC-AUCs of 0.897, 0.933, and 0.927 for the prediction of progression risks at 3, 5 and 7 days, respectively. Conclusion: CT radiomics features combined with clinical variables were predictive of COVID-19 severity and progression to critical illness with fairly high accuracy.

Review of Research Topics on Consumptive Disease and Chronic Fatigue (허로에 대한 국내 연구동향 분석 및 연구방향 제안)

  • Kim, Ji Hye;Kim, Jae Uk;Kim, Keun Ho
    • Journal of Physiology & Pathology in Korean Medicine
    • /
    • v.27 no.5
    • /
    • pp.587-593
    • /
    • 2013
  • Exhaustion syndrome(虛勞) became broadly experienced symptoms in Korean population. In this work, we carried out a systematic literature review on exhaustion syndrome(ES) and chronic fatigue. We searched through the databases Koreanstudies Information Service System, Oriental Medicine Advanced Searching Integrated System, DataBase Periodical Information Academic for the articles published between 1994 and 2013, with the keywords 'exhaustion syndrome(虛勞)', 'consumption(虛損)', 'overexertion syndrome(勞倦)', 'fatigue', 'chronic fatigue' and 'degree of fatigue'. Among the first-run rough-searched 602 articles, we narrowed down the scope within the field of Oriental medicine (126 articles), and finally selected 28 articles appropriate to the intended research field; the selected articles were categorized by literature study(7 papers), clinical treatment (7), clinical diagnosis (5), treatment effects of herbal medicine (2), diagnosis in Sasang medicine and treatment effect of dry cupping therapy (2), and questionnaire-based diagnosis (5). We found that the overall research level on ES remained in the preliminary stages, and more efforts are needed in the field of terminology definition and standardization of diagnosis, and treatment efficacy validation beyond muscle fatigue. Finally, to develop reliable diagnostic devices on ES, we proposed a study design that included the development of objective ES diagnostic indicators and a clinical validation procedure.

Development and Validation of Future Teacher Competency Diagnostic Scale for Pre-service Teachers (예비교사에게 요구되는 미래 교사역량 진단도구 개발 및 타당화)

  • Baek, Jongnam;Kim, Suran
    • Journal of the Korea Convergence Society
    • /
    • v.11 no.2
    • /
    • pp.331-339
    • /
    • 2020
  • The purpose of this study was to develop and validate future teacher competencies diagnosis tools required for pre-service teachers. In this study, the hypothesis model was established by hierarchizing basic competency and job competency in three dimensions such as knowledge, practice, and personality as teachers' competencies required in future society. Based on this hypothesis model, 54 preliminary questions were developed, and competencies diagnosis test was conducted for 237 pre-service teachers in J area, Korea. The results of this study are as follows: First, as a result of this study, a total of 53 questions were extracted, including 18 questions with 6 factors in the knowledge dimension, 17 questions with 6 factors in the practice dimension, and 18 questions with 6 factors in the personality dimension. Second, the goodness-of-fit of future teacher competencies diagnosis model required was verified, and convergence and discriminant validity were verified. The results of this study were discussed. Finally, the implications and suggestions for further research were presented.

Validation of Electrical Impedance Tomography Qualitative and Quantitative Values and Comparison of the Numeric Pain Distress Score against Mammography

  • Juliana, Norsham;Shahar, Suzana;Chelliah, Kanaga Kumari;Ghazali, Ahmad Rohi;Osman, Fazilah;Sahar, Mohd Azmani
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.15 no.14
    • /
    • pp.5759-5765
    • /
    • 2014
  • Electrical impedance tomography (EIT) is a potential supplement for mammogram screening. This study aimed to evaluate and feasibility of EIT as opposed to mammography and to determine pain perception with both imaging methods. Women undergoing screening mammography at the Radiology Department of National University of Malaysia Medical Centre were randomly selected for EIT imaging. All women were requested to give a pain score after each imaging session. Two independent raters were chosen to define the image findings of EIT. A total of 164 women in the age range from 40 to 65-year-old participated and were divided into two groups; normal and abnormal. EIT sensitivity and specificity for rater 1 were 69.4% and 63.3, whereas for rater 2 they were 55.3% and 57.0% respectively. The reliability for each rater ranged between good to very good (p<0.05). Quantitative values of EIT showed there were significant differences in all values between groups (ANCOVA, p<0.05). Interestingly, EIT scored a median pain score of $1.51{\pm}0.75$ whereas mammography scored $4.15{\pm}0.87$ (Mann Whitney U test, p<0.05). From these quantitative values, EIT has the potential as a health discriminating index. Its ability to replace image findings from mammography needs further investigation.

A Validation Study of the Questionnaire of Sasang Constitution Classification(QSCC) (사상체질분류검사(四象體質分類檢査)(QSCC)의 타당화연구(妥當化硏究))

  • Kim, Sun Ho;Ko, Byung-Hee;Song, Il-Byung
    • Journal of Sasang Constitutional Medicine
    • /
    • v.5 no.1
    • /
    • pp.67-85
    • /
    • 1993
  • The Purpose of this study was to evaluate the reliability and the validation of four scales of Questionnaire for Sasang Constitution Classification (QSCC), newly constructed through statistical item analysis and to examine their diagnostic discrimination power. QSCC was administered to 105 inpatient at Kyung-Hee Oriental Medicine Hospital and local oriental clinics and 136 undergraduated students. 2 weeks later, QSCC was readministered to 220 same subjects. Data were collected during about 2 months from february to Apr. 20, 1992. For the purposes of this study, the collected data were analyzed by internal consistancy, test-retest reliability, ANOVA, Pearson correlation and discrimination analysis of spss pc+ v3.0 program. The results were as follows: 1. The reliability of four scales of QSCC was relatively favorable. The internal consistancy and test-retest reliability of Tae-Yaung-In(太陽人) scale were respectively Cronbach's ${\alpha}=0.9$ and r=0.89. Those of So-Yaung-In(少陽人) scale were respectively ${\alpha}=0.81$ and r=0.93. Those of Tae-Em-In(太陰人) scale were respectively ${\alpha}=0.72$ and r=0.74. Those of So-Em-In(少陰人) scale were respectively ${\alpha}=0.81$ and r=0.93. 2. The diagnostic discrimination abilities(Hit-ratio=56%)of QSCC were found to have more about 20% improvement than propotional chance criteria(37%). Especially, Hit-ratios for So-Yaung-In(63%) and Tae-Em-In(60%) were more high than that for So-Em-In(48%) 3. For male subjects, the construct validity of QSCC was founded to be relatively favorable. But that of QSCC for females was poor.

  • PDF

Diagnostic Performance of Deep Learning-Based Lesion Detection Algorithm in CT for Detecting Hepatic Metastasis from Colorectal Cancer

  • Kiwook Kim;Sungwon Kim;Kyunghwa Han;Heejin Bae;Jaeseung Shin;Joon Seok Lim
    • Korean Journal of Radiology
    • /
    • v.22 no.6
    • /
    • pp.912-921
    • /
    • 2021
  • Objective: To compare the performance of the deep learning-based lesion detection algorithm (DLLD) in detecting liver metastasis with that of radiologists. Materials and Methods: This clinical retrospective study used 4386-slice computed tomography (CT) images and labels from a training cohort (502 patients with colorectal cancer [CRC] from November 2005 to December 2010) to train the DLLD for detecting liver metastasis, and used CT images of a validation cohort (40 patients with 99 liver metastatic lesions and 45 patients without liver metastasis from January 2011 to December 2011) for comparing the performance of the DLLD with that of readers (three abdominal radiologists and three radiology residents). For per-lesion binary classification, the sensitivity and false positives per patient were measured. Results: A total of 85 patients with CRC were included in the validation cohort. In the comparison based on per-lesion binary classification, the sensitivity of DLLD (81.82%, [81/99]) was comparable to that of abdominal radiologists (80.81%, p = 0.80) and radiology residents (79.46%, p = 0.57). However, the false positives per patient with DLLD (1.330) was higher than that of abdominal radiologists (0.357, p < 0.001) and radiology residents (0.667, p < 0.001). Conclusion: DLLD showed a sensitivity comparable to that of radiologists when detecting liver metastasis in patients initially diagnosed with CRC. However, the false positives of DLLD were higher than those of radiologists. Therefore, DLLD could serve as an assistant tool for detecting liver metastasis instead of a standalone diagnostic tool.

A Validation Study of the Sasang Constitution Questionnaire for Japanese(SSCQ-J) (일본인용 사상체질진단지의 타당화 연구)

  • Jo, Hoon-Seuk;Jeon, Soo-Hyung;Jeong, Jong-Hun;Kim, Kyu-Kon;Kim, Jong-Won
    • Journal of Sasang Constitutional Medicine
    • /
    • v.25 no.4
    • /
    • pp.289-296
    • /
    • 2013
  • Objectives This study was aimed to validate Sasang Constitution Questionnaire for Japanese (SSCQ-J). Methods Sasang Constitution Questionnaire for Patients (SSCQ-P) was developed by joint researches between the Society of Sasang Constitutional Medicine and Korea Institute of Oriental Medicine. We translated SSCQ-P into Japanese and modified some items of that for Japanese. By getting approval from the Institutional Review Board(IRB)of School of Medicine, Keio University, we conducted a questionnaire survey of patients who visited Oriental Medicine Center from early January until mid-February 2011. The total of 364 patients filled out that Questionnaire and gave an interview with a Sasang constitution specialist. Using this Questionnaire data, we made Sasang constitutional classification functions and calculated diagnostic accuracy rate of SSCQ-J using discrimination analysis. Results 1. Male group's diagnostic accuracy rate of SSCQ-J was 77.01% and female was 78.10%. 2. Diagnostic accuracy of SSCQ-J was a little higher than SSCQ-P Conclusions 1. SSCQ-J can be considered to have good discriminant power compared with SSCQ-P 2. Further research with SSCQ-J will be helpful in the comparison study on the usual symptoms between Korean and Japanese as well as development of good discriminant function.

Hindi version of short form of douleur neuropathique 4 (S-DN4) questionnaire for assessment of neuropathic pain component: a cross-cultural validation study

  • Gudala, Kapil;Ghai, Babita;Bansal, Dipika
    • The Korean Journal of Pain
    • /
    • v.30 no.3
    • /
    • pp.197-206
    • /
    • 2017
  • Background: Pain with neuropathic characteristics is generally more severe and associated with a lower quality of life compared to nociceptive pain (NcP). Short form of the Douleur Neuropathique en 4 Questions (S-DN4) is one of the most used and reliable screening questionnaires and is reported to have good diagnostic properties. This study was aimed to cross-culturally validate the Hindi version of the S-DN4 in patients with various chronic pain conditions. Methods: The S-DN4 is already translated into the Hindi language by Mapi Research Trust. This study assessed the psychometric properties of the Hindi version of the S-DN4 including internal consistency and test-retest reliability after 3 days' post-baseline assessment. Diagnostic performance was also assessed. Results: One hundred sixty patients with chronic pain, 80 each in the neuropathic pain (NeP) present and NeP absent groups, were recruited. Patients with NeP present reported significantly higher S-DN4 scores in comparison to patients in the NeP absent group (mean (SD), 4.7 (1.7) vs. 1.8 (1.6), P < 0.01). The S-DN4 was found to have an AUC of 0.88 with adequate internal consistency (Cronbach's ${\alpha}=0.80$) and a test-retest reliability (ICC = 0.92) with an optimal cut-off value of 3 (Youden's index = 0.66, sensitivity and specificity of 88.7% and 77.5%). The diagnostic concordance rate between clinician diagnosis and the S-DN4 questionnaire was 83.1% (kappa = 0.66). Conclusions: Overall, the Hindi version of the S-DN4 has good internal consistency and test-retest reliability along with good diagnostic accuracy.

Second Opinion Diagnoses of Cytologic Specimens on Consultation : Asan Medical Center Experience (세포 병리 자문 재판독의 진단 불일치에 관한 연구: 서울아산병원의 경험)

  • Park, So-Hyung;Ro, Jae-Y.;Cho, Kyung-Ja;Gong, Gyung-Yub;Cho, Yong-Mee;Khang, Shin-Kwang
    • The Korean Journal of Cytopathology
    • /
    • v.19 no.2
    • /
    • pp.99-106
    • /
    • 2008
  • Background : Second opinion diagnosis of outside pathology slides is a common practice for efficient and proper patient management. We analyzed cytology slides from outside hospitals submitted for a second opinion diagnosis to determine whether the second opinion diagnosis had any influence on patient care. Methods : We reviewed 1,153 outside cytology slides referred to Asan Medical Center for second opinions from January, 2007, to December, 2007. All cases were categorized into three groups; no diagnostic discrepancy, minor diagnostic discrepancies (no impact on the management), and major diagnostic discrepancies (significant impact on the management and subsequent follow-up). Results : The thyroid was the most common organ system (933 cases, 80.9%), Forty cases (3.6%) belonged to the major diagnostic discrepancy group and 149 cases (12.8%) to the minor discrepancy group. For validation of second opinion diagnoses in major discrepancy cases, subsequent biopsy or surgical resection specimens and clinical information were reviewed, which were available in 29 cases. The second opinion diagnoses resulted in alteration of clinical management in 21 of 29 cases. Conclusion : For all referred patients, second opinion diagnosis is important and mandatory for appropriate patient care.