• Title/Summary/Keyword: weighted kappa

Search Result 89, Processing Time 0.034 seconds

Interpretation of Complete Tumor Response on MRI Following Chemoradiotherapy of Rectal Cancer: Inter-Reader Agreement and Associated Factors in Multi-Center Clinical Practice

  • Hae Young Kim;Seung Hyun Cho;Jong Keon Jang;Bohyun Kim;Chul-min Lee;Joon Seok Lim;Sung Kyoung Moon;Soon Nam Oh;Nieun Seo;Seong Ho Park
    • Korean Journal of Radiology
    • /
    • v.25 no.4
    • /
    • pp.351-362
    • /
    • 2024
  • Objective: To measure inter-reader agreement and identify associated factors in interpreting complete response (CR) on magnetic resonance imaging (MRI) following chemoradiotherapy (CRT) for rectal cancer. Materials and Methods: This retrospective study involved 10 readers from seven hospitals with experience of 80-10210 cases, and 149 patients who underwent surgery after CRT for rectal cancer. Using MRI-based tumor regression grading (mrTRG) and methods employed in daily practice, the readers independently assessed mrTRG, CR on T2-weighted images (T2WI) denoted as mrCRT2W, and CR on all images including diffusion-weighted images (DWI) denoted as mrCRoverall. The readers described their interpretation patterns and how they utilized DWI. Inter-reader agreement was measured using multi-rater kappa, and associated factors were analyzed using multivariable regression. Correlation between sensitivity and specificity of each reader was analyzed using Spearman coefficient. Results: The mrCRT2W and mrCRoverall rates varied widely among the readers, ranging 18.8%-40.3% and 18.1%-34.9%, respectively. Nine readers used DWI as a supplement sequence, which modified interpretations on T2WI in 2.7% of cases (36/1341 [149 patients × 9 readers]) and mostly (33/36) changed mrCRT2W to non-mrCRoverall. The kappa values for mrTRG, mrCRT2W, and mrCRoverall were 0.56 (95% confidence interval: 0.49, 0.62), 0.55 (0.52, 0.57), and 0.54 (0.51, 0.57), respectively. No use of rectal gel, larger initial tumor size, and higher initial cT stage exhibited significant association with a higher interreader agreement for assessing mrCRoverall (P ≤ 0.042). Strong negative correlations were observed between the sensitivity and specificity of individual readers (coefficient, -0.718 to -0.963; P ≤ 0.019). Conclusion: Inter-reader agreement was moderate for assessing CR on post-CRT MRI. Readers' varying standards on MRI interpretation (i.e., threshold effect), along with the use of rectal gel, initial tumor size, and initial cT stage, were significant factors associated with inter-reader agreement.

Diagnostic Performance of Simulated Abbreviated MRI for Early-Stage Hepatocellular Carcinoma Screening: A Comparison to Conventional Dynamic Contrast-Enhanced MRI (초기 간암 선별 검사로서 단축 자기공명영상 검사의 진단능: 고식적 역동학적 자기공명영상검사와의 비교)

  • Eun Sol Lim;Sung Mo Kim;Sang Soo Shin;Suk Hee Heo;Jong Eun Lee;Yong Yeon Jeong
    • Journal of the Korean Society of Radiology
    • /
    • v.82 no.5
    • /
    • pp.1218-1230
    • /
    • 2021
  • Purpose To compare the per-patient diagnostic performance of simulated abbreviated MRI (AMRI) to that of conventional MRI (CMRI) with full-sequence dynamic gadoxetic acid (GA) enhancement for early-stage hepatocellular carcinoma (HCC) screening in high-risk patients. Materials and Methods A total of 201 consecutive patients at high-risk for HCC, who underwent 3T liver MRI, were included in this retrospective study. The AMRI protocol comprised T2-weighted imaging, hepatobiliary phase imaging after GA injection, and diffusion-weighted imaging. For each patient, two AMRI and CMRI image sets were independently reviewed by two radiologists. Inter-reader agreement was assessed using Cohen's kappa value. A composite reference standard was used to determine the diagnostic performance of each image set for each reader. Results A total of 93 HCCs were detected in 79 patients. The inter-reader agreement was almost perfect for both image sets (κ = 0.839, 0.948). In AMRI, the per-patient sensitivity and negative predictive values (NPV) were 94.9% and 96.4%, respectively. In CMRI, the per-patient sensitivity and NPV were 96.2% and 97.5%, respectively. Conclusion AMRI, using only three sequences, had a comparable diagnostic performance to CMRI in screening early-stage HCC. AMRI could be an alternative HCC screening tool for high-risk HCC patients.

Comparison of the reproducibility of results of a new peri-implantitis assessment system (implant success index) with the Misch classification

  • Abrishami, Mohammad Reza;Sabour, Siamak;Nasiri, Maryam;Amid, Reza;Kadkhodazadeh, Mahdi
    • Journal of the Korean Association of Oral and Maxillofacial Surgeons
    • /
    • v.40 no.2
    • /
    • pp.61-67
    • /
    • 2014
  • Objectives: The present study was conducted to determine the reproducibility of peri-implant tissue assessment using the new implant success index (ISI) in comparison with the Misch classification. Materials and Methods: In this descriptive study, 22 cases of peri-implant soft tissue with different conditions were selected, and color slides were prepared from them. The slides were shown to periodontists, maxillofacial surgeons, prosthodontists and general dentists, and these professionals were asked to score the images according to the Misch classification and ISI. The intra- and inter-observer reproducibility scores of the viewers were assessed and reported using kappa and weighted kappa (WK) tests. Results: Inter-observer reproducibility of the ISI technique between the prosthodontists-periodontists (WK=0.85), prosthodontists-maxillofacial surgeons (WK=0.86) and periodontists-maxillofacial surgeons (WK=0.9) was better than that between general dentists and other specialists. In the two groups of general dentists and maxillofacial surgeons, ISI was more reproducible than the Misch classification system (WK=0.99 versus WK non-calculable, WK=1 and WK=0.86). The intra-observer reproducibility of both methods was equally excellent among periodontists (WK=1). For prosthodontists, the WK was not calculable via any of the methods. Conclusion: The intra-observer reproducibility of both the ISI and Misch classification techniques depends on the specialty and expertise of the clinician. Although ISI has more classes, it also has higher reproducibility than simpler classifications due to its ability to provide more detail.

A Computer Algorithm for the evaluation of elements in Face Stimulus Assessment (얼굴자극검사의 평가를 위한 컴퓨터 알고리즘)

  • Kim, Jong-Hoon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.11 no.6
    • /
    • pp.1961-1968
    • /
    • 2010
  • The Face Stimulus Assessment is an efficient projective drawing test developed by Betts. This paper categorizes scales which Betts suggested into the following five groups: accuracy of painting, color fit, perception of shape, precision of drawing, and space usage. In this paper, a computer algorithm which objectively evaluates these five scales is suggested. The proposed algorithm defines the areas of the lip, eyes, hair, etc. which take on significant roles in the evaluation of the FSA and based on these factors, it calculates the grade of each scale through the main color and color ratio. The consistency of evaluations between the computer algorithm and the art therapist is measured by the Quadratic Weighted Kappa. By providing objectivity and consistency, the computer algorithm is expected to solve the problem of uncertainty found in art therapists' evaluations of projective drawing tests caused by their subjective judgment, experience, and intuition.

Deep learning for the classification of cervical maturation degree and pubertal growth spurts: A pilot study

  • Mohammad-Rahimi, Hossein;Motamadian, Saeed Reza;Nadimi, Mohadeseh;Hassanzadeh-Samani, Sahel;Minabi, Mohammad A. S.;Mahmoudinia, Erfan;Lee, Victor Y.;Rohban, Mohammad Hossein
    • The korean journal of orthodontics
    • /
    • v.52 no.2
    • /
    • pp.112-122
    • /
    • 2022
  • Objective: This study aimed to present and evaluate a new deep learning model for determining cervical vertebral maturation (CVM) degree and growth spurts by analyzing lateral cephalometric radiographs. Methods: The study sample included 890 cephalograms. The images were classified into six cervical stages independently by two orthodontists. The images were also categorized into three degrees on the basis of the growth spurt: pre-pubertal, growth spurt, and post-pubertal. Subsequently, the samples were fed to a transfer learning model implemented using the Python programming language and PyTorch library. In the last step, the test set of cephalograms was randomly coded and provided to two new orthodontists in order to compare their diagnosis to the artificial intelligence (AI) model's performance using weighted kappa and Cohen's kappa statistical analyses. Results: The model's validation and test accuracy for the six-class CVM diagnosis were 62.63% and 61.62%, respectively. Moreover, the model's validation and test accuracy for the three-class classification were 75.76% and 82.83%, respectively. Furthermore, substantial agreements were observed between the two orthodontists as well as one of them and the AI model. Conclusions: The newly developed AI model had reasonable accuracy in detecting the CVM stage and high reliability in detecting the pubertal stage. However, its accuracy was still less than that of human observers. With further improvements in data quality, this model should be able to provide practical assistance to practicing dentists in the future.

Comparing Korea Occupational Safety & Health Agency and National Health Insurance Service's cardio-cerebrovascular diseases risk-assessment tools using data from one hospital's health checkups

  • Yunrae Cho;Dong Geon Kim;Byung-Chan Park;Seonhee Yang;Sang Kyu Kim
    • Annals of Occupational and Environmental Medicine
    • /
    • v.35
    • /
    • pp.35.1-35.11
    • /
    • 2023
  • Background: Cardio-cerebrovascular diseases (CVDs) are the most common cause of death worldwide. Various CVD risk assessment tools have been developed. In South Korea, the Korea Occupational Safety & Health Agency (KOSHA) and the National Health Insurance Service (NHIS) have provided CVD risk assessments with health checkups. Since 2018, the KOSHA guide has stated that NHIS CVD risk assessment tool could be used as an alternative of KOSHA assessment tool for evaluating CVD risk of workers. The objective of this study was to determine the correlation and agreement between the KOSHA and the NHIS CVD risk assessment tools. Methods: Subjects of this study were 17,485 examinees aged 20 to 64 years who had undergone medical examinations from January 2021 to December 2021 at a general hospital. We classified subjects into low-risk, moderate-risk, high-risk, and highest-risk groups according to KOSHA and NHIS's CVD risk assessment tools. We then compared them with cross-analysis, Spearman correlation analysis, and linearly weighted kappa coefficient. Results: The correlation between KOSHA and NHIS tools was statistically significant (p-value < 0.001), with a correlation coefficient of 0.403 and a kappa coefficient of 0.203. When we compared risk group distribution using KOSHA and NHIS tools, CVD risk of 6,498 (37.1%) participants showed a concordance. Compared to the NHIS tool, the KOSHA tool classified 9,908 (56.7%) participants into a lower risk category and 1,079 (6.2%) participants into a higher risk category. Conclusions: In this study, KOSHA and NHIS tools showed a moderate correlation with a fair agreement. The NHIS tool showed a tendency to classify participants to higher CVD risk group than the KOSHA tool. To prevent CVD more effectively, a higher estimation tool among verified CVD risk assessment methods should be selected and managements such as early intervention and treatment of risk factors should be performed targeting the high-risk group.

MR diagnosis of cranial neuritis focusing on facial neuritis: Performance of contrast-enhanced 3D-FLAIR technique

  • Lee, Ho Kyu;Koh, Myeong Ju;Kim, Seung Hyoung;Oh, Jung-Hwan
    • Journal of Medicine and Life Science
    • /
    • v.16 no.1
    • /
    • pp.1-5
    • /
    • 2019
  • Our purpose was to evaluate usefulness of the contrast-enhanced 3 dimensional fluid attenuated inversion recovery (3D-FLAIR) technique of half brain volume to diagnose the patients with facial neuritis based on segment-based analysis. We assessed retrospectively 17 consecutive patients who underwent brain MR imaging at 3 tesla for facial neuritis: 11 patients with idiopathic facial neuritis and 6 with herpes zoster oticus. Contrast enhanced 3D-FLAIR sequences of the half brain volume were analyzed and 3D T1-weighted sequence of the full brain volume were used as the base-line exam. Enhancement of the facial nerve was determined in each segment of 5 facial nerve segments by two radiologists. Sensitivity, specificity and accuracy of enhancement of each segment were assessed. The authors experienced a prompt fuzzy CSF enhancement in the fundus of the internal auditory canal in patients with enhancement of the canalicular segment. Interobserver agreement of CE 3D-FLAIR was excellent(${\kappa}$-value 0.885). Sensitivity, specificity, and accuracy of each segment are 1.0, 0.823, 0.912 in the canalicular segment; 0.118, 1.0, 0.559 in the labyrinthine segment; 0.823, 0.294, 0.559 in the anterior genu; 0.823, 0.529, 0.676 in the tympanic segment; 0.823, 0.235, 0.529 in the mastoid segment, respectively. In addition, those of prompt fuzzy enhancement were 0.647, 1.0, and 0.824, respectively. Incidence of prompt fuzzy enhancement with enhancement of the canalicular segment was 11 sites(55%): 6 (54.5%) in idiopathic facial neuritis and 5 (83.3%) in herpes zoster. Enhancement of the canalicular segment and prompt fuzzy enhancement on CE 3D-FLAIR was significantly correlated with occurrence of facial neuritis (p<0.001). CE 3D-FLAIR technique of the half brain volume is useful to evaluate the patients with facial neuritis as an adjunct sequence in addition to contrast-enhanced 3D T1-weighted sequence. On segment-based analysis, contrast enhancement of the canalicular segment is the most reliable. Prompt fuzzy enhancement is seen in not only herpes zoster, but in idiopathic facial neuritis.

Imaging Patterns of Bacillus Calmette-Guérin-Related Granulomatous Prostatitis Based on Multiparametric MRI

  • Seungsoo Lee;Young Taik Oh;Hye Min Kim;Dae Chul Jung;Hyesuk Hong
    • Korean Journal of Radiology
    • /
    • v.23 no.1
    • /
    • pp.60-67
    • /
    • 2022
  • Objective: To categorize multiparametric MRI features of Bacillus Calmette-Guérin (BCG)-related granulomatous prostatitis (GP) and discover potential manifestations for its differential diagnosis from prostate cancer. Materials and Methods: The cases of BCG-related GP in 24 male (mean age ± standard deviation, 66.0 ± 9.4 years; range, 50-88 years) pathologically confirmed between January 2011 and April 2019 were retrospectively reviewed. All patients underwent intravesical BCG therapy followed by a MRI scan. Additional follow-up MRI scans, including diffusion-weighted imaging (DWI), were performed in 19 patients. The BCG-related GP cases were categorized into three: A, B, or C. The lesions with diffusion restriction and homogeneous enhancement were classified as type A. The lesions with diffusion restriction and a poorly enhancing component were classified as type B. A low signal intensity on high b-value DWI (b = 1000 s/mm2) was considered characteristic of type C. Two radiologists independently interpreted the MRI scans before making a consensus about the types. Results: The median lesion size was 22 mm with the interquartile range (IQR) of 18-26 mm as measured using the initial MRI scans. The lesion types were A, B, and C in 7, 15, and 2 patients, respectively. Cohen's kappa value for the inter-reader agreement for the interpretation of the lesion types was 0.837. On the last follow-up MRI scans of 19 patients, the size decreased (median, 5.8 mm; IQR, 3.4-8.5 mm), and the type changed from A or B to C in 11 patients. The lesions resolved in four patients. In five patients who underwent prostatectomy, caseous necrosis on histopathology matched with the non-enhancing components of type B lesions and the entire type C lesions. Conclusion: BCG-related GP demonstrated three imaging patterns on multiparametric MRI. Contrast-enhanced T1-weighted imaging and DWI may play a role in its differential diagnosis from prostate cancer.

Diagnostic Performance of Rectal CT for Staging Rectal Cancer: Comparison with Rectal MRI and Histopathology (직장암 병기결정에서 직장 CT의 진단능: 직장 MRI 및 병리결과와의 비교분석)

  • Seok Yoon Son;Yun Seok Seo;Jeong Hee Yoon;Bo Yun Hur;Jae Seok Bae;Se Hyung Kim
    • Journal of the Korean Society of Radiology
    • /
    • v.84 no.6
    • /
    • pp.1290-1308
    • /
    • 2023
  • Purpose To compare the diagnostic performance of rectal CT with that of high-resolution rectal MRI and histopathology in assessing rectal cancer. Materials and Methods Sixty-seven patients with rectal cancer who underwent rectal CT with rectal distension using sonographic gel and high-resolution MRI were enrolled in this study. The distance from the anal verge/anorectal junction, distance to the mesorectal fascia (MRF), extramural depth (EMD), extramesorectal lymph node (LN) involvement, extramural venous invasion (EMVI), and T/N stages in rectal CT/MRI were analyzed by two gastrointestinal radiologists. The CT findings of 20 patients who underwent radical surgery without concurrent chemoradiotherapy were compared using histopathology. Interclass correlations and kappa statistics were used. Results The distance from the anal verge/anorectal junction showed an excellent intraclass correlation between CT and MRI for both reviewers. For EMD, the distance to the MRF, presence of LNs, extramesorectal LN metastasis, EMVI, T stage, and intermodality kappa or weighted kappa values between CT and MRI showed excellent agreement. Among the 20 patients who underwent radical surgery, T staging, circumferential resection margin involvement, EMVI, and LN metastasis on rectal CT showed acceptable concordance rates with histopathology. Conclusion Dedicated rectal CT may be on par with rectal MRI in providing critical information to patients with rectal cancer.

Multi-Objective Optimization of a Fan Blade Using NSGA-II (NSGA-II 를 통한 송풍기 블레이드의 다중목적함수 최적화)

  • Lee, Ki-Sang;Kim, Kwang-Yong;Samad, Abdus
    • Proceedings of the KSME Conference
    • /
    • 2007.05b
    • /
    • pp.2690-2695
    • /
    • 2007
  • This work presents numerical optimization for design of a blade stacking line of a low speed axial flow fan with a fast and elitist Non-Dominated Sorting of Genetic Algorithm (NSGA-II) of multi-objective optimization using three-dimensional Navier-Stokes analysis. Reynolds-averaged Navier-Stokes (RANS) equations with ${\kappa}-{\varepsilon}$ turbulence model are discretized with finite volume approximations and solved on unstructured grids. Regression analysis is performed to get second order polynomial response which is used to generate Pareto optimal front with help of NSGA-II and local search strategy with weighted sum approach to refine the result obtained by NSGA-II to get better Pareto optimal front. Four geometric variables related to spanwise distributions of sweep and lean of blade stacking line are chosen as design variables to find higher performed fan blade. The performance is measured in terms of the objectives; total efficiency, total pressure and torque. Hence the motive of the optimization is to enhance total efficiency and total pressure and to reduce torque.

  • PDF