• Communications for Statistical Applications and Methods
• /
• 제31권2호
• /
• pp.203-212
• /
• 2024

The area under the ROC curve (AUC) is one of the most common criteria used to measure the overall performance of binary classifiers for a wide range of machine learning problems. In this article, we propose a L₁-penalized AUC-optimization classifier that directly maximizes the AUC for high-dimensional data. Toward this, we employ the AUC-consistent surrogate loss function and combine the L₁-norm penalty which enables us to estimate coefficients and select informative variables simultaneously. In addition, we develop an efficient optimization algorithm by adopting k-means clustering and proximal gradient descent which enjoys computational advantages to obtain solutions for the proposed method. Numerical simulation studies demonstrate that the proposed method shows promising performance in terms of prediction accuracy, variable selectivity, and computational costs.

• 지역사회간호학회지
• /
• 제22권3호
• /
• pp.302-314
• /
• 2011

Purpose: This study aimed to validate instruments to classify the frailty of Korean elderly people in community. Methods: For this study, 632 elders were selected from community-based elderly houses and home visiting registries, and data on frailty were collected using three instruments during November, 2008. The Korean Frail Scale (KFS) was composed of 10 domains with the maximum score of 20. The Edmonton Frail Scale (EFS) had 10 domains with the maximum score of 17. The 25_Japan Frail Scale (25_JFS) was composed of 6 domains with the maximum score of 25. Internal consistency was measured with Cronbach's ${\alpha}$. Sensitivity, specificity and area under the curve (AUC) of ROC were measured to see validity with long.term care insurance grade as a gold standard. Results: The Cronbach's ${\alpha}$ was .72 for KFS, .55 for EFS, and .80 for 25_JFS. Sensitivity, specificity, and AUC were 70.0%, 83.2%, and .83, respectively, at cutting point 10.5 for the KFS, 50.0%, 80.9%, and .66, respectively, at 8.5 for EFS, and 80.0%, 85.9%, and .86, respectively, at 12.5 for 25_JFS. Conclusion: KFS and three JFS showed favorable internal consistency and predictive validity. Further longitudinal studies are recommended to confirm predictive validity.

• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2023년도 추계학술발표대회
• /
• pp.461-464
• /
• 2023

최근 GPT-3 와 LLaMa 같은 생성형 거대 언어모델을 활용한 서비스가 공개되었고, 실제로 많은 사람들이 사용하고 있다. 해당 모델들은 사용자들의 다양한 질문에 대해 유창한 답변을 한다는 이유로 주목받고 있다. 하지만 LLMs 의 답변에는 종종 Inconsistent content 와 non-factual statement 가 존재하며, 이는 사용자들로 하여금 잘못된 정보의 전파 등의 문제를 야기할 수 있다. 이에 논문에서는 동일한 질문에 대한 LLM 의 답변 샘플과 외부 지식을 활용한 Hallucination Detection 방법을 제안한다. 제안한 방법은 동일한 질문에 대한 LLM 의 답변들을 이용해 일관성 점수(Consistency score)를 계산한다. 거기에 외부 지식을 이용한 사실검증을 통해 사실성 점수(Factuality score)를 계산한다. 계산된 일관성 점수와 사실성 점수를 활용하여 문장 수준의 Hallucination Detection 을 가능하게 했다. 실험에는 GPT-3 를 이용하여 WikiBio dataset 에 있는 인물에 대한 passage 를 생성한 데이터셋을 사용하였으며, 우리는 해당 방법을 통해 문장 수준에서의 Hallucination Detection 성능이 baseline 보다 AUC-PR scores 에서 향상됨을 보였다.

• The Korean Journal of Pain
• /
• 제30권3호
• /
• pp.197-206
• /
• 2017

Background: Pain with neuropathic characteristics is generally more severe and associated with a lower quality of life compared to nociceptive pain (NcP). Short form of the Douleur Neuropathique en 4 Questions (S-DN4) is one of the most used and reliable screening questionnaires and is reported to have good diagnostic properties. This study was aimed to cross-culturally validate the Hindi version of the S-DN4 in patients with various chronic pain conditions. Methods: The S-DN4 is already translated into the Hindi language by Mapi Research Trust. This study assessed the psychometric properties of the Hindi version of the S-DN4 including internal consistency and test-retest reliability after 3 days' post-baseline assessment. Diagnostic performance was also assessed. Results: One hundred sixty patients with chronic pain, 80 each in the neuropathic pain (NeP) present and NeP absent groups, were recruited. Patients with NeP present reported significantly higher S-DN4 scores in comparison to patients in the NeP absent group (mean (SD), 4.7 (1.7) vs. 1.8 (1.6), P < 0.01). The S-DN4 was found to have an AUC of 0.88 with adequate internal consistency (Cronbach's ${\alpha}=0.80$) and a test-retest reliability (ICC = 0.92) with an optimal cut-off value of 3 (Youden's index = 0.66, sensitivity and specificity of 88.7% and 77.5%). The diagnostic concordance rate between clinician diagnosis and the S-DN4 questionnaire was 83.1% (kappa = 0.66). Conclusions: Overall, the Hindi version of the S-DN4 has good internal consistency and test-retest reliability along with good diagnostic accuracy.

• 대한간호학회지
• /
• 제46권1호
• /
• pp.149-158
• /
• 2016

Purpose: This study was done to develop of the Korean intensive care delirium screening tool (KICDST). Methods: The KICDST was developed in 5 steps: Configuration of conceptual frame, development of preliminary tool, pilot study, reliability and validity test, development of final KICDST. Reliability tests were done using degree of agreement between evaluators and internal consistency. For validity tests, CVI (Content Validity Index), ROC (Receiver Operating Characteristics) analysis, known group technique and factor analysis were used. Results: In the reliability test, the degree of agreement between evaluators showed .80~1.00 and the internal consistency was KR-20=.84. The CVI was .83~1.00. In ROC analysis, the AUC (Area Under the ROC Curve) was .98. Assessment score was 4 points. The values for sensitivity, specificity, correct classification rate, positive predictive value, and negative predictive value were found to be 95.0%, 93.7%, 94.4%, 95.0% and 93.7%, respectively. In the known group technique, the average delirium screening tool score of the non-delirium group was $1.25{\pm}0.99$ while that of delirium group was $5.07{\pm}1.89$ (t= - 16.33, p <.001). The factors were classified into 3 factors (cognitive change, symptom fluctuation, psychomotor retardation), which explained 67.4% of total variance. Conclusion: Findings show that the KICDST has high sensitivity and specificity. Therefore, this screening tool is recommended for early identification of delirium in intensive care patients.

• 한국산업정보학회논문지
• /
• 제20권3호
• /
• pp.61-69
• /
• 2015

본 논문에서는 이산 웨이블릿 변환을 통해 추출된 시간 영역과 주파수 영역의 특징들을 활용하여 심박수변이도를 확률적인 지식으로 분석할 수 있는 방법을 제안하였다. 제안된 방법에서 지식획득 알고리즘은 규칙생성과 규칙평가 단계로 구성되어 있으며, 규칙생성에서는 ROC 분석을 통해 수치적인 속성값을 이산화된 구간으로 변환하고, 서로 다른 의사결정값을 포함하는 구간들 사이에 일관성 정도를 비교함으로써 감축된 규칙-집합을 생성한다. 이때 규칙-집합 내에 각 규칙에 대해서 확률적 해석을 위한 3가지 척도를 추정하였다. 제안된 모형의 효과성은 심혈관질환 병력을 가진 58명의 심전도 데이터로부터 심방세동을 식별할 수 있는 5가지 규칙을 생성하였고, 이들 규칙의 분별력을 평가하였다. 실험결과, 제안된 모형으로부터 생성된 지식은 4가지 성능평가 척도에 대해서 각각 93%의 정확도를 보여주었다.