• Title/Summary/Keyword: 한글문서 정보

Search Result 572, Processing Time 0.047 seconds

Developing a New Algorithm for Conversational Agent to Detect Recognition Error and Neologism Meaning: Utilizing Korean Syllable-based Word Similarity (대화형 에이전트 인식오류 및 신조어 탐지를 위한 알고리즘 개발: 한글 음절 분리 기반의 단어 유사도 활용)

  • Jung-Won Lee;Il Im
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.3
    • /
    • pp.267-286
    • /
    • 2023
  • The conversational agents such as AI speakers utilize voice conversation for human-computer interaction. Voice recognition errors often occur in conversational situations. Recognition errors in user utterance records can be categorized into two types. The first type is misrecognition errors, where the agent fails to recognize the user's speech entirely. The second type is misinterpretation errors, where the user's speech is recognized and services are provided, but the interpretation differs from the user's intention. Among these, misinterpretation errors require separate error detection as they are recorded as successful service interactions. In this study, various text separation methods were applied to detect misinterpretation. For each of these text separation methods, the similarity of consecutive speech pairs using word embedding and document embedding techniques, which convert words and documents into vectors. This approach goes beyond simple word-based similarity calculation to explore a new method for detecting misinterpretation errors. The research method involved utilizing real user utterance records to train and develop a detection model by applying patterns of misinterpretation error causes. The results revealed that the most significant analysis result was obtained through initial consonant extraction for detecting misinterpretation errors caused by the use of unregistered neologisms. Through comparison with other separation methods, different error types could be observed. This study has two main implications. First, for misinterpretation errors that are difficult to detect due to lack of recognition, the study proposed diverse text separation methods and found a novel method that improved performance remarkably. Second, if this is applied to conversational agents or voice recognition services requiring neologism detection, patterns of errors occurring from the voice recognition stage can be specified. The study proposed and verified that even if not categorized as errors, services can be provided according to user-desired results.

Package Design Proposal to Improve Medical Information Transmission on Regular Medicine Sold in Convenience Stores (편의점 판매용 안전상비의약품의 정보 전달 개선을 위한 포장디자인 제안)

  • Kim, Min Ji;Jung, Eui-Tay;Paik, JinKyung
    • Design Convergence Study
    • /
    • v.15 no.4
    • /
    • pp.17-29
    • /
    • 2016
  • Over-the-counter pharmaceuticals sold in convenience stores are accurate and easy to understand on its containers, packaging, or attached materials for the prevention of drug misuse. The limited space of the packaging, however, makes the information difficult to read and consequently, consumers have had difficulty in purchasing the products. This research conducted an analysis of 8 packages of pill-type drugs and conducted a survey and interviews targeting 10 foreigners and 20 Korean consumers to analyze the user consciousness. The findings of this study showed that fever remedies are the most frequently bought drug. And consumers first looked for the information of the drugs in the following order: the effects, dosage, and warnings. While domestic consumers had little difficulty in comprehending the information, more than 40% of the foreigners who were studies failed to comprehend it, which was because the package was entirely written in Korean. This study presented some ideas for improving the design by using English and foreign expressions and pictograms, consumers can easily and quickly understand, and by placing the information on the symptoms, dosage and age information so that the type of drugs will be easily discernible.

A Study on Word Learning and Error Type for Character Correction in Hangul Character Recognition (한글 문자 인식에서의 오인식 문자 교정을 위한 단어 학습과 오류 형태에 관한 연구)

  • Lee, Byeong-Hui;Kim, Tae-Gyun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.3 no.5
    • /
    • pp.1273-1280
    • /
    • 1996
  • In order perform high accuracy recognition of text recognition systems, the recognized text must be processed through a post-processing stage using contextual information. We present a system that combines multiple knowledge sources to post-process the output of an optical character recognition(OCR) system. The multiple knowledge sources include characteristics of word, wrongly recognized types of Hangul characters, and Hangul word learning In this paper, the wrongly recognized characters which are made by OCR systems are collected and analyzed. We imput a Korean dictionary with approximately 15 0,000 words, and Korean language texts of Korean elementary/middle/high school. We found that only 10.7% words in Korean language texts of Korean elementary/middle /high school were used in a Korean dictionary. And we classified error types of Korean character recognition with OCR systems. For Hangul word learning, we utilized indexes of texts. With these multiple knowledge sources, we could predict a proper word in large candidate words.

  • PDF

Privacy-Preserving Language Model Fine-Tuning Using Offsite Tuning (프라이버시 보호를 위한 오프사이트 튜닝 기반 언어모델 미세 조정 방법론)

  • Jinmyung Jeong;Namgyu Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.4
    • /
    • pp.165-184
    • /
    • 2023
  • Recently, Deep learning analysis of unstructured text data using language models, such as Google's BERT and OpenAI's GPT has shown remarkable results in various applications. Most language models are used to learn generalized linguistic information from pre-training data and then update their weights for downstream tasks through a fine-tuning process. However, some concerns have been raised that privacy may be violated in the process of using these language models, i.e., data privacy may be violated when data owner provides large amounts of data to the model owner to perform fine-tuning of the language model. Conversely, when the model owner discloses the entire model to the data owner, the structure and weights of the model are disclosed, which may violate the privacy of the model. The concept of offsite tuning has been recently proposed to perform fine-tuning of language models while protecting privacy in such situations. But the study has a limitation that it does not provide a concrete way to apply the proposed methodology to text classification models. In this study, we propose a concrete method to apply offsite tuning with an additional classifier to protect the privacy of the model and data when performing multi-classification fine-tuning on Korean documents. To evaluate the performance of the proposed methodology, we conducted experiments on about 200,000 Korean documents from five major fields, ICT, electrical, electronic, mechanical, and medical, provided by AIHub, and found that the proposed plug-in model outperforms the zero-shot model and the offsite model in terms of classification accuracy.

A study on the ambiguous adnominal constructions in product documentation (제품 설명서에 나타나는 중의적 명사 수식 구문 연구 - 통제 언어의 관점에서-)

  • Park, Arum;Ji, Eun-Byul;Hong, Munpyo
    • Annual Conference on Human and Language Technology
    • /
    • 2012.10a
    • /
    • pp.23-28
    • /
    • 2012
  • 번역을 지원하는 도구로 자동 번역 시스템을 효율적으로 활용하기 위해 중요한 것은 자동 번역에 적합하도록 원문을 작성하거나 이미 작성된 원문에 대한 전처리 작업을 하는 것이다. 본 연구의 궁극적인 목표는 제품 설명서 작성자가 통제언어 체커를 통해 통제언어 규칙들을 적용하여 원문을 작성하도록 하는 것이다. 본 논문은 그 중간 단계로써 제품 설명서에 나타나는 문제 사항이 번역 품질에 어떠한 영향을 미치는지 밝혀내는 것을 목적으로 한다. 연구 대상은 제품 설명서에서 자동 번역의 성능을 저해시키는 요소 중 중의적 명사 수식 구문이다. 이러한 명사 수식 구문들은 분석 단계에서 구조적인 모호성을 초래하여 한국어 분석의 정확도를 떨어뜨리기 때문에 결과적으로 번역 품질을 악화시킬 수 있다. 이를 검증하기 위해 우선 제품 설명서 데이터를 분석하여 자동 번역 결과에 부정적인 영향을 미치는 명사 수식 구문을 다음과 같이 4가지로 유형화 하였다. (유형 1) 관형격 명사구 + 명사 병렬 접속, (유형 2) 동사의 관형형이 수식하는 명사구 + 명사 병렬 접속, (유형 3) 관형격 조사 '의' 중복, (유형 4) 병렬 접속어를 잘못 쓴 경우, 각각의 유형에 대해서 한국어 분석 단계에서 발생할 수 있는 문제에 대해 설명하였으며, 문제 사항에 대해 통제언어 규칙을 제시하였다. 통제언어 규칙에 따라 중의적 명사 수식 구문을 수정한 결과, 한국어 원문의 번역결과보다 한국어 수정문의 번역결과가 작성자의 의도를 더 잘 나타낸다는 것을 확인할 수 있었다.

  • PDF

A Study on Equation Recognition Using Tree Structure (트리 구조를 이용한 수식 인식 연구)

  • Park, Byung-Joon;Kim, Hyun-Sik;Kim, Wan-Tae
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.4
    • /
    • pp.340-345
    • /
    • 2018
  • The Compared to general sentences, the Equation uses a complex structure and various characters and symbols, so that it is not possible to input all the character sets by simply inputting a keyboard. Therefore, the editor is implemented in a text editor such as Hangul or Word. In order to express the Equation properly, it is necessary to have the learner information which can be meaningful to interpret the syntax. Even if a character is input, it can be represented by another expression depending on the relationship between the size and the position. In other words, the form of the expression is expressed as a tree model considering the relationship between characters and symbols such as the position and size to be expressed. As a field of character recognition application, a technique of recognizing characters or symbols(code) has been widely known, but a method of inputting and interpreting a Equation requires a more complicated analysis process than a general text. In this paper, we have implemented a Equation recognizer that recognizes characters in expressions and quickly analyzes the position and size of expressions.

Wordnet Extension for IT terminology Using Web Search (웹 검색을 활용한 워드넷에서의 IT 전문 용어 확장)

  • Park, Kyeong-Kook;Lee, Kwang-Mo;Kim, Yu-Seop
    • Annual Conference on Human and Language Technology
    • /
    • 2007.10a
    • /
    • pp.189-193
    • /
    • 2007
  • In this paper, we designed a methodology to expand the WordNet. We added unknown terms like IT technical terms to the existing WordNet by using web search. The WordNet is an online taxonomy representing the relationships among terms, but it usually showed limitation to contain new technical terminologies. That's why we tried to expand the WordNet. Firstly, when we met unregistered terms in WordNet, we built a query of those terms for web search. Given a web search results, we tried to find out terms with a high-level relatedness with the unregistered terms. We used the Korean Morphological Analyzer to score the relatedness between terms and located the unregistered term as a hyponym of terms with high score of relatedness.

  • PDF

An automatic extraction of newspaper articles using activation degree of 5W1H (육하원칙 활성화도를 이용한 신문기사 자동요약)

  • Yoon, Jae-Min;Kang, In-Su;Kwon, Oh-Woog;Bae, Jae-Hak;Lee, Jong-Hyeok
    • Annual Conference on Human and Language Technology
    • /
    • 2002.10e
    • /
    • pp.277-284
    • /
    • 2002
  • 본 논문은 신문기사에서 중요한 문장을 추출(Extract)하는데 있어서, 기존에 기장 우수한 방법인 전문기반 방법(Lead-based method)과 제목을 이용한 유사도 측정방법(Title-based method)의 문제점을 해결하기 위해서, 육하원칙 활성화도를 이용하여 신문기사를 효과적으로 요약할 수 있는 방법과 알고리즘을 제안하였다. 본 연구에서는 먼저, 제목(Title)과 전문(Lead)에서 중복출현하지 않는 육하원칙 구성성분을 결합하고, 본문은 각 문장에서 육하원칙 구성성분의 재사용성과 육하원칙 구성성분의 범주 증감을 파악하여 육하원칙 활성화도를 구하고, 전문기반 방법을 응용하여 각 문장의 상대적인 중요도에 따라 최종적인 가중치를 부여함으로써, 신문기사에서 중요한 문장을 효과적으로 추출할 수 있는 가중치 계산식을 제안하였다. 실험문서는 조선일보 웹사이트에서 제공하는 신문기사 100건을 대상으로 하였으며, 요약율이 30%일 경우 제안한 방법의 정확률은 74.7%로 기존의 전문기반(Lead-based method)방법보다 6.7% 향상되었다.

  • PDF

A Study on the Feature Extraction of Strokes using the Maximum Block Methode (최대 블록화 방법을 이용한 묵자획 특징 추출에 관한 연구)

  • Kim, Ui-Jeong;Kim, Tae-Gyun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.4
    • /
    • pp.1141-1151
    • /
    • 1997
  • In this paper the Maximum Block Method is suggested for the Feature Extraction of stokes of off-line Chinese characters.The Maximum Block Method is a technique which enlarges the block from the first found pixel that wxtracts the skeleton and features of the input characters.The maximum Block mthod is an adequate technique for the correct extraction of the features since the exsting thining methods have shortcomings of making the feature extraction difficult from the distoritions generated from the effiects of the parial noises,inflection points and blemishes. The printed outputs and chinese books of the middle and high school students,and other materials are used for the test.It was found that the Maxthod is also an effective technique for the extraction of skeleton line and features,which is the preoprocessing of the pattern recognition,for the Korean chracters and English as well as chinese chracters.

  • PDF

A Deep Learning-based Depression Trend Analysis of Korean on Social Media (딥러닝 기반 소셜미디어 한글 텍스트 우울 경향 분석)

  • Park, Seojeong;Lee, Soobin;Kim, Woo Jung;Song, Min
    • Journal of the Korean Society for information Management
    • /
    • v.39 no.1
    • /
    • pp.91-117
    • /
    • 2022
  • The number of depressed patients in Korea and around the world is rapidly increasing every year. However, most of the mentally ill patients are not aware that they are suffering from the disease, so adequate treatment is not being performed. If depressive symptoms are neglected, it can lead to suicide, anxiety, and other psychological problems. Therefore, early detection and treatment of depression are very important in improving mental health. To improve this problem, this study presented a deep learning-based depression tendency model using Korean social media text. After collecting data from Naver KonwledgeiN, Naver Blog, Hidoc, and Twitter, DSM-5 major depressive disorder diagnosis criteria were used to classify and annotate classes according to the number of depressive symptoms. Afterwards, TF-IDF analysis and simultaneous word analysis were performed to examine the characteristics of each class of the corpus constructed. In addition, word embedding, dictionary-based sentiment analysis, and LDA topic modeling were performed to generate a depression tendency classification model using various text features. Through this, the embedded text, sentiment score, and topic number for each document were calculated and used as text features. As a result, it was confirmed that the highest accuracy rate of 83.28% was achieved when the depression tendency was classified based on the KorBERT algorithm by combining both the emotional score and the topic of the document with the embedded text. This study establishes a classification model for Korean depression trends with improved performance using various text features, and detects potential depressive patients early among Korean online community users, enabling rapid treatment and prevention, thereby enabling the mental health of Korean society. It is significant in that it can help in promotion.