• The Journal of Korean Association of Computer Education
• /
• v.14 no.2
• /
• pp.83-94
• /
• 2011

In the biomedical domain, the longest matching method is frequently used for recognizing named entity written in the literature. This method uses a dictionary as a resource for named entity recognition. If there exist appropriated dictionary about target domain, the longest matching method has the advantage of being able to recognize the entities of target domain quickly and exactly. However, the longest matching method is difficult to recognize the enumerated named entities, because these entities are frequently expressed as being omitted some words. In order to resolve this problem, we propose the partial matching method using a dictionary. The proposed method makes several candidate entities on the assumption that the ellipses may be included. After that, the method selects the most valid one among candidate entities through the optimization algorithm. We tested the longest and partial matching method about HLA entities: HLA gene, antigen, and allele entities, which are frequently enumerated among biomedical entities. As preparing for named entity recognition, we built two new resource, extended dictionary and tag-based dictionary about HLA entities. And later, we performed the longest and partial matching method using each dictionary. According to our experiment result, the longest matching method was effective in recognizing HLA antigen entities, in which the ellipses are rare, and the partial matching method was effective in recognizing HLA gene and allele entities, in which the ellipses are frequent. Especially, the partial matching method had a high F-score 95.59% about HLA alleles.

• Journal of the Korean Society for Library and Information Science
• /
• v.57 no.1
• /
• pp.93-114
• /
• 2023

Information extraction can facilitate the intensive analysis of documents by providing semantic triples which consist of named entities and their relations recognized in the texts. However, most of the research so far has been carried out separately for named entity recognition and relation extraction as individual studies, and as a result, the effective performance evaluation of the entire information extraction systems was not performed properly. This paper introduces two models of end-to-end information extraction that can extract various entity names in clinical records and their relationships in the form of semantic triples, namely pipeline and joint models and compares their performances in depth. The pipeline model consists of an entity recognition sub-system based on bidirectional GRU-CRFs and a relation extraction module using multiple encoding scheme, whereas the joint model was implemented with a single bidirectional GRU-CRFs equipped with multi-head labeling method. In the experiments using i2b2/VA 2010, the performance of the pipeline model was 5.5% (F-measure) higher. In addition, through a comparative experiment with existing state-of-the-art systems using large-scale neural language models and manually constructed features, the objective performance level of the end-to-end models implemented in this paper could be identified properly.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.34 no.4
• /
• pp.577-586
• /
• 2024

As new technologies are developed, new security threats such as the emergence of AI technologies that create ransomware are also increasing. New security equipment such as XDR has been developed to cope with these security threats, but when using various security equipment together rather than a single security equipment environment, there is a difficulty in creating numerous regular expressions for identifying and classifying essential data. To solve this problem, this paper proposes a method of identifying essential information for identifying threat information by introducing artificial intelligence-based entity name recognition technology in various security equipment usage environments. After analyzing the security equipment log data to select essential information, the storage format of information and the tag list for utilizing artificial intelligence were defined, and the method of identifying and extracting essential data is proposed through entity name recognition technology using artificial intelligence. As a result of various security equipment log data and 23 tag-based entity name recognition tests, the weight average of f1-score for each tag is 0.44 for Bi-LSTM-CRF and 0.99 for BERT-CRF. In the future, we plan to study the process of integrating the regular expression-based threat information identification and extraction method and artificial intelligence-based threat information and apply the process based on new data.

• 한국어정보학회:학술대회논문집
• /
• 2017.10a
• /
• pp.338-339
• /
• 2017

본 논문은 개체명 인식을 위해 CRF 모델을 이용해 분류를 수행했다. 개체명 후보를 개체명으로 식별에서 중의성 문제가 필요하다. 본 논문에서는 이러한 중의성 문제 해결을 위해 학습 셋으로부터 패턴과 형태적 특성을 고려해 개체명 후보를 최대로 선택하고 선택된 개체명 후보의 중의성과 정확도를 높이기 위해 주변의 문맥 자질과 분별 확률 모델인 CRF를 이용해 중의성 문제를 해결한다.

• Journal of Information Management
• /
• v.40 no.4
• /
• pp.1-22
• /
• 2009

Large-scaled information extraction plays an important role in advanced information retrieval as well as question answering and summarization. Information extraction can be defined as a process of converting unstructured documents into formalized, tabular information, which consists of named-entity recognition, terminology extraction, coreference resolution and relation extraction. Since all the elementary technologies have been studied independently so far, it is not trivial to integrate all the necessary processes of information extraction due to the diversity of their input/output formation approaches and operating environments. As a result, it is difficult to handle scientific documents to extract both named-entities and technical terms at once. In this study, we define scientific as a set of 10 types of named entities and technical terminologies in a biomedical domain. in order to automatically extract these entities from scientific documents at once, we develop a framework for scientific core entity extraction which embraces all the pivotal language processors, named-entity recognizer, co-reference resolver and terminology extractor. Each module of the integrated system has been evaluated with various corpus as well as KEEC 2009. The system will be utilized for various information service areas such as information retrieval, question-answering(Q&A), document indexing, dictionary construction, and so on.

• Journal of Internet Computing and Services
• /
• v.24 no.6
• /
• pp.127-135
• /
• 2023

With the development of information and communication technology, the amount of information produced and shared in the battlefield and stored and managed in the system dramatically increased. This means that the amount of information which cansupport situational awareness and decision making of the commanders has increased, but on the other hand, it is also a factor that hinders rapid decision making by increasing the information overload on the commanders. To overcome this limitation, this study proposes a method to automatically search, select, and summarize documents that can help the commanders to understand the battlefield situation reports that he or she received. First, named entities are discovered from the battlefield situation report using a named entity recognition method. Second, the documents related to each named entity are discovered. Third, a language model and collaborative filtering are used to select the documents. At this time, the language model is used to calculate the similarity between the received report and the discovered documents, and collaborative filtering is used to reflect the commander's document reading history. Finally, sentences containing each named entity are selected from the documents and sorted. The experiment was carried out using academic papers since their characteristics are similar to military documents, and the validity of the proposed method was verified.

• 한국어정보학회:학술대회논문집
• /
• 2016.10a
• /
• pp.105-110
• /
• 2016

개체명 인식이란 문서 내에서 인명, 기관명, 지명, 시간, 날짜 등 고유한 의미를 가지는 개체명을 추출하여 그 종류를 결정하는 것을 말한다. 최근 개체명 인식 연구에서는 bidirectional LSTM CRFs가 가장 우수한 성능을 보여주고 있다. 하지만 LSTM 기반의 딥 러닝 모델은 입력이 되는 단어 표상에 의존적이기 때문에 입력이 되는 단어 표상을 확장하는 방법에 대한 연구가 많이 진행되어지고 있다. 본 논문에서는 한국어 개체명 인식을 위하여 bidirectional LSTM CRFs모델을 사용하고, 그 입력으로 사용되는 단어 표상을 확장하기 위해 사전 학습된 단어 임베딩 벡터, 품사 임베딩 벡터, 그리고 음절 기반에서 확장된 단어 임베딩 벡터를 사용한다. 음절 기반에서 단어 기반 임베딩 벡터로 확장하기 위하여 bidirectional LSTM을 이용하고, 그 입력으로 학습 데이터에서 추출한 개체명 분포를 이용하였다. 그 결과 사전 학습된 단어 임베딩 벡터만 사용한 것보다 4.93%의 성능 향상을 보였다.

• Annual Conference on Human and Language Technology
• /
• 2013.10a
• /
• pp.82-85
• /
• 2013

개체명 인식기에 대한 연구에서 개체명 사전은 필수적으로 필요하다. 그러나 공개된 개체명 사전은 거의 없기 때문에, 본 논문에서는 디비피디아의 데이터로부터 개체명을 효과적으로 추출하여 자동으로 구축할 수 있는 방법을 제안한다. 제안 방법은 엔트리의 '이름'과 '분류' 정보를 사용한다. 엔트리의 '이름'은 개체명으로 사용하고, 엔트리의 '분류'는 각 개체명 클래스와의 상호정보량을 계산하여 엔트리와 개체명 클래스 사이의 점수를 계산한다. 이렇게 계산된 점수를 이용하여 개체명과 개체명 클래스를 매핑한다. 그 결과 76.7%의 평균 정확률을 보였다.

• Annual Conference on Human and Language Technology
• /
• 2016.10a
• /
• pp.105-110
• /
• 2016

개체명 인식이란 문서 내에서 인명, 기관명, 지명, 시간, 날짜 등 고유한 의미를 가지는 개체명을 추출하여 그 종류를 결정하는 것을 말한다. 최근 개체명 인식 연구에서는 bidirectional LSTM CRFs가 가장 우수한 성능을 보여주고 있다. 하지만 LSTM 기반의 딥 러닝 모델은 입력이 되는 단어 표상에 의존적이기 때문에 입력이 되는 단어 표상을 확장하는 방법에 대한 연구가 많이 진행되어지고 있다. 본 논문에서는 한국어 개체명 인식을 위하여 bidirectional LSTM CRFs모델을 사용하고, 그 입력으로 사용되는 단어 표상을 확장하기 위해 사전 학습된 단어 임베딩 벡터, 품사 임베딩 벡터, 그리고 음절 기반에서 확장된 단어 임베딩 벡터를 사용한다. 음절 기반에서 단어 기반 임베딩 벡터로 확장하기 위하여 bidirectional LSTM을 이용하고, 그 입력으로 학습 데이터에서 추출한 개체명 분포를 이용하였다. 그 결과 사전 학습된 단어 임베딩 벡터만 사용한 것보다 4.93%의 성능 향상을 보였다.

• Journal of Internet Computing and Services
• /
• v.13 no.2
• /
• pp.99-108
• /
• 2012

Many methods had been developed to improve the accuracy of extracting information from a vast amount of data. This paper combined a number of natural language processing methods such as NER (named entity recognition), sentence extraction, and part of speech tagging to carry out text analysis. The data source is comprised of texts obtained from the web using a domain-specific data extraction agent. A framework for the extraction of information from unstructured data was developed using the aforementioned natural language processing methods. We simulated the performance of our work in the extraction and analysis of texts for the detection of organizational structures. Simulation shows that our study outperformed other NER classifiers such as MUC and CoNLL on information extraction.