• Journal of Intelligence and Information Systems
• /
• v.21 no.1
• /
• pp.1-13
• /
• 2015

As opinion mining in big data applications has been highlighted, a lot of research on unstructured data has made. Lots of social media on the Internet generate unstructured or semi-structured data every second and they are often made by natural or human languages we use in daily life. Many words in human languages have multiple meanings or senses. In this result, it is very difficult for computers to extract useful information from these datasets. Traditional web search engines are usually based on keyword search, resulting in incorrect search results which are far from users' intentions. Even though a lot of progress in enhancing the performance of search engines has made over the last years in order to provide users with appropriate results, there is still so much to improve it. Word sense disambiguation can play a very important role in dealing with natural language processing and is considered as one of the most difficult problems in this area. Major approaches to word sense disambiguation can be classified as knowledge-base, supervised corpus-based, and unsupervised corpus-based approaches. This paper presents a method which automatically generates a corpus for word sense disambiguation by taking advantage of examples in existing dictionaries and avoids expensive sense tagging processes. It experiments the effectiveness of the method based on Naïve Bayes Model, which is one of supervised learning algorithms, by using Korean standard unabridged dictionary and Sejong Corpus. Korean standard unabridged dictionary has approximately 57,000 sentences. Sejong Corpus has about 790,000 sentences tagged with part-of-speech and senses all together. For the experiment of this study, Korean standard unabridged dictionary and Sejong Corpus were experimented as a combination and separate entities using cross validation. Only nouns, target subjects in word sense disambiguation, were selected. 93,522 word senses among 265,655 nouns and 56,914 sentences from related proverbs and examples were additionally combined in the corpus. Sejong Corpus was easily merged with Korean standard unabridged dictionary because Sejong Corpus was tagged based on sense indices defined by Korean standard unabridged dictionary. Sense vectors were formed after the merged corpus was created. Terms used in creating sense vectors were added in the named entity dictionary of Korean morphological analyzer. By using the extended named entity dictionary, term vectors were extracted from the input sentences and then term vectors for the sentences were created. Given the extracted term vector and the sense vector model made during the pre-processing stage, the sense-tagged terms were determined by the vector space model based word sense disambiguation. In addition, this study shows the effectiveness of merged corpus from examples in Korean standard unabridged dictionary and Sejong Corpus. The experiment shows the better results in precision and recall are found with the merged corpus. This study suggests it can practically enhance the performance of internet search engines and help us to understand more accurate meaning of a sentence in natural language processing pertinent to search engines, opinion mining, and text mining. Naïve Bayes classifier used in this study represents a supervised learning algorithm and uses Bayes theorem. Naïve Bayes classifier has an assumption that all senses are independent. Even though the assumption of Naïve Bayes classifier is not realistic and ignores the correlation between attributes, Naïve Bayes classifier is widely used because of its simplicity and in practice it is known to be very effective in many applications such as text classification and medical diagnosis. However, further research need to be carried out to consider all possible combinations and/or partial combinations of all senses in a sentence. Also, the effectiveness of word sense disambiguation may be improved if rhetorical structures or morphological dependencies between words are analyzed through syntactic analysis.

• MALSORI
• /
• no.50
• /
• pp.123-137
• /
• 2004

In this paper, we consider the implementation of connected digit recognition system using tree structured lexicon model. To implement efficiently the fixed or variable length digit recognition system, finite state network (FSN) is required. We merge the word network algorithm that implements the FSN with lexical tree search algorithm that is used for general speech recognition system for fast search and large vocabulary systems. To find the efficient modeling of digit recognition system, we investigate some performance changes when the lexical tree search is applied.

• IE interfaces
• /
• v.19 no.3
• /
• pp.225-235
• /
• 2006

In this paper, we propose an intelligent e-commerce meta-search engine which integrates distributed e-commerce sites and provides a unified search to the sites. The meta-search engine performs the following functions: (1) the user is able to create a category-based user query, (2) by using the WordNet, the query is semantically refined for increasing search accuracy, and (3) the meta-search engine recommends an e-commerce site which has the closest product information to the user’s search intention by matching the user query with the product catalogs in the e-commerce sites linked to the meta-search engine. An experiment shows that the performance of our model is better than that of general keyword-based search.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2005.11a
• /
• pp.346-355
• /
• 2005

In this paper, we propose a meta-search engine which provides distributed product information through a unified access to multiple e-commerce. The meta-search engine proposed in this paper performs the following functions: (I) The user is able to create a category-based user query, (2) by using the WordNet, the query is semantical refined fined for increasing search accuracy, and (3) the meta-search engine recommends an e-commerce site which has the closest product information to the user's search intention, by matching the user query with the product catalogs in the e-commerce sites linked to the meta-search engine. An experiment shows that the performance of our model is better than that of general keyword-based search.

• The KIPS Transactions:PartB
• /
• v.15B no.2
• /
• pp.171-178
• /
• 2008

Foreign word transliterations are not consistently used in documents, which hinders retrieving some important relevant documents in exact term matching information retrieval systems. In this paper, a meta search method is proposed, which expands and refines relevant variant queries from an original input foreign word transliteration query to retrieve the more relevant documents. The method firstly expands a transliteration query to the variants using a statistical method. Secondly the method selects the valid variants: it queries each variant to the retrieval systems beforehand and checks the validity of each variant by counting the number of appearance of the variant in the retrieved document and calculating the similarity of the context of the variant. Experiment result showed that querying with the variants produced at the first step, which is a base method of the test, performed 38% in average F measure, and querying with the refined variants at the second step, which is a proposed method, significantly improved the performance to 81% in average F measure.

• Journal of Korean Society for Quality Management
• /
• v.50 no.4
• /
• pp.751-776
• /
• 2022

Purpose: The purpose of this paper is to present implications by analyzing research trends on CSR, CSV and ESG by text analysis and visual analysis(Comprehensive/ Fields / Years-based) which are big data analyses, by collecting data based on previous studies on CSR, CSV and ESG. Methods: For the collection of analysis data, deep learning was used in the integrated search on the Academic Research Information Service (www.riss.kr) to search for "CSR", "CSV" and "ESG" as search terms, and the Korean abstracts and keyword were scrapped out of the extracted paper and they are organize into EXCEL. For the final step, CSR 2,847 papers, CSV 395 papers, ESG 555 papers derived were analyzed using the Rx64 4.0.2 program and Rstudio using text mining, one of the big data analysis techniques, and Word Cloud for visualization. Results: The results of this study are as follows; CSR, CSV, and ESG studies showed that research slowed down somewhat before 2010, but research increased rapidly until recently in 2019. Research have been found to be heavily researched in the fields of social science, art and physical education, and engineering. As a result of the study, there were many keyword of 'corporate', 'social', and 'responsibility', which were similar in the word cloud analysis. Looking at the frequent keyword and word cloud analysis by field and year, overall keyword were derived similar to all keyword by year. However, some differences appeared in each field. Conclusion: Government support and expert support for CSR, CSV and ESG should be activated, and researches on technology-based strategies are needed. In the future, it is necessary to take various approaches to them. If researches are conducted in consideration of the environment or energy, it is judged that bigger implications can be presented.

• Proceedings of the Korea Society of Mathematical Education Conference
• /
• 1990.07a
• /
• pp.64-64
• /
• 1990

• The Journal of the Acoustical Society of Korea
• /
• v.21 no.6
• /
• pp.543-551
• /
• 2002

In this paper, we present an efficient decoding method that performs in real time for 20k word continuous speech recognition task. Basic search method is a one-pass Viterbi decoder on the search space constructed from the novel language model network. With the consistent search space representation derived from various language models by the LM network, we incorporate basic pruning strategies, from which tokens alive constitute a dynamic search space. To facilitate post-processing, it produces a word graph and a N-best list subsequently. The decoder is tested on the database of 20k words and evaluated with respect to accuracy and RTF.

• Journal of Korea Multimedia Society
• /
• v.22 no.3
• /
• pp.374-385
• /
• 2019

A Web Crawler is a program, which is commonly used by search engines to find the new brainchild on the internet. The use of crawlers has made the web easier for users. In this paper, we have used unstructured data by structuralization to collect data from the web pages. Our system is able to choose the word near our keyword in more than one document using unstructured way. Neighbor data were collected on the keyword through word2vec. The system goal is filtered at the data acquisition level and for a large taxonomy. The main problem in text taxonomy is how to improve the classification accuracy. In order to improve the accuracy, we propose a new weighting method of TF-IDF. In this paper, we modified TF-algorithm to calculate the accuracy of unstructured data. Finally, our system proposes a competent web pages search crawling algorithm, which is derived from TF-IDF and RL Web search algorithm to enhance the searching efficiency of the relevant information. In this paper, an attempt has been made to research and examine the work nature of crawlers and crawling algorithms in search engines for efficient information retrieval.

• Annual Conference on Human and Language Technology
• /
• 2019.10a
• /
• pp.503-505
• /
• 2019

본 논문에서는 기존 웹툰 장르 검색 시스템의 단점을 보완하기 위해 키워드 기반 유사 장르 검색 시스템을 제안한다. 기존 웹툰의 장르와 키워드를 분석하여 44개의 장르를 설정하고 해당 장르에 적합한 웹툰을 수집한다. 나무위키와 위키피디아 문서로 학습된 Word2Vec모델에 기반하여 계산한 사용자 입력 키워드와 44개의 장르간 유사도로 사용자 입력에 가장 유사한 장르를 찾는다. 유사 장르에 포함되는 웹툰을 결과로 출력하여 사용자가 선호하는 장르의 웹툰을 제시한다. 실험 결과에서는 나무위키에서 '장르'로 검색하여 얻는 작은 크기의 문서 집합에서 Word2Vec을 학습한 모델에서 가장 높은 검색 성능을 보였다.