• Journal of Intelligence and Information Systems
• /
• v.26 no.1
• /
• pp.1-21
• /
• 2020

With the rapid development of artificial intelligence technology, various techniques have been developed to extract meaningful information from unstructured text data which constitutes a large portion of big data. Over the past decades, text mining technologies have been utilized in various industries for practical applications. In the field of business intelligence, it has been employed to discover new market and/or technology opportunities and support rational decision making of business participants. The market information such as market size, market growth rate, and market share is essential for setting companies' business strategies. There has been a continuous demand in various fields for specific product level-market information. However, the information has been generally provided at industry level or broad categories based on classification standards, making it difficult to obtain specific and proper information. In this regard, we propose a new methodology that can estimate the market sizes of product groups at more detailed levels than that of previously offered. We applied Word2Vec algorithm, a neural network based semantic word embedding model, to enable automatic market size estimation from individual companies' product information in a bottom-up manner. The overall process is as follows: First, the data related to product information is collected, refined, and restructured into suitable form for applying Word2Vec model. Next, the preprocessed data is embedded into vector space by Word2Vec and then the product groups are derived by extracting similar products names based on cosine similarity calculation. Finally, the sales data on the extracted products is summated to estimate the market size of the product groups. As an experimental data, text data of product names from Statistics Korea's microdata (345,103 cases) were mapped in multidimensional vector space by Word2Vec training. We performed parameters optimization for training and then applied vector dimension of 300 and window size of 15 as optimized parameters for further experiments. We employed index words of Korean Standard Industry Classification (KSIC) as a product name dataset to more efficiently cluster product groups. The product names which are similar to KSIC indexes were extracted based on cosine similarity. The market size of extracted products as one product category was calculated from individual companies' sales data. The market sizes of 11,654 specific product lines were automatically estimated by the proposed model. For the performance verification, the results were compared with actual market size of some items. The Pearson's correlation coefficient was 0.513. Our approach has several advantages differing from the previous studies. First, text mining and machine learning techniques were applied for the first time on market size estimation, overcoming the limitations of traditional sampling based- or multiple assumption required-methods. In addition, the level of market category can be easily and efficiently adjusted according to the purpose of information use by changing cosine similarity threshold. Furthermore, it has a high potential of practical applications since it can resolve unmet needs for detailed market size information in public and private sectors. Specifically, it can be utilized in technology evaluation and technology commercialization support program conducted by governmental institutions, as well as business strategies consulting and market analysis report publishing by private firms. The limitation of our study is that the presented model needs to be improved in terms of accuracy and reliability. The semantic-based word embedding module can be advanced by giving a proper order in the preprocessed dataset or by combining another algorithm such as Jaccard similarity with Word2Vec. Also, the methods of product group clustering can be changed to other types of unsupervised machine learning algorithm. Our group is currently working on subsequent studies and we expect that it can further improve the performance of the conceptually proposed basic model in this study.

• Journal of Intelligence and Information Systems
• /
• v.24 no.2
• /
• pp.59-83
• /
• 2018

With the increasing importance of sentiment analysis to grasp the needs of customers and the public, various types of deep learning models have been actively applied to English texts. In the sentiment analysis of English texts by deep learning, natural language sentences included in training and test datasets are usually converted into sequences of word vectors before being entered into the deep learning models. In this case, word vectors generally refer to vector representations of words obtained through splitting a sentence by space characters. There are several ways to derive word vectors, one of which is Word2Vec used for producing the 300 dimensional Google word vectors from about 100 billion words of Google News data. They have been widely used in the studies of sentiment analysis of reviews from various fields such as restaurants, movies, laptops, cameras, etc. Unlike English, morpheme plays an essential role in sentiment analysis and sentence structure analysis in Korean, which is a typical agglutinative language with developed postpositions and endings. A morpheme can be defined as the smallest meaningful unit of a language, and a word consists of one or more morphemes. For example, for a word '예쁘고', the morphemes are '예쁘(= adjective)' and '고(=connective ending)'. Reflecting the significance of Korean morphemes, it seems reasonable to adopt the morphemes as a basic unit in Korean sentiment analysis. Therefore, in this study, we use 'morpheme vector' as an input to a deep learning model rather than 'word vector' which is mainly used in English text. The morpheme vector refers to a vector representation for the morpheme and can be derived by applying an existent word vector derivation mechanism to the sentences divided into constituent morphemes. By the way, here come some questions as follows. What is the desirable range of POS(Part-Of-Speech) tags when deriving morpheme vectors for improving the classification accuracy of a deep learning model? Is it proper to apply a typical word vector model which primarily relies on the form of words to Korean with a high homonym ratio? Will the text preprocessing such as correcting spelling or spacing errors affect the classification accuracy, especially when drawing morpheme vectors from Korean product reviews with a lot of grammatical mistakes and variations? We seek to find empirical answers to these fundamental issues, which may be encountered first when applying various deep learning models to Korean texts. As a starting point, we summarized these issues as three central research questions as follows. First, which is better effective, to use morpheme vectors from grammatically correct texts of other domain than the analysis target, or to use morpheme vectors from considerably ungrammatical texts of the same domain, as the initial input of a deep learning model? Second, what is an appropriate morpheme vector derivation method for Korean regarding the range of POS tags, homonym, text preprocessing, minimum frequency? Third, can we get a satisfactory level of classification accuracy when applying deep learning to Korean sentiment analysis? As an approach to these research questions, we generate various types of morpheme vectors reflecting the research questions and then compare the classification accuracy through a non-static CNN(Convolutional Neural Network) model taking in the morpheme vectors. As for training and test datasets, Naver Shopping's 17,260 cosmetics product reviews are used. To derive morpheme vectors, we use data from the same domain as the target one and data from other domain; Naver shopping's about 2 million cosmetics product reviews and 520,000 Naver News data arguably corresponding to Google's News data. The six primary sets of morpheme vectors constructed in this study differ in terms of the following three criteria. First, they come from two types of data source; Naver news of high grammatical correctness and Naver shopping's cosmetics product reviews of low grammatical correctness. Second, they are distinguished in the degree of data preprocessing, namely, only splitting sentences or up to additional spelling and spacing corrections after sentence separation. Third, they vary concerning the form of input fed into a word vector model; whether the morphemes themselves are entered into a word vector model or with their POS tags attached. The morpheme vectors further vary depending on the consideration range of POS tags, the minimum frequency of morphemes included, and the random initialization range. All morpheme vectors are derived through CBOW(Continuous Bag-Of-Words) model with the context window 5 and the vector dimension 300. It seems that utilizing the same domain text even with a lower degree of grammatical correctness, performing spelling and spacing corrections as well as sentence splitting, and incorporating morphemes of any POS tags including incomprehensible category lead to the better classification accuracy. The POS tag attachment, which is devised for the high proportion of homonyms in Korean, and the minimum frequency standard for the morpheme to be included seem not to have any definite influence on the classification accuracy.

• Tuberculosis and Respiratory Diseases
• /
• v.43 no.5
• /
• pp.671-682
• /
• 1996

Background : Endobronchial tuberculosis is definded as tuberculous infection of the tracheobronchial tree with microbiological and histopathological evidence. Endobronchial tuberculosis has clinical significance due to its sequela of cicatrical stenosis which causes atelectasis, dyspnea and secondary pneumonia and may mimic bronchial asthma and pulmanary malignancy. Method : The authors carried out, retrospectively, a clinical study on 201 patients confirmed with endobronchial tuberculosis who visited the Department of Pulmonary Medicine at Hangyang University Hospital from January 1990 10 April 1996. The following results were obtained. Results: 1) Total 201 parients(l9.5%) were confirmed as endobronchial tuberculosis among 1031 patients who had been undergone flexible bronchofiberscopic examination. The number of male patients were 55 and that of female patients were 146. and the male to female ratio was 1 : 2.7. 2) The age distribution were as follows: there were 61(30.3%) cases in the third decade, 40 cases(19.9%) in the fourth decade, 27 cases(13.4%) in the sixth decade, 21 cases(10.4%) in the fifth decade, 19 cases(9.5%) in the age group between 15 and 19 years, 19 cases(9.5%) in the seventh decade, and 14 cases(7.0%) over 70 years, in decreasing order. 3) The most common symptom, in 192 cases, was cough 74.5%, followed by sputum 55.2%, dyspnea 28.6%, chest discomfort 19.8%, fever 17.2%, hemoptysis 11.5%, in decreasing order, and localized wheezing was heard in 15.6%. 4) In chest X-ray of 189 cases, consolidation was the most frequent finding in 67.7%, followed by collapse 43.9%. cavitary lesion 11.6%, pleural effusion 7.4%, in decreasing order, and there was no abnormal findings in 3.2%. 5) In the 76 pulmanary function tests, a normal pattern was found in 44.7%, restrictive pattern in 39.5 %, obstructive pattern in 11.8%, and combined pattern in 3.9%. 6) Among total 201 patients, bronchoscopy showed caseous pseudomembrane in 70 cases(34.8%), mucosal erythema and edema in 54 cases(26.9%), hyperplastic lesion in 52 cases(25.9%), fibrous s.enosis in 22 cases(10.9%), and erosion or ulcer in 3 cases(1.5%). 7) In total 201 cases, bronchial washing AFB stain was positive in 103 cases(51.2%), bronchial washing culture for tuberculous bacilli in 55 cases(27.4%). In the 99 bronchoscopic biopsies, AFB slain positive in 36.4%. granuloma without AFB stain positive in 13.1%, chronic inflammation only in 36.4%. and non diagnostic biopsy finding in 14.1%. Conclusions : Young female patients, whose cough resistant to genenal antitussive agents, should be evaluated for endobronchial tuberculosis, even with clear chest roentgenogram and negative sputum AFB stain. Furthermore, we would like to emphasize that the bronchoscopic approach is a substantially useful means of making a differential diagnosis of atelectasis in older patients of cancer age. At this time we have to make a standard endoscopic classification of endobronchial tuberculosis, and well designed prospective studies are required to elucidate the effect of combination therapy using antituberculous chemotherapy with steroids on bronchial stenosis in patients with endobronchial tuberculosis.