• The Korean Journal of Applied Statistics
• /
• 제36권6호
• /
• pp.501-514
• /
• 2023

When analyzing high dimensional data such as text data, if we input all the variables as explanatory variables, statistical learning procedures may suffer from over-fitting problems. Furthermore, computational efficiency can deteriorate with a large number of variables. Dimensionality reduction techniques such as feature selection or feature extraction are useful for dealing with these problems. The sparse principal component analysis (SPCA) is one of the regularized least squares methods which employs an elastic net-type objective function. The SPCA can be used to remove insignificant principal components and identify important variables from noisy observations. In this study, we propose a dimension reduction procedure for text data based on the SPCA. Applying the proposed procedure to real data, we find that the reduced feature set maintains sufficient information in text data while the size of the feature set is reduced by removing redundant variables. As a result, the proposed procedure can improve classification accuracy and computational efficiency, especially for some classifiers such as the k-nearest neighbors algorithm.

• Journal of IKEEE
• /
• 제8권2호
• /
• pp.186-194
• /
• 2004

Due to the exponential growth of information on the internet, it is getting difficult to find and organize relevant informations. To reduce heavy overload of accesses to information, automatic text classification for handling enormous documents is necessary. In this paper, we describe structure and implementation of a document classification system for web documents. We utilize SVM for documentation classification model that is constructed based on training set and its representative terms in a directory. In our system, SVM is trained and is used for document classification by using word set that is extracted from information and communication related web documents. In addition, we use vector-space model in order to represent characteristics based on TFiDF and training data consists of positive and negative classes that are represented by using characteristic set with weight. Experiments show the results of categorization and the correlation of vector length.

• Journal of the Korean Society of Earth Science Education
• /
• 제5권3호
• /
• pp.256-266
• /
• 2012

The purpose of this study is to use DARTs (Directed Activities Related to text) to foster inquiry problems while actively engaging accelerated gifted elementary students in the field of earth sciences. This study is continually evolving in the classroom on the proposition that accelerate the scientific thought whether inquiry problems show any change according to the extent of prior background knowledge through DARTs. Researchers appointed the accelerated gifted elementary students with 14 investigation problems and it was their duty to not only classify the inquiry problems, but to analyze using interviewing methods according to type classification framework. Many scientific terms were used concretely in the inquiry problems that were propose after DART. The students gave a direct effect to the inquiry problem to be proposed according to the level of the content that it is presented in the DARTs worksheet. As a result, the NP-IP type and the EC-IP, NC-IP inquiry problem type proposed above much as a whole in DARTs former and prior. Particularly, the EMC-IP type and etc. was variously proposed after the DARTs. And the students proposing the inquiry problem of above average proposed the inquiry problem of the EP-IP type much unlike the general average student after the DARTs. The EC-IP, NC-IP and NF-IP type were changed much after DARTs used. Particularly, the EC-IP and NC-IP type were changed much.

• Journal of the Korean Society for information Management
• /
• 제36권2호
• /
• pp.255-273
• /
• 2019

Recently, the table of contents (TOC) has been becoming increasingly accessible and utilized. The study conducted descriptive statistics and comparative analysis of the table of contents in terms of parts of speech and subject in text. For this purpose, this study chose the books of the social sciences field from acquisition lists of an academic library, obtained Dewey class numbers of target books from KERIS union catalog, and extracted TOC data from online bookstore. Morphological analysis was performed on each book titles and TOCs, and descriptive statistics and frequency analysis were carried out. As a result, nouns made up roughly half of the morphemes of titles or the TOCs. TOCs had about 50 times more nouns than titles. The percentage of unique nouns that appeared only in the table of contents is estimated to be 95.2% of the TOC's total nouns. The table of contents also showed a differences in its lengths depending on the field of social science.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• 제17권11호
• /
• pp.1175-1184
• /
• 1992

A new classification method and recognition algorithms for printed Korean and Chinese character is studied for Korean text which contains both Korean and Chinese characters. The proposed method utilizes structural features of the vertical and horizontal vowel in Korean character. Korean characters are classified into 6 groups. Vowel and consonant are separated by means of different vowel extraction methods applied to each group. Time consuming thinning process is excluded. A modified crossing distance feature is measured to recognize extracted consonant. For Chinese character, an average of stroke crossing number is calculated on every characters, which allows the characters to be classified into several groups. A recognition process is then followed in terms of the stroke crossing number and the black dot rate of character. Classification between Korean and Chinese character was at the rate of 90.5%, and classification rate of Ming-style 2512 Korean characters was 90.0%. The recognition algorithm was applied on 1278 characters. The recognition rate was 92.2%. The densest class after classification of 4585 Chinese characters was found to contain only 124 characters, only 1/40 of total numbers. The recognition rate was 89.2%.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제6권11호
• /
• pp.2866-2879
• /
• 2012

Among the various security threats in online games, the use of game bots is the most serious problem. Previous studies on game bot detection have proposed many methods to find out discriminable behaviors of bots from humans based on the fact that a bot's playing pattern is different from that of a human. In this paper, we look at the chatting data that reflects gamers' communication patterns and propose a communication pattern analysis framework for online game bot detection. In massive multi-user online role playing games (MMORPGs), game bots use chatting message in a different way from normal users. We derive four features; a network feature, a descriptive feature, a diversity feature and a text feature. To measure the diversity of communication patterns, we propose lightly summarized indices, which are computationally inexpensive and intuitive. For text features, we derive lexical, syntactic and semantic features from chatting contents using text mining techniques. To build the learning model for game bot detection, we test and compare three classification models: the random forest, logistic regression and lazy learning. We apply the proposed framework to AION operated by NCsoft, a leading online game company in Korea. As a result of our experiments, we found that the random forest outperforms the logistic regression and lazy learning. The model that employs the entire feature sets gives the highest performance with a precision value of 0.893 and a recall value of 0.965.

• Journal of The Korean Association of Information Education
• /
• 제23권6호
• /
• pp.543-550
• /
• 2019

In 2006, Janet Wing defined computational thinking and operated SW education as a formal curriculum in the UK in 2013. This study collected related research papers by using computational thinking, which has recently increased in importance, and analyzed it using text mining. In the first, CONCOR analysis was conducted with the keyword of computational thinking. In the second, text mining of the components of computational thinking was selected by the repr23esentative academic journals at domestic and foreign. As a result of the two-time analysis, first, abstraction, algorithm, data processing, problem decomposition, and pattern recognition were the core of the study of computational thinking component. Second, research on convergence education centered on computational thinking and science and mathematics subjects was actively conducted. Third, research on computational thinking has been expanding since 2010. Research and development of the classification and definition of computational thinking and components and applying them to education sites should be conducted steadily.

• Smart Media Journal
• /
• 제10권1호
• /
• pp.39-46
• /
• 2021

This study is about a method of extracting a summary from a news article in consideration of the importance of each sentence constituting the article. We propose a method of calculating sentence importance by extracting the probabilities of topic sentence, similarity with article title and other sentences, and sentence position as characteristics that affect sentence importance. At this time, a hypothesis is established that the Topic Sentence will have a characteristic distinct from the general sentence, and a deep learning-based classification model is trained to obtain a topic sentence probability value for the input sentence. Also, using the pre-learned ELMo language model, the similarity between sentences is calculated based on the sentence vector value reflecting the context information and extracted as sentence characteristics. The topic sentence classification performance of the LSTM and BERT models was 93% accurate, 96.22% recall, and 89.5% precision, resulting in high analysis results. As a result of calculating the importance of each sentence by combining the extracted sentence characteristics, it was confirmed that the performance of extracting the topic sentence was improved by about 10% compared to the existing TextRank algorithm.

• Journal of The Korean Association of Information Education
• /
• 제26권5호
• /
• pp.439-448
• /
• 2022

This study aims to search for education-related datasets provided by public data portals and examine what data types are constructed through classification using topic modeling methods. Regarding the data of the public data portal, 3,072 cases of file data in the education field were collected based on the classification system. Text mining analysis was performed using the LDA-based topic modeling method with stopword processing and data pre-processing for each dataset. Program information and student-supporting notifications were usually provided in the pre-classified dataset for education from the data portal. On the other hand, the characteristics of educational programs and supporting information for the disabled, parents, the elderly, and children through the perspective of lifelong education were generally indicated in the dataset collected by searching for education. The results of data analysis through this study show that providing sufficient educational information through the public data portal would be better to help the students' data science-based decision-making and problem-solving skills.

• Journal of KIISE
• /
• 제42권4호
• /
• pp.512-521
• /
• 2015

Internet users purchase goods on the Internet and express their positive or negative emotions of the goods in product reviews. Analysis of the product reviews become critical data to both potential consumers and to the decision making of enterprises. Therefore, the importance of opinion mining techniques which derive opinions by analyzing meaningful data from large numbers of Internet reviews. Existing studies were mostly based on comments written in English, yet analysis in Korean has not actively been done. Unlike English, Korean has characteristics of complex adjectives and suffixes. Existing studies did not consider the characteristics of the Internet language. This study proposes an emotional classification method which increases the accuracy of emotional classification by analyzing the characteristics of the Internet language connoting feelings. We can classify positive and negative comments about products automatically using the Internet emoticon. Also we can check the validity of the proposed algorithm through the result of high precision, recall and coverage for the evaluation of this method.