• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.11
• /
• pp.521-526
• /
• 2016

As on-line informal text data have massive in its volume and have unstructured characteristics in nature, there are limitations in applying traditional relational data model technologies for data storage and data analysis jobs. Moreover, using dynamically generating massive social data, social user's real-time reaction analysis tasks is hard to accomplish. In the paper, to capture easily the semantics of massive and informal on-line documents with unsupervised learning mechanism, we design and implement automatic topic extraction systems according to the mass of the words that consists a document. The input data set to the proposed system are generated first, using N-gram algorithm to build multiple words to capture the meaning of the sentences precisely, and Hadoop and Spark (In-memory distributed computing framework) are adopted to run topic model. In the experiment phases, TB level input data are processed for data preprocessing and proposed topic extraction steps are applied. We conclude that the proposed system shows good performance in extracting meaningful topics in time as the intermediate results come from main memories directly instead of an HDD reading.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.32 no.5
• /
• pp.933-943
• /
• 2022

As the Internet develops and the utilization rate of computers increases, the threats posed by malware keep increasing. This leads to the demand for a system to automatically analyzes a large amount of malware. In this paper, an automatic malware analysis technique using a deep learning algorithm is introduced. Our proposed method uses CNN (Convolutional Neural Network) to analyze the malicious features represented as images. To reflect semantic information of malware for detection, our method uses the opcode frequency data of binary for image generation, rather than using bytes of binary. As a result of the experiments using the datasets consisting of 20,000 samples, it was found that the proposed method can detect malicious codes with 91% accuracy.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.2
• /
• pp.263-283
• /
• 2024

Users' comments after online shopping are critical to product reputation and business improvement. These comments, sometimes known as e-commerce reviews, influence other customers' purchasing decisions. To confront large amounts of e-commerce reviews, automatic analysis based on machine learning and deep learning draws more and more attention. A core task therein is sentiment analysis. However, the e-commerce reviews exhibit the following characteristics: (1) inconsistency between comment content and the star rating; (2) a large number of unlabeled data, i.e., comments without a star rating, and (3) the data imbalance caused by the sparse negative comments. This paper employs Bidirectional Encoder Representation from Transformers (BERT), one of the best natural language processing models, as the base model. According to the above data characteristics, we propose the F_MixBERT framework, to more effectively use inconsistently low-quality and unlabeled data and resolve the problem of data imbalance. In the framework, the proposed MixBERT incorporates the MixMatch approach into BERT's high-dimensional vectors to train the unlabeled and low-quality data with generated pseudo labels. Meanwhile, data imbalance is resolved by Focal loss, which penalizes the contribution of large-scale data and easily-identifiable data to total loss. Comparative experiments demonstrate that the proposed framework outperforms BERT and MixBERT for sentiment analysis of e-commerce comments.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.11
• /
• pp.527-534
• /
• 2016

The emergent information society requires the talent for creative thinking based on problem-solving skills and comprehensive thinking rather than simple memorization. Therefore, the Korean curriculum has also changed into the direction of the creative thinking through increasing short-answer questions that can determine the overall thinking of the students. However, their scoring results are a little bit inconsistency because scoring short-answer questions depends on the subjective scoring of human raters. In order to alleviate this point, an automated scoring system using a machine learning has been used as a scoring tool in overseas. Linguistically, Korean and English is totally different in the structure of the sentences. Thus, the automated scoring system used in English cannot be applied to Korean. In this paper, we introduce an automated scoring system for Korean short-answer questions using predictability and unanimity. We also verify the practicality of the automatic scoring system through the correlation coefficient between the results of the automated scoring system and those of human raters. In the experiment of this paper, the proposed system is evaluated for constructed-response items of Korean language, social studies, and science in the National Assessment of Educational Achievement. The analysis was used Pearson correlation coefficients and Kappa coefficient. Results of the experiment had showed a strong positive correlation with all the correlation coefficients at 0.7 or higher. Thus, the scoring results of the proposed scoring system are similar to those of human raters. Therefore, the automated scoring system should be found to be useful as a scoring tool.

• Korean Chemical Engineering Research
• /
• v.59 no.2
• /
• pp.209-218
• /
• 2021

First principle-based modeling studies have been performed to improve the heat exchange efficiency of ORV and optimize operation, but the heat transfer coefficient of ORV is an irregular system according to time and location, and it undergoes a complex modeling process. In this study, FNN, LSTM, and AutoML-based modeling were performed to confirm the effectiveness of data-based modeling for complex systems. The prediction accuracy indicated high performance in the order of LSTM > AutoML > FNN in MSE. The performance of AutoML, an automatic design method for machine learning models, was superior to developed FNN, and the total time required for model development was 1/15 compared to LSTM, showing the possibility of using AutoML. The prediction of NG and seawater discharged temperatures using LSTM and AutoML showed an error of less than 0.5K. Using the predictive model, real-time optimization of the amount of LNG vaporized that can be processed using ORV in winter is performed, confirming that up to 23.5% of LNG can be additionally processed, and an ORV optimal operation guideline based on the developed dynamic prediction model was presented.

• Journal of Bio-Environment Control
• /
• v.31 no.1
• /
• pp.1-7
• /
• 2022

Strawberry is a stand-out cultivating fruit in Korea. The optimum production of strawberry is highly dependent on growing environment. Smart farm technology, and automatic monitoring and control system maintain a favorable environment for strawberry growth in greenhouses, as well as play an important role to improve production. Moreover, physiological parameters of strawberry plant and it is surrounding environment may allow to give an idea on production of strawberry. Therefore, this study intends to build a machine learning model to predict strawberry's yield, cultivated in greenhouse. The environmental parameter like as temperature, humidity and CO₂ and physiological parameters such as length of leaves, number of flowers and fruits and chlorophyll content of 'Seolhyang' (widely growing strawberry cultivar in Korea) were collected from three strawberry greenhouses located in Sacheon of Gyeongsangnam-do during the period of 2019-2020. A predictive model, Lasso regression was designed and validated through 5-fold cross-validation. The current study found that performance of the Lasso regression model is good to predict the number of flowers and fruits, when the MAPE value are 0.511 and 0.488, respectively during the model validation. Overall, the present study demonstrates that using AI based regression model may be convenient for farms and agricultural companies to predict yield of crops with fewer input attributes.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2023.05a
• /
• pp.100-103
• /
• 2023

Recently, development of maritime autonomoust surface ships and eco-friendly ships, production and evaluation research considering various marine environments is needed in the field of optimal routes as the demand for accurate and detailed real-time marine environment prediction information expands. An algorithm that can calculate the optimal route while reducing the risk of the marine environment and uncertainty in energy consumption in smart ships was developed in 2 stages. In the first stage, a profile was created by combining marine environmental information with ship location and status information within the Automatic Ship Identification System(AIS). In the second stage, a model was developed that could define the marine environment energy map using the configured profile results, A regression equation was generated by applying Random Forest among machine learning techniques to reflect about 600,000 data. The Random Forest coefficient of determination (R²) was 0.89, showing very high reliability. The Dijikstra shortest path algorithm was applied to the marine environment prediction at June 1 to 3, 2021, and to calculate the optimal safety route and express it on the map. The route calculated by the random forest regression model was streamlined, and the route was derived considering the state of the marine environment prediction information. The concept of route calculation based on real-time marine environment prediction information in this study is expected to be able to calculate a realistic and safe route that reflects the movement tendency of ships, and to be expanded to a range of economic, safety, and eco-friendliness evaluation models in the future.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.26 no.3
• /
• pp.182-189
• /
• 2016

For high accuracy of forecast using numerical weather prediction models, we need to get weather observation data that are large and high dense. Korea Meteorological Administration (KMA) mantains Automatic Weather Stations (AWSs) to get weather observation data, but their installation and maintenance costs are high. Mini-AWS is a very compact automatic weather station that can measure and record temperature, humidity, and pressure. In contrast to AWS, costs of Mini-AWS's installation and maintenance are low. It also has a little space restraints for installing. So it is easier than AWS to install mini-AWS on places where we want to get weather observation data. But we cannot use the data observed from Mini-AWSs directly, because it can be affected by surrounding. In this paper, we suggest a correcting method for using pressure data observed from Mini-AWS as weather observation data. We carried out preconditioning process on pressure data from Mini-AWS. Then they were corrected by using machine learning methods with the aim of adjusting to pressure data of the AWS closest to them. Our experimental results showed that corrected pressure data are in regulation and our correcting method using SVR showed very good performance.

• Journal of IKEEE
• /
• v.8 no.2 s.15
• /
• pp.216-223
• /
• 2004

The customer's bulletin board is the important channel to get opinions from customers directly. The effective management of the bulletin board for the customer improves the reliance by providing the best replies and by accepting opinions of the customer and furthermore, that can raise the customer's reliance of the whole shopping mall is the important eCRM method. But, the present mostly customer's bulletin board is been replied without any classifying about many kinds of question. Consequently, The shopping mall should do systematic management of the best professional reply about many kinds of question. In order to resolve this problem, we implement a classifier called Naive Bayesian classifier is classified automatically bulletin board for eCRM of shopping mall.

• Journal of KIISE
• /
• v.44 no.1
• /
• pp.51-56
• /
• 2017

The research area for solving question answering (QA) problems using artificial intelligence models is in a methodological transition period, and one such architecture, the dynamic memory network (DMN), is drawing attention for two key attributes: its attention mechanism defined by neural network operations and its modular architecture imitating cognition processes during QA of human. In this paper, we increased accuracy of the inferred answers, by adapting an automatic data augmentation method for lacking amount of training data, and by improving the ability of time perception. The experimental results showed that in the 1K-bAbI tasks, the modified DMN achieves 89.21% accuracy and passes twelve tasks which is 13.58% higher with passing four more tasks, as compared with one implementation of DMN. Additionally, DMN's word embedding vectors form strong clusters after training. Moreover, the number of episodic passes and that of supporting facts shows direct correlation, which affects the performance significantly.