• Title/Summary/Keyword: Ensemble methods

Search Result 289, Processing Time 0.022 seconds

Methodology on e-Navigation-Assisted Ocean Monitoring and Big Data Analysis (이내비게이션을 활용한 해양환경관측 및 빅데이터 분석방안)

  • LEE, GUAN-HONG;PARK, JAE-HUN;HA, HO KYUNG;KIM, DO WAN;LEE, WOOJOO;KIM, HONGTAE;SHIN, HYUN-JUNG
    • The Sea:JOURNAL OF THE KOREAN SOCIETY OF OCEANOGRAPHY
    • /
    • v.23 no.4
    • /
    • pp.204-217
    • /
    • 2018
  • This study proposes a cost-effective method to monitor coastal environments using e-Navigation-implemented domestic and international ferries, and to analyze big data of records such as wind, temperature, salinity, waves, and currents that are gathered through e-Navigation system. First, we present the concept and architecture of e-Navigation operation system based on the General Information Center on Maritime Safety and Security. Then, the marine observation system that can be applied to ferries operating in our nation's territory is discussed. Analytical methods, such as spatio-temporal mixed effects model, ensemble method, and meshfree method, in handling real-time big data obtained by the e-Navigation observing system are then explained in detail. This study will support the implementation of the Korean e-Navigation project that focuses on the safety of small vessels such as coasters and fishing vessels.

Distributed Processing System Design and Implementation for Feature Extraction from Large-Scale Malicious Code (대용량 악성코드의 특징 추출 가속화를 위한 분산 처리 시스템 설계 및 구현)

  • Lee, Hyunjong;Euh, Seongyul;Hwang, Doosung
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.8 no.2
    • /
    • pp.35-40
    • /
    • 2019
  • Traditional Malware Detection is susceptible for detecting malware which is modified by polymorphism or obfuscation technology. By learning patterns that are embedded in malware code, machine learning algorithms can detect similar behaviors and replace the current detection methods. Data must collected continuously in order to learn malicious code patterns that change over time. However, the process of storing and processing a large amount of malware files is accompanied by high space and time complexity. In this paper, an HDFS-based distributed processing system is designed to reduce space complexity and accelerate feature extraction time. Using a distributed processing system, we extract two API features based on filtering basis, 2-gram feature and APICFG feature and the generalization performance of ensemble learning models is compared. In experiments, the time complexity of the feature extraction was improved about 3.75 times faster than the processing time of a single computer, and the space complexity was about 5 times more efficient. The 2-gram feature was the best when comparing the classification performance by feature, but the learning time was long due to high dimensionality.

Fragility-based performance evaluation of mid-rise reinforced concrete frames in near field and far field earthquakes

  • Ansari, Mokhtar;Safiey, Amir;Abbasi, Mehdi
    • Structural Engineering and Mechanics
    • /
    • v.76 no.6
    • /
    • pp.751-763
    • /
    • 2020
  • Available records of recent earthquakes show that near-field earthquakes have different characteristics than far-field earthquakes. In general, most of these unique characteristics of near-fault records can be attributed to their forward directivity. This phenomenon causes the records of ground motion normal to the fault to entail pulses with long periods in the velocity time history. The energy of the earthquake is almost accumulated in these pulses causing large displacements and, accordingly, severe damages in the building. Damage to structures caused by past earthquakes raises the need to assess the chance of future earthquake damage. There are a variety of methods to evaluate building seismic vulnerabilities with different computational cost and accuracy. In the meantime, fragility curves, which defines the possibility of structural damage as a function of ground motion characteristics and design parameters, are more common. These curves express the percentage of probability that the structural response will exceed the allowable performance limit at different seismic intensities. This study aims to obtain the fragility curve for low- and mid-rise structures of reinforced concrete moment frames by incremental dynamic analysis (IDA). These frames were exposed to an ensemble of 18 ground motions (nine records near-faults and nine records far-faults). Finally, after the analysis, their fragility curves are obtained using the limit states provided by HAZUS-MH 2.1. The result shows the near-fault earthquakes can drastically influence the fragility curves of the 6-story building while it has a minimal impact on those of the 3-story building.

Hot Keyword Extraction of Sci-tech Periodicals Based on the Improved BERT Model

  • Liu, Bing;Lv, Zhijun;Zhu, Nan;Chang, Dongyu;Lu, Mengxin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.6
    • /
    • pp.1800-1817
    • /
    • 2022
  • With the development of the economy and the improvement of living standards, the hot issues in the subject area have become the main research direction, and the mining of the hot issues in the subject currently has problems such as a large amount of data and a complex algorithm structure. Therefore, in response to this problem, this study proposes a method for extracting hot keywords in scientific journals based on the improved BERT model.It can also provide reference for researchers,and the research method improves the overall similarity measure of the ensemble,introducing compound keyword word density, combining word segmentation, word sense set distance, and density clustering to construct an improved BERT framework, establish a composite keyword heat analysis model based on I-BERT framework.Taking the 14420 articles published in 21 kinds of social science management periodicals collected by CNKI(China National Knowledge Infrastructure) in 2017-2019 as the experimental data, the superiority of the proposed method is verified by the data of word spacing, class spacing, extraction accuracy and recall of hot keywords. In the experimental process of this research, it can be found that the method proposed in this paper has a higher accuracy than other methods in extracting hot keywords, which can ensure the timeliness and accuracy of scientific journals in capturing hot topics in the discipline, and finally pass Use information technology to master popular key words.

Research on Data Tuning Methods to Improve the Anomaly Detection Performance of Industrial Control Systems (산업제어시스템의 이상 탐지 성능 개선을 위한 데이터 보정 방안 연구)

  • JUN, SANGSO;Lee, Kyung-ho
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.4
    • /
    • pp.691-708
    • /
    • 2022
  • As the technology of machine learning and deep learning became common, it began to be applied to research on anomaly(abnormal) detection of industrial control systems. In Korea, the HAI dataset was developed and published to activate artificial intelligence research for abnormal detection of industrial control systems, and an AI contest for detecting industrial control system security threats is being conducted. Most of the anomaly detection studies have been to create a learning model with improved performance through the ensemble model method, which is applied either by modifying the existing deep learning algorithm or by applying it together with other algorithms. In this study, a study was conducted to improve the performance of anomaly detection with a post-processing method that detects abnormal data and corrects the labeling results, rather than the learning algorithm and data pre-processing process. Results It was confirmed that the results were improved by about 10% or more compared to the anomaly detection performance of the existing model.

Prediction Model of CNC Processing Defects Using Machine Learning (머신러닝을 이용한 CNC 가공 불량 발생 예측 모델)

  • Han, Yong Hee
    • Journal of the Korea Convergence Society
    • /
    • v.13 no.2
    • /
    • pp.249-255
    • /
    • 2022
  • This study proposed an analysis framework for real-time prediction of CNC processing defects using machine learning-based models that are recently attracting attention as processing defect prediction methods, and applied it to CNC machines. Analysis shows that the XGBoost, CatBoost, and LightGBM models have the same best accuracy, precision, recall, F1 score, and AUC, of which the LightGBM model took the shortest execution time. This short run time has practical advantages such as reducing actual system deployment costs, reducing the probability of CNC machine damage due to rapid prediction of defects, and increasing overall CNC machine utilization, confirming that the LightGBM model is the most effective machine learning model for CNC machines with only basic sensors installed. In addition, it was confirmed that classification performance was maximized when an ensemble model consisting of LightGBM, ExtraTrees, k-Nearest Neighbors, and logistic regression models was applied in situations where there are no restrictions on execution time and computing power.

Enhancing Medium-Range Forecast Accuracy of Temperature and Relative Humidity over South Korea using Minimum Continuous Ranked Probability Score (CRPS) Statistical Correction Technique (연속 순위 확률 점수를 활용한 통합 앙상블 모델에 대한 기온 및 습도 후처리 모델 개발)

  • Hyejeong Bok;Junsu Kim;Yeon-Hee Kim;Eunju Cho;Seungbum Kim
    • Atmosphere
    • /
    • v.34 no.1
    • /
    • pp.23-34
    • /
    • 2024
  • The Korea Meteorological Administration has improved medium-range weather forecasts by implementing post-processing methods to minimize numerical model errors. In this study, we employ a statistical correction technique known as the minimum continuous ranked probability score (CRPS) to refine medium-range forecast guidance. This technique quantifies the similarity between the predicted values and the observed cumulative distribution function of the Unified Model Ensemble Prediction System for Global (UM EPSG). We evaluated the performance of the medium-range forecast guidance for surface air temperature and relative humidity, noting significant enhancements in seasonal bias and root mean squared error compared to observations. Notably, compared to the existing the medium-range forecast guidance, temperature forecasts exhibit 17.5% improvement in summer and 21.5% improvement in winter. Humidity forecasts also show 12% improvement in summer and 23% improvement in winter. The results indicate that utilizing the minimum CRPS for medium-range forecast guidance provide more reliable and improved performance than UM EPSG.

Development of AI-based Smart Agriculture Early Warning System

  • Hyun Sim;Hyunwook Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.12
    • /
    • pp.67-77
    • /
    • 2023
  • This study represents an innovative research conducted in the smart farm environment, developing a deep learning-based disease and pest detection model and applying it to the Intelligent Internet of Things (IoT) platform to explore new possibilities in the implementation of digital agricultural environments. The core of the research was the integration of the latest ImageNet models such as Pseudo-Labeling, RegNet, EfficientNet, and preprocessing methods to detect various diseases and pests in complex agricultural environments with high accuracy. To this end, ensemble learning techniques were applied to maximize the accuracy and stability of the model, and the model was evaluated using various performance indicators such as mean Average Precision (mAP), precision, recall, accuracy, and box loss. Additionally, the SHAP framework was utilized to gain a deeper understanding of the model's prediction criteria, making the decision-making process more transparent. This analysis provided significant insights into how the model considers various variables to detect diseases and pests.

Development of Type 2 Prediction Prediction Based on Big Data (빅데이터 기반 2형 당뇨 예측 알고리즘 개발)

  • Hyun Sim;HyunWook Kim
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.18 no.5
    • /
    • pp.999-1008
    • /
    • 2023
  • Early prediction of chronic diseases such as diabetes is an important issue, and improving the accuracy of diabetes prediction is especially important. Various machine learning and deep learning-based methodologies are being introduced for diabetes prediction, but these technologies require large amounts of data for better performance than other methodologies, and the learning cost is high due to complex data models. In this study, we aim to verify the claim that DNN using the pima dataset and k-fold cross-validation reduces the efficiency of diabetes diagnosis models. Machine learning classification methods such as decision trees, SVM, random forests, logistic regression, KNN, and various ensemble techniques were used to determine which algorithm produces the best prediction results. After training and testing all classification models, the proposed system provided the best results on XGBoost classifier with ADASYN method, with accuracy of 81%, F1 coefficient of 0.81, and AUC of 0.84. Additionally, a domain adaptation method was implemented to demonstrate the versatility of the proposed system. An explainable AI approach using the LIME and SHAP frameworks was implemented to understand how the model predicts the final outcome.

A Bi-directional Information Learning Method Using Reverse Playback Video for Fully Supervised Temporal Action Localization (완전지도 시간적 행동 검출에서 역재생 비디오를 이용한 양방향 정보 학습 방법)

  • Huiwon Gwon;Hyejeong Jo;Sunhee Jo;Chanho Jung
    • Journal of IKEEE
    • /
    • v.28 no.2
    • /
    • pp.145-149
    • /
    • 2024
  • Recently, research on temporal action localization has been actively conducted. In this paper, unlike existing methods, we propose two approaches for learning bidirectional information by creating reverse playback videos for fully supervised temporal action localization. One approach involves creating training data by combining reverse playback videos and forward playback videos, while the other approach involves training separate models on videos with different playback directions. Experiments were conducted on the THUMOS-14 dataset using TALLFormer. When using both reverse and forward playback videos as training data, the performance was 5.1% lower than that of the existing method. On the other hand, using a model ensemble shows a 1.9% improvement in performance.