• 제목/요약/키워드: Data Collection and Preprocessing

Search Result 59, Processing Time 0.035 seconds

For Improving Security Log Big Data Analysis Efficiency, A Firewall Log Data Standard Format Proposed (보안로그 빅데이터 분석 효율성 향상을 위한 방화벽 로그 데이터 표준 포맷 제안)

  • Bae, Chun-sock;Goh, Sung-cheol
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.1
    • /
    • pp.157-167
    • /
    • 2020
  • The big data and artificial intelligence technology, which has provided the foundation for the recent 4th industrial revolution, has become a major driving force in business innovation across industries. In the field of information security, we are trying to develop and improve an intelligent security system by applying these techniques to large-scale log data, which has been difficult to find effective utilization methods before. The quality of security log big data, which is the basis of information security AI learning, is an important input factor that determines the performance of intelligent security system. However, the difference and complexity of log data by various product has a problem that requires excessive time and effort in preprocessing big data with poor data quality. In this study, we research and analyze the cases related to log data collection of various firewall. By proposing firewall log data collection format standard, we hope to contribute to the development of intelligent security systems based on security log big data.

Automatic Correlation Generation using the Alternating Conditional Expectation Algorithm

  • Kim, Han-Gon;Kim, Byong-Sup;Cho, Sung-Jae
    • Proceedings of the Korean Nuclear Society Conference
    • /
    • 1997.05a
    • /
    • pp.292-297
    • /
    • 1997
  • An alternating conditional expectation (ACE) algorithm, a kind of non-parametric regression method, is proposed to generate empirical correlations automatically. The ACE algorithm yields an optimal relationship between a dependent variable and multiple independent variables without any preprocessing and initial assumption on the functional forms. This algorithm is applied to a collection of 12,879 CHF data points for forced convective boiling hi vertical tubes to develop a new critical heat flux (CHF) correlation. The meat root mean square, and maximum errors of our new correlation are -0.558%, 12.5%, and 122.6%, respectively. Our CHF correlation represents the entire set of CHF data with an overall accuracy equivalent to or better than that of three existing correlations.

  • PDF

A Proposal for Processor for Improved Utilization of High resolution Satellite Images

  • Choi, Kyeong-Hwan;Kim, Sung-Jae;Jo, Yun-Won;Jo, Myung-Hee
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.211-214
    • /
    • 2007
  • With the recent development of spatial information technology, the relative importance of satellite image contents has increased to about 62%, the techniques related to satellite images have improved, and their demand is gradually increasing. Accordingly, a standard processing method for the whole process of collection from satellites to distribution of satellite images is required in many countries for efficient distribution of images and improvement of their utilization. This study presents the processor standardization technique for the preprocessing of satellite images including geometric correction, orthorectification, color adjustment, interpolation for DEM (Digital Elevation Model) production, rearrangement, and image data management, which will standardize the subjective, complex process and improve their utilization by making it easy for general users to use them

  • PDF

Pig Face Recognition Using Deep Learning (딥러닝을 이용한 돼지 얼굴 인식)

  • MA, RUIHAN;Kim, Sang-Cheol
    • Annual Conference of KIPS
    • /
    • 2022.11a
    • /
    • pp.493-494
    • /
    • 2022
  • The development of livestock faces intensive farming results in a rising need for recognition of individual animals such as cows and pigs is related to high traceability. In this paper, we present a non-invasive biometrics systematic approach based on the deep-learning classification model to pig face identification. Firstly, in our systematic method, we build a ROS data collection system block to collect 10 pig face data images. Secondly, we proposed a preprocessing block in that we utilize the SSIM method to filter some images of collected images that have high similarity. Thirdly, we employ the improved image classification model of CNN (ViT), which uses the finetuning and pretraining technique to recognize the individual pig face. Finally, our proposed method achieves the accuracy about 98.66%.

Big Data Patent Analysis Using Social Network Analysis (키워드 네트워크 분석을 이용한 빅데이터 특허 분석)

  • Choi, Ju-Choel
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.2
    • /
    • pp.251-257
    • /
    • 2018
  • As the use of big data is necessary for increasing business value, the size of the big data market is getting bigger. Accordingly, it is important to apply competitive patents in order to gain the big data market. In this study, we conducted the patent analysis based keyword network to analyze the trend of big data patents. The analysis procedure consists of big data collection and preprocessing, network construction, and network analysis. The results of the study are as follows. Most of big data patents are related to data processing and analysis, and the keywords with high degree centrality and between centrality are "analysis", "process", "information", "data", "prediction", "server", "service", and "construction". we expect that the results of this study will offer useful information in applying big data patent.

Modeling Differential Global Positioning System Pseudorange Correction

  • Mohasseb, M.;El-Rabbany, A.;El-Alim, O. Abd;Rashad, R.
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • v.1
    • /
    • pp.21-26
    • /
    • 2006
  • This paper focuses on modeling and predicting differential GPS corrections transmitted by marine radio-beacon systems using artificial neural networks. Various neural network structures with various training algorithms were examined, including Linear, Radial Biases, and Feedforward. Matlab Neural Network toolbox is used for this purpose. Data sets used in building the model are the transmitted pseudorange corrections and broadcast navigation message. Model design is passed through several stages, namely data collection, preprocessing, model building, and finally model validation. It is found that feedforward neural network with automated regularization is the most suitable for our data. In training the neural network, different approaches are used to take advantage of the pseudorange corrections history while taking into account the required time for prediction and storage limitations. Three data structures are considered in training the neural network, namely all round, compound, and average. Of the various data structures examined, it is found that the average data structure is the most suitable. It is shown that the developed model is capable of predicting the differential correction with an accuracy level comparable to that of beacon-transmitted real-time DGPS correction.

  • PDF

Design of Low Complexity Human Anxiety Classification Model based on Machine Learning (기계학습 기반 저 복잡도 긴장 상태 분류 모델)

  • Hong, Eunjae;Park, Hyunggon
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.66 no.9
    • /
    • pp.1402-1408
    • /
    • 2017
  • Recently, services for personal biometric data analysis based on real-time monitoring systems has been increasing and many of them have focused on recognition of emotions. In this paper, we propose a classification model to classify anxiety emotion using biometric data actually collected from people. We propose to deploy the support vector machine to build a classification model. In order to improve the classification accuracy, we propose two data pre-processing procedures, which are normalization and data deletion. The proposed algorithms are actually implemented based on Real-time Traffic Flow Measurement structure, which consists of data collection module, data preprocessing module, and creating classification model module. Our experiment results show that the proposed classification model can infers anxiety emotions of people with the accuracy of 65.18%. Moreover, the proposed model with the proposed pre-processing techniques shows the improved accuracy, which is 78.77%. Therefore, we can conclude that the proposed classification model based on the pre-processing process can improve the classification accuracy with lower computation complexity.

Development of Customer Review Ranking Model Considering Product and Service Aspects Using Random Forest Regression Method

  • Arif Djunaidy;Nisrina Fadhilah Fano
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.8
    • /
    • pp.2137-2156
    • /
    • 2024
  • Customer reviews are the second-most reliable source of information, followed by family and friend referrals. However, there are many existing customer reviews. Some online shopping platforms address this issue by ranking customer reviews according to their usefulness. However, we propose an alternative method to rank customer reviews, given that this system is easily manipulable. This study aims to create a ranking model for reviews based on their usefulness by combining product and seller service aspects from customer reviews. This methodology consists of six primary steps: data collection and preprocessing, aspect extraction and sentiment analysis, followed by constructing a regression model using random forest regression, and the review ranking process. The results demonstrate that the ranking model with service considerations outperformed the model without service considerations. This demonstrates the model's superiority in the three tests, which include a comparison of the regression results, the aggregate helpfulness ratio, and the matching score.

A Light-weight ANN-based Hand Motion Recognition Using a Wearable Sensor (웨어러블 센서를 활용한 경량 인공신경망 기반 손동작 인식기술)

  • Lee, Hyung Gyu
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.17 no.4
    • /
    • pp.229-237
    • /
    • 2022
  • Motion recognition is very useful for implementing an intuitive HMI (Human-Machine Interface). In particular, hands are the body parts that can move most precisely with relatively small portion of energy. Thus hand motion has been used as an efficient communication interface with other persons or machines. In this paper, we design and implement a light-weight ANN (Artificial Neural Network)-based hand motion recognition using a state-of-the-art flex sensor. The proposed design consists of data collection from a wearable flex sensor, preprocessing filters, and a light-weight NN (Neural Network) classifier. For verifying the performance and functionality of the proposed design, we implement it on a low-end embedded device. Finally, our experiments and prototype implementation demonstrate that the accuracy of the proposed hand motion recognition achieves up to 98.7%.

Development of Career Exploration Program for Student Athletes : Focusing on Artificial Intelligence and Big Data Fields (운동선수부 학생을 위한 진로탐구 프로그램 개발 : 인공지능과 빅데이터 분야를 중심으로)

  • Kangsoo You
    • Journal of Practical Engineering Education
    • /
    • v.15 no.2
    • /
    • pp.401-408
    • /
    • 2023
  • In this study, a career exploration program was developed for athletic students. Therefore, existing research on career exploration for athletics was analyzed, requirements were identified, and a learning plan was designed. Based on this, a step-by-step educational program was developed. In addition, since research on career exploration for athletic students was not active in previous studies, 'problem definition' - 'data collection' - 'data preprocessing' - 'data analysis' by referring to existing career exploration studies that were studied in the school field. - 'Data visualization' - 'Simulation analysis' were divided into stages to conduct the study. Through this study, it is expected that research on vocational education for athletic students will be more active.