• Title/Summary/Keyword: pre-processing step

Search Result 140, Processing Time 0.023 seconds

Supervised Rank Normalization for Support Vector Machines (SVM을 위한 교사 랭크 정규화)

  • Lee, Soojong;Heo, Gyeongyong
    • Journal of the Korea Society of Computer and Information
    • /
    • v.18 no.11
    • /
    • pp.31-38
    • /
    • 2013
  • Feature normalization as a pre-processing step has been widely used in classification problems to reduce the effect of different scale in each feature dimension and error as a result. Most of the existing methods, however, assume some distribution function on feature distribution. Even worse, existing methods do not use the labels of data points and, as a result, do not guarantee the optimality of the normalization results in classification. In this paper, proposed is a supervised rank normalization which combines rank normalization and a supervised learning technique. The proposed method does not assume any feature distribution like rank normalization and uses class labels of nearest neighbors in classification to reduce error. SVM, in particular, tries to draw a decision boundary in the middle of class overlapping zone, the reduction of data density in that area helps SVM to find a decision boundary reducing generalized error. All the things mentioned above can be verified through experimental results.

Cleaning Noises from Time Series Data with Memory Effects

  • Cho, Jae-Han;Lee, Lee-Sub
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.4
    • /
    • pp.37-45
    • /
    • 2020
  • The development process of deep learning is an iterative task that requires a lot of manual work. Among the steps in the development process, pre-processing of learning data is a very costly task, and is a step that significantly affects the learning results. In the early days of AI's algorithm research, learning data in the form of public DB provided mainly by data scientists were used. The learning data collected in the real environment is mostly the operational data of the sensors and inevitably contains various noises. Accordingly, various data cleaning frameworks and methods for removing noises have been studied. In this paper, we proposed a method for detecting and removing noises from time-series data, such as sensor data, that can occur in the IoT environment. In this method, the linear regression method is used so that the system repeatedly finds noises and provides data that can replace them to clean the learning data. In order to verify the effectiveness of the proposed method, a simulation method was proposed, and a method of determining factors for obtaining optimal cleaning results was proposed.

Impact Analysis of Deep Learning Super-resolution Technology for Improving the Accuracy of Ship Detection Based on Optical Satellite Imagery (광학 위성 영상 기반 선박탐지의 정확도 개선을 위한 딥러닝 초해상화 기술의 영향 분석)

  • Park, Seongwook;Kim, Yeongho;Kim, Minsik
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.5_1
    • /
    • pp.559-570
    • /
    • 2022
  • When a satellite image has low spatial resolution, it is difficult to detect small objects. In this research, we aim to check the effect of super resolution on object detection. Super resolution is a software method that increases the resolution of an image. Unpaired super resolution network is used to improve Sentinel-2's spatial resolution from 10 m to 3.2 m. Faster-RCNN, RetinaNet, FCOS, and S2ANet were used to detect vessels in the Sentinel-2 images. We experimented the change in vessel detection performance when super resolution is applied. As a result, the Average Precision (AP) improved by at least 12.3% and up to 33.3% in the ship detection models trained with the super-resolution image. False positive and false negative cases also decreased. This implies that super resolution can be an important pre-processing step in object detection, and it is expected to greatly contribute to improving the accuracy of other image-based deep learning technologies along with object detection.

Fault Diagnosis of PV String Using Deep-Learning and I-V Curves (딥러닝과 I-V 곡선을 이용한 태양광 스트링 고장 진단)

  • Shin, Woo Gyun;Oh, Hyun Gyu;Bae, Soo Hyun;Ju, Young Chul;Hwang, Hye Mi;Ko, Suk Whan
    • Current Photovoltaic Research
    • /
    • v.10 no.3
    • /
    • pp.77-83
    • /
    • 2022
  • Renewable energy is receiving attention again as a way to realize carbon neutrality to overcome the climate change crisis. Among renewable energy sources, the installation of Photovoltaic is continuously increasing, and as of 2020, the global cumulative installation amount is about 590 GW and the domestic cumulative installation amount is about 17 GW. Accordingly, O&M technology that can analyze the power generation and fault diagnose about PV plants the is required. In this paper, a study was conducted to diagnose fault using I-V curves of PV strings and deep learning. In order to collect the fault I-V curves for learning in the deep learning, faults were simulated. It is partial shade and voltage mismatch, and I-V curves were measured on a sunny day. A two-step data pre-processing technique was applied to minimize variations depending on PV string capacity, irradiance, and PV module temperature, and this was used for learning and validation of deep learning. From the results of the study, it was confirmed that the PV fault diagnosis using I-V curves and deep learning is possible.

An Algorithm For Approximating The Performance of Multi-mode Network System Using Algebraic Property of System States (시스템 상태의 대수적 성질을 이용한 다중모드 네트워크 시스템 성능 근사계산 알고리즘)

  • Oh, Dae-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.12
    • /
    • pp.127-137
    • /
    • 2009
  • A practical algorithm of generating most probable states in decreasing order of probability of the network system state is suggested for approximating the performance of multi-mode network system using algebraic structure of the system states. Most complex system having network structure with multi-mode unit state is difficult to evaluate the performance or reliability due to exponentially increasing size of state space. Hence not an exact computing method but an approximated one is reasonable approach to solve the problem. To achieve the goal we should enumerate the network system states in order as a pre-processing step. In this paper, we suggest an improved algorithm of generating most probable multi-mode states to get the ordered system states efficiently. The method is compared with the previous algorithms in respective to memory requirement and empirical computing time. From the experiment proposed method has some advantages with regard to the criterion of algorithm performance. We investigate the advantages and disadvantage by illustrating experiment examples.

An Improved Area Edge Detection for Real-time Image Processing (실시간 영상 처리를 위한 향상된 영역 경계 검출)

  • Kim, Seung-Hee;Nam, Si-Byung;Lim, Hae-Jin
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.1
    • /
    • pp.99-106
    • /
    • 2009
  • Though edge detection, an important stage that significantly affecting the performance of image recognition, has been given numerous researches on its execution methods, it still remains as difficult problem and it is one of the components for image recognition applications while it is not the only way to identify an object or track a specific area. This paper, unlike gradient operator using edge detection method, found out edge pixel by referring to 2 neighboring pixels information in binary image and comparing them with pre-defined 4 edge pixels pattern, and detected binary image edge by determining the direction of the next edge detection exploring pixel and proposed method to detect binary image edge by repeating step of edge detection to detect another area edge. When recognizing image, if edge is detected with the use of gradient operator, thinning process, the stage next to edge detection, can be omitted, and with the edge detection algorithm executing time reduced compared with existing area edge tracing method, the entire image recognizing time can be reduced by applying real-time image recognizing system.

A Robust Staff Line Height and Staff Line Space Estimation for the Preprocessing of Music Score Recognition (악보인식 전처리를 위한 강건한 오선 두께와 간격 추정 방법)

  • Na, In-Seop;Kim, Soo-Hyung;Nquyen, Trung Quy
    • Journal of Internet Computing and Services
    • /
    • v.16 no.1
    • /
    • pp.29-37
    • /
    • 2015
  • In this paper, we propose a robust pre-processing module for camera-based Optical Music Score Recognition (OMR) on mobile device. The captured images likely suffer for recognition from many distortions such as illumination, blur, low resolution, etc. Especially, the complex background music sheets recognition are difficult. Through any symbol recognition system, the staff line height and staff line space are used many times and have a big impact on recognition module. A robust and accurate staff line height and staff line space are essential. Some staff line height and staff line space are proposed for binary image. But in case of complex background music sheet image, the binarization results from common binarization algorithm are not satisfactory. It can cause incorrect staff line height and staff line space estimation. We propose a robust staff line height and staff line space estimation by using run-length encoding technique on edge image. Proposed method is composed of two steps, first step, we conducted the staff line height and staff line space estimation based on edge image using by Sobel operator on image blocks. Each column of edge image is encoded by run-length encoding algorithm Second step, we detect the staff line using by Stable Path algorithm and removal the staff line using by adaptive Line Track Height algorithm which is to track the staff lines positions. The result has shown that robust and accurate estimation is possible even in complex background cases.

Optimization of Input Features for Vegetation Classification Based on Random Forest and Sentinel-2 Image (랜덤포레스트와 Sentinel-2를 이용한 식생 분류의 입력특성 최적화)

  • LEE, Seung-Min;JEONG, Jong-Chul
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.23 no.4
    • /
    • pp.52-67
    • /
    • 2020
  • Recently, the Arctic has been exposed to snow-covered land due to melting permafrost every year, and the Korea Geographic Information Institute(NGII) provides polar spatial information service by establishing spatial information of the polar region. However, there is a lack of spatial information on vegetation sensitive to climate change. This research used a multi-temporal Sentinel-2 image to perform land cover classification of the Ny-Ålesund in Arctic Svalbard. In the pre-processing step, 10 bands and 6 vegetation spectral index were generated from multi-temporal Sentinel-2 images. In image-classification step is consisted of extracting the vegetation area through 8-class land cover classification and performing the vegetation species classification. The image classification algorithm used Random Forest to evaluate the accuracy and calculate feature importance through Out-Of-Bag(OOB). To identify the advantages of multi- temporary Sentinel-2 for vegetation classification, the overall accuracy was compared according to the number of images stacked and vegetation spectral index. Overall accuracy was 77% when using single-time Sentinel-2 images, but improved to 81% when using multi-time Sentinel-2 images. In addition, the overall accuracy improved to about 83% in learning when the vegetation index was used additionally. The most important spectral variables to distinguish between vegetation classes are located in the Red, Green, and short wave infrared-1(SWIR1). This research can be used as a basic study that optimizes input characteristics in performing the classification of vegetation in the polar regions.

Artificial Intelligence-based Security Control Construction and Countermeasures (인공지능기반 보안관제 구축 및 대응 방안)

  • Hong, Jun-Hyeok;Lee, Byoung Yup
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.1
    • /
    • pp.531-540
    • /
    • 2021
  • As cyber attacks and crimes increase exponentially and hacking attacks become more intelligent and advanced, hacking attack methods and routes are evolving unpredictably and in real time. In order to reinforce the enemy's responsiveness, this study aims to propose a method for developing an artificial intelligence-based security control platform by building a next-generation security system using artificial intelligence to respond by self-learning, monitoring abnormal signs and blocking attacks.The artificial intelligence-based security control platform should be developed as the basis for data collection, data analysis, next-generation security system operation, and security system management. Big data base and control system, data collection step through external threat information, data analysis step of pre-processing and formalizing the collected data to perform positive/false detection and abnormal behavior analysis through deep learning-based algorithm, and analyzed data Through the operation of a security system of prevention, control, response, analysis, and organic circulation structure, the next generation security system to increase the scope and speed of handling new threats and to reinforce the identification of normal and abnormal behaviors, and management of the security threat response system, Harmful IP management, detection policy management, security business legal system management. Through this, we are trying to find a way to comprehensively analyze vast amounts of data and to respond preemptively in a short time.

Microbiological Evaluation for HACCP System Application of Green Vegetable Juice Containing Lactic Acid Bacteria (유산균을 함유한 녹즙의 HACCP 시스템 적용을 위한 미생물학적 위해도 평가)

  • Kwon, Sang-Chul
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.12 no.11
    • /
    • pp.4924-4931
    • /
    • 2011
  • This research performed to evaluate a production processes reporting by the HACCP system of green vegetable juice products, containing lactic acid bacteria, stage of processing raw materials agricultural products and production facilities of general bacteria and pathogenic micro organism. General bacteria are found from four samples of storage of agricultural products at process stage and water was detected 8.67~14.67 CFU/ml. However, all samples were detected less than 105 CFU/ml as a legal standards after the process of UV sterilization. For the outcome of experiment of E.coli, E.coli O157:H7, B.cereus, L.moonocytogenes, Salmonella spp, Staph.aureus as the food poisoning bacterial, E.coli was detected until UV pre-step process in storage process and B.cereus was detected partly till 1st washing. Since all bacterial, Yeast and Mold are detected in main materials, pre-control method is a necessary to establish for decreasing with a number of initial bacteria of main materials and it is considered to establish the effective ways of washing and sterilization such as production facilities for cross contamination prevention of bacteria and Sthaphylococcus. Based on above results, the process of UV sterilization should be managed with CCP as an important process to reduce or eliminate the general and food poisoning bacterial of green vegetable juice products, including lactic acid bacteria. Therefore, it is considered to need an exhaustive HACCP plan such as control manual of UV sterilization, solution method, verification, education and training and record management.