• Title/Summary/Keyword: Outlier elimination

Search Result 16, Processing Time 0.017 seconds

Relevancy contemplation in medical data analytics and ranking of feature selection algorithms

  • P. Antony Seba;J. V. Bibal Benifa
    • ETRI Journal
    • /
    • v.45 no.3
    • /
    • pp.448-461
    • /
    • 2023
  • This article performs a detailed data scrutiny on a chronic kidney disease (CKD) dataset to select efficient instances and relevant features. Data relevancy is investigated using feature extraction, hybrid outlier detection, and handling of missing values. Data instances that do not influence the target are removed using data envelopment analysis to enable reduction of rows. Column reduction is achieved by ranking the attributes through feature selection methodologies, namely, extra-trees classifier, recursive feature elimination, chi-squared test, analysis of variance, and mutual information. These methodologies are ranked via Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) using weight optimization to identify the optimal features for model building from the CKD dataset to facilitate better prediction while diagnosing the severity of the disease. An efficient hybrid ensemble and novel similarity-based classifiers are built using the pruned dataset, and the results are thereafter compared with random forest, AdaBoost, naive Bayes, k-nearest neighbors, and support vector machines. The hybrid ensemble classifier yields a better prediction accuracy of 98.31% for the features selected by extra tree classifier (ETC), which is ranked as the best by TOPSIS.

Camera pose estimation framework for array-structured images

  • Shin, Min-Jung;Park, Woojune;Kim, Jung Hee;Kim, Joonsoo;Yun, Kuk-Jin;Kang, Suk-Ju
    • ETRI Journal
    • /
    • v.44 no.1
    • /
    • pp.10-23
    • /
    • 2022
  • Despite the significant progress in camera pose estimation and structure-from-motion reconstruction from unstructured images, methods that exploit a priori information on camera arrangements have been overlooked. Conventional state-of-the-art methods do not exploit the geometric structure to recover accurate camera poses from a set of patch images in an array for mosaic-based imaging that creates a wide field-of-view image by sewing together a collection of regular images. We propose a camera pose estimation framework that exploits the array-structured image settings in each incremental reconstruction step. It consists of the two-way registration, the 3D point outlier elimination and the bundle adjustment with a constraint term for consistent rotation vectors to reduce reprojection errors during optimization. We demonstrate that by using individual images' connected structures at different camera pose estimation steps, we can estimate camera poses more accurately from all structured mosaic-based image sets, including omnidirectional scenes.

Image Fusion of High Resolution SAR and Optical Image Using High Frequency Information (고해상도 SAR와 광학영상의 고주파 정보를 이용한 다중센서 융합)

  • Byun, Young-Gi;Chae, Tae-Byeong
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.30 no.1
    • /
    • pp.75-86
    • /
    • 2012
  • Synthetic Aperture Radar(SAR) imaging system is independent of solar illumination and weather conditions; however, SAR image is difficult to interpret as compared with optical images. It has been increased interest in multi-sensor fusion technique which can improve the interpretability of $SAR^{\circ\circ}$ images by fusing the spectral information from multispectral(MS) image. In this paper, a multi-sensor fusion method based on high-frequency extraction process using Fast Fourier Transform(FFT) and outlier elimination process is proposed, which maintain the spectral content of the original MS image while retaining the spatial detail of the high-resolution SAR image. We used TerraSAR-X which is constructed on the same X-band SAR system as KOMPSAT-5 and KOMPSAT-2 MS image as the test data set to evaluate the proposed method. In order to evaluate the efficiency of the proposed method, the fusion result was compared visually and quantitatively with the result obtained using existing fusion algorithms. The evaluation results showed that the proposed image fusion method achieved successful results in the fusion of SAR and MS image compared with the existing fusion algorithms.

Travel Time Forecasting in an Interrupted Traffic Flow by adopting Historical Profile and Time-Space Data Fusion (히스토리컬 프로파일 구축과 시.공간 자료합성에 의한 단속류 통행시간 예측)

  • Yeo, Tae-Dong;Han, Gyeong-Su;Bae, Sang-Hun
    • Journal of Korean Society of Transportation
    • /
    • v.27 no.2
    • /
    • pp.133-144
    • /
    • 2009
  • In Korea, the ITS project has been progressed to improve traffic mobility and safety. Further, it is to relieve traffic jam by supply real time travel information for drivers and to promote traffic convenience and safety. It is important that the traffic information is provided accurately. This study was conducted outlier elimination and missing data adjustment to improve accuracy of raw data. A method for raise reliability of travel time prediction information was presented. We developed Historical Profile model and adjustment formula to reflect quality of interrupted flow. We predicted travel time by developed Historical Profile model and adjustment formula and verified by comparison between developed model and existing model such as Neural Network model and Kalman Filter model. The results of comparative analysis clarified that developed model and Karlman Filter model similarity predicted in general situation but developed model was more accurate than other models in incident situation.

A Time Series-based Algorithm for Eliminating Outliers of GPS Probe Data (시계열기반의 GPS 프로브 자료의 이상치 제거 알고리즘 개발)

  • Choi, Kee-Choo;Jang, Jeong-A
    • Journal of Korean Society of Transportation
    • /
    • v.22 no.6
    • /
    • pp.67-77
    • /
    • 2004
  • A treatment of outlier has been discussed. Outliers disrupt the reliability of information systems and they should be eliminated prior to the information and/or data fusion. A time series-based elimination algorithm were proposed and prediction interval, as a criterion of acceptable value width, was obtained with the model. Ten actual link values were used and the best model was identified as IMA(1,1). Although the actual verification was difficult in a sense that the matching process between the eliminated data and model data was not readily available, the proposed model can be successfully used in practice with some calibration efforts.

A three-stage deep-learning-based method for crack detection of high-resolution steel box girder image

  • Meng, Shiqiao;Gao, Zhiyuan;Zhou, Ying;He, Bin;Kong, Qingzhao
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.29-39
    • /
    • 2022
  • Crack detection plays an important role in the maintenance and protection of steel box girder of bridges. However, since the cracks only occupy an extremely small region of the high-resolution images captured from actual conditions, the existing methods cannot deal with this kind of image effectively. To solve this problem, this paper proposed a novel three-stage method based on deep learning technology and morphology operations. The training set and test set used in this paper are composed of 360 images (4928 × 3264 pixels) in steel girder box. The first stage of the proposed model converted high-resolution images into sub-images by using patch-based method and located the region of cracks by CBAM ResNet-50 model. The Recall reaches 0.95 on the test set. The second stage of our method uses the Attention U-Net model to get the accurate geometric edges of cracks based on results in the first stage. The IoU of the segmentation model implemented in this stage attains 0.48. In the third stage of the model, we remove the wrong-predicted isolated points in the predicted results through dilate operation and outlier elimination algorithm. The IoU of test set ascends to 0.70 after this stage. Ablation experiments are conducted to optimize the parameters and further promote the accuracy of the proposed method. The result shows that: (1) the best patch size of sub-images is 1024 × 1024. (2) the CBAM ResNet-50 and the Attention U-Net achieved the best results in the first and the second stage, respectively. (3) Pre-training the model of the first two stages can improve the IoU by 2.9%. In general, our method is of great significance for crack detection.