• Title/Summary/Keyword: Noise Problem

Search Result 2,513, Processing Time 0.025 seconds

A Study of Anomaly Detection for ICT Infrastructure using Conditional Multimodal Autoencoder (ICT 인프라 이상탐지를 위한 조건부 멀티모달 오토인코더에 관한 연구)

  • Shin, Byungjin;Lee, Jonghoon;Han, Sangjin;Park, Choong-Shik
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.3
    • /
    • pp.57-73
    • /
    • 2021
  • Maintenance and prevention of failure through anomaly detection of ICT infrastructure is becoming important. System monitoring data is multidimensional time series data. When we deal with multidimensional time series data, we have difficulty in considering both characteristics of multidimensional data and characteristics of time series data. When dealing with multidimensional data, correlation between variables should be considered. Existing methods such as probability and linear base, distance base, etc. are degraded due to limitations called the curse of dimensions. In addition, time series data is preprocessed by applying sliding window technique and time series decomposition for self-correlation analysis. These techniques are the cause of increasing the dimension of data, so it is necessary to supplement them. The anomaly detection field is an old research field, and statistical methods and regression analysis were used in the early days. Currently, there are active studies to apply machine learning and artificial neural network technology to this field. Statistically based methods are difficult to apply when data is non-homogeneous, and do not detect local outliers well. The regression analysis method compares the predictive value and the actual value after learning the regression formula based on the parametric statistics and it detects abnormality. Anomaly detection using regression analysis has the disadvantage that the performance is lowered when the model is not solid and the noise or outliers of the data are included. There is a restriction that learning data with noise or outliers should be used. The autoencoder using artificial neural networks is learned to output as similar as possible to input data. It has many advantages compared to existing probability and linear model, cluster analysis, and map learning. It can be applied to data that does not satisfy probability distribution or linear assumption. In addition, it is possible to learn non-mapping without label data for teaching. However, there is a limitation of local outlier identification of multidimensional data in anomaly detection, and there is a problem that the dimension of data is greatly increased due to the characteristics of time series data. In this study, we propose a CMAE (Conditional Multimodal Autoencoder) that enhances the performance of anomaly detection by considering local outliers and time series characteristics. First, we applied Multimodal Autoencoder (MAE) to improve the limitations of local outlier identification of multidimensional data. Multimodals are commonly used to learn different types of inputs, such as voice and image. The different modal shares the bottleneck effect of Autoencoder and it learns correlation. In addition, CAE (Conditional Autoencoder) was used to learn the characteristics of time series data effectively without increasing the dimension of data. In general, conditional input mainly uses category variables, but in this study, time was used as a condition to learn periodicity. The CMAE model proposed in this paper was verified by comparing with the Unimodal Autoencoder (UAE) and Multi-modal Autoencoder (MAE). The restoration performance of Autoencoder for 41 variables was confirmed in the proposed model and the comparison model. The restoration performance is different by variables, and the restoration is normally well operated because the loss value is small for Memory, Disk, and Network modals in all three Autoencoder models. The process modal did not show a significant difference in all three models, and the CPU modal showed excellent performance in CMAE. ROC curve was prepared for the evaluation of anomaly detection performance in the proposed model and the comparison model, and AUC, accuracy, precision, recall, and F1-score were compared. In all indicators, the performance was shown in the order of CMAE, MAE, and AE. Especially, the reproduction rate was 0.9828 for CMAE, which can be confirmed to detect almost most of the abnormalities. The accuracy of the model was also improved and 87.12%, and the F1-score was 0.8883, which is considered to be suitable for anomaly detection. In practical aspect, the proposed model has an additional advantage in addition to performance improvement. The use of techniques such as time series decomposition and sliding windows has the disadvantage of managing unnecessary procedures; and their dimensional increase can cause a decrease in the computational speed in inference.The proposed model has characteristics that are easy to apply to practical tasks such as inference speed and model management.

Seismic wave propagation through surface basalts - implications for coal seismic surveys (지표 현무암을 통해 전파하는 탄성파의 거동 - 석탄 탄성파탐사에 적용)

  • Sun, Weijia;Zhou, Binzhong;Hatherly, Peter;Fu, Li-Yun
    • Geophysics and Geophysical Exploration
    • /
    • v.13 no.1
    • /
    • pp.1-8
    • /
    • 2010
  • Seismic reflection surveying is one of the most widely used and effective techniques for coal seam structure delineation and risk mitigation for underground longwall mining. However, the ability of the method can be compromised by the presence of volcanic cover. This problem arises within parts of the Bowen and Sydney Basins of Australia and seismic surveying can be unsuccessful. As a consequence, such areas are less attractive for coal mining. Techniques to improve the success of seismic surveying over basalt flows are needed. In this paper, we use elastic wave-equation-based forward modelling techniques to investigate the effects and characteristics of seismic wave propagation under different settings involving changes in basalt properties, its thickness, lateral extent, relative position to the shot position and various forms of inhomogeneity. The modelling results suggests that: 1) basalts with high impedance contrasts and multiple flows generate strong multiples and weak reflectors; 2) thin basalts have less effect than thick basalts; 3) partial basalt cover has less effect than full basalt cover; 4) low frequency seismic waves (especially at large offsets) have better penetration through the basalt than high frequency waves; and 5) the deeper the coal seams are below basalts of limited extent, the less influence the basalts will have on the wave propagation. In addition to providing insights into the issues that arise when seismic surveying under basalts, these observations suggest that careful management of seismic noise and the acquisition of long-offset seismic data with low-frequency geophones have the potential to improve the seismic results.

Evaluation of Clinical Availability for Shoulder Forced Traction Method to Minimize the Beam Hardening Artifact in Cervical-spine Computed Tomography (CT) (경추부 전산화단층촬영에서 선속 경화 인공물을 최소화하기 위한 견부 강제 견인법에 대한 임상적 유용성 평가)

  • Kim, Moonjeung;Cho, Wonjin;Kang, Suyeon;Lee, Wonseok;Park, Jinwoo;Yu, Yunsik;Im, Inchul;Lee, Jaeseung;Kim, Hyeonjin;Kwak, Byungjoon
    • Journal of the Korean Society of Radiology
    • /
    • v.7 no.1
    • /
    • pp.37-44
    • /
    • 2013
  • In study suggested clinical availability to shoulder forced traction method in term of quality of image, the patient's convenience and stability, according to whether to use of shoulder forced traction bend using computed tomography(CT) that X-ray calibration and various mathematic calibration algorithm application can be applied by AEC. To achieve this, 79 patients is complaining of cervical pain oriented that shoulder forced traction bend use the before and after acquires lateral projection scout image and transverse image. transverse image of a fixed size in concern field of pixel and figure the average HU value compare that quantitative analysis. Artifact and pixel and resolution to qualitative clinical estimation image analysis. the patient feel inconvenience degree that self-diagnosis survey that estimate. As a result, lateral projection scout image if you used shoulder forced traction bend for the depicted has been an increase in the number of a cervical vertebrae. transverse image concern field shoulder forced traction bend use the before and after for pixel and the average HU-value changes was judged to be almost irrelevant. Artifact and resolution and contrast, in qualitative analysis of the results relating the observer to the unusual result. So, the patients of 82.27% complained discomfort that use of shoulder forced traction bend in self-diagnosis survey. No merit of medical image by using of bend from result was analyzed quality of image to quantitative and qualitative method judged. Nowadays, CT is supplied possible revision of quality of radiation by reduction of slice and automatic exposure controller, etc and application of preconditioning filter process due to various mathematic revision algorithm. So, image noise by beam hardening artifact should not be a problem. shoulder forced traction bend of use no longer judged clinically availability because have not influence of image quality and give discomfort, have extra dangerousness.

Interactive analysis tools for the wide-angle seismic data for crustal structure study (Technical Report) (지각 구조 연구에서 광각 탄성파 자료를 위한 대화식 분석 방법들)

  • Fujie, Gou;Kasahara, Junzo;Murase, Kei;Mochizuki, Kimihiro;Kaneda, Yoshiyuki
    • Geophysics and Geophysical Exploration
    • /
    • v.11 no.1
    • /
    • pp.26-33
    • /
    • 2008
  • The analysis of wide-angle seismic reflection and refraction data plays an important role in lithospheric-scale crustal structure study. However, it is extremely difficult to develop an appropriate velocity structure model directly from the observed data, and we have to improve the structure model step by step, because the crustal structure analysis is an intrinsically non-linear problem. There are several subjective processes in wide-angle crustal structure modelling, such as phase identification and trial-and-error forward modelling. Because these subjective processes in wide-angle data analysis reduce the uniqueness and credibility of the resultant models, it is important to reduce subjectivity in the analysis procedure. From this point of view, we describe two software tools, PASTEUP and MODELING, to be used for developing crustal structure models. PASTEUP is an interactive application that facilitates the plotting of record sections, analysis of wide-angle seismic data, and picking of phases. PASTEUP is equipped with various filters and analysis functions to enhance signal-to-noise ratio and to help phase identification. MODELING is an interactive application for editing velocity models, and ray-tracing. Synthetic traveltimes computed by the MODELING application can be directly compared with the observed waveforms in the PASTEUP application. This reduces subjectivity in crustal structure modelling because traveltime picking, which is one of the most subjective process in the crustal structure analysis, is not required. MODELING can convert an editable layered structure model into two-way traveltimes which can be compared with time-sections of Multi Channel Seismic (MCS) reflection data. Direct comparison between the structure model of wide-angle data with the reflection data will give the model more credibility. In addition, both PASTEUP and MODELING are efficient tools for handling a large dataset. These software tools help us develop more plausible lithospheric-scale structure models using wide-angle seismic data.

Comparison of Electrical Signal Properties about Top Electrode Size on Photoconductor Film (광도전체 필름 상부 전극크기에 따른 전기적 신호 특성 비교)

  • Kang, Sang-Sik;Jung, Bong-Jae;Noh, Si-Cheul;Cho, Chang-Hoon;Yoon, Ju-Sun;Jeon, Sung-Pyo;Park, Ji-Koon
    • Journal of the Korean Society of Radiology
    • /
    • v.5 no.2
    • /
    • pp.93-96
    • /
    • 2011
  • Currently, the development of direct conversion radiation detector using photoconductor materials is progressing in widely. Among of theses photoconductor materials, mercuric iodide compound than amorphous selenium has excellent absorption and sensitivity of high energy radiation. Also, the detection efficiency of signal generated in photoconductor film varies by electric filed and geometric distribution according to top-bottom electrode size. Therefore, in this work, the x-ray detection characteristics are investigated about the size of top electrode in $HgI_2$ photoconductor film. For sample fabrication, to solve the problem that is difficult to make a large area film, we used the spatial paste screen-print method. And the sample thickness is $150{\mu}m$ and an film area size is $3cm{\times}3cm$ on ITO-coated glass substrate. ITO(Indium-Tin-Oxide) electrode was used as top electrode using a magnetron sputtering system and each area is $3cm{\times}3cm$, $2cm{\times}2cm$ and $1cm{\times}1cm$. From experimental measurement, the dark current, sensitivity and SNR of the $HgI_2$ film are obtained from I-V test. From the experimental results, it shows that the sensitivity increases in accordance with the area of the electrode but the SNR is decreased because of the high dark current. Therefore, the optimized size of electrode is importance for the development of photoconductor based x-ray imaging detector.

Closed Integral Form Expansion for the Highly Efficient Analysis of Fiber Raman Amplifier (라만증폭기의 효율적인 성능분석을 위한 라만방정식의 적분형 전개와 수치해석 알고리즘)

  • Choi, Lark-Kwon;Park, Jae-Hyoung;Kim, Pil-Han;Park, Jong-Han;Park, Nam-Kyoo
    • Korean Journal of Optics and Photonics
    • /
    • v.16 no.3
    • /
    • pp.182-190
    • /
    • 2005
  • The fiber Raman amplifier(FRA) is a distinctly advantageous technology. Due to its wider, flexible gain bandwidth, and intrinsically lower noise characteristics, FRA has become an indispensable technology of today. Various FRA modeling methods, with different levels of convergence speed and accuracy, have been proposed in order to gain valuable insights for the FRA dynamics and optimum design before real implementation. Still, all these approaches share the common platform of coupled ordinary differential equations(ODE) for the Raman equation set that must be solved along the long length of fiber propagation axis. The ODE platform has classically set the bar for achievable convergence speed, resulting exhaustive calculation efforts. In this work, we propose an alternative, highly efficient framework for FRA analysis. In treating the Raman gain as the perturbation factor in an adiabatic process, we achieved implementation of the algorithm by deriving a recursive relation for the integrals of power inside fiber with the effective length and by constructing a matrix formalism for the solution of the given FRA problem. Finally, by adiabatically turning on the Raman process in the fiber as increasing the order of iterations, the FRA solution can be obtained along the iteration axis for the whole length of fiber rather than along the fiber propagation axis, enabling faster convergence speed, at the equivalent accuracy achievable with the methods based on coupled ODEs. Performance comparison in all co-, counter-, bi-directionally pumped multi-channel FRA shows more than 102 times faster with the convergence speed of the Average power method at the same level of accuracy(relative deviation < 0.03dB).

A Study on Rail Vibration and Its Reduction Plan in Central Daejeon Area (대전 도심지역의 철도진동의 영향과 대책)

  • Ryu, Myoung-Ik;Suh, Man-Cheol;Lee, Won-Kook
    • Journal of the Korean Geophysical Society
    • /
    • v.3 no.4
    • /
    • pp.269-280
    • /
    • 2000
  • Rail vibration in city zone is becoming a serious environmental problem. In order to make a reduction plan for rail vibration, the research was conducted in which many experiments to measure actual rail vibration along the railroad through the central Deajeon area. A digital vibration level meter was used to measure rail vibration. Vibration levels of Z-axis were measured at every second for the duration of the train passing. The measuring station was placed at every 5m for the distance of 55m. A total of 353 different sets of vibration level were obtained. The signals were processed to get $L_{10}$ value and analyzed in terms of distance, train velocity, and number of trains. As a result, it has been found that rail vibration exceed the allowable vibraton limit of 60 dB, at the point of 25 m far from the railroad center, which is regulated by the las of vibration and noise. Train velocity was found to affect a little for vibration level within the zone. It was also found that a trench installed along a railroad could reduce vibration level up to approximately 10 percent. A model test was conducted to investigate the influence of the location and size of trench, on the transfer of vibration. A heavy steel ball was used to generate vibrations. On the basis obtained from this study, it could be concluded that the application of distance-attenuation and the installment of a trench along railroad could be applied as a reduction plan for rail vibration. Because limitions might exist to depend on the effect of distance attenuation, trenchs excavated along a railroad might be suggested as the most efficient solution to reduce railroad vibration.

  • PDF

A Single Index Approach for Time-Series Subsequence Matching that Supports Moving Average Transform of Arbitrary Order (단일 색인을 사용한 임의 계수의 이동평균 변환 지원 시계열 서브시퀀스 매칭)

  • Moon Yang-Sae;Kim Jinho
    • Journal of KIISE:Databases
    • /
    • v.33 no.1
    • /
    • pp.42-55
    • /
    • 2006
  • We propose a single Index approach for subsequence matching that supports moving average transform of arbitrary order in time-series databases. Using the single index approach, we can reduce both storage space overhead and index maintenance overhead. Moving average transform is known to reduce the effect of noise and has been used in many areas such as econometrics since it is useful in finding overall trends. However, the previous research results have a problem of occurring index overhead both in storage space and in update maintenance since tile methods build several indexes to support arbitrary orders. In this paper, we first propose the concept of poly-order moving average transform, which uses a set of order values rather than one order value, by extending the original definition of moving average transform. That is, the poly-order transform makes a set of transformed windows from each original window since it transforms each window not for just one order value but for a set of order values. We then present theorems to formally prove the correctness of the poly-order transform based subsequence matching methods. Moreover, we propose two different subsequence matching methods supporting moving average transform of arbitrary order by applying the poly-order transform to the previous subsequence matching methods. Experimental results show that, for all the cases, the proposed methods improve performance significantly over the sequential scan. For real stock data, the proposed methods improve average performance by 22.4${\~}$33.8 times over the sequential scan. And, when comparing with the cases of building each index for all moving average orders, the proposed methods reduce the storage space required for indexes significantly by sacrificing only a little performance degradation(when we use 7 orders, the methods reduce the space by up to 1/7.0 while the performance degradation is only $9\%{\~}42\%$ on the average). In addition to the superiority in performance, index space, and index maintenance, the proposed methods have an advantage of being generalized to many sorts of other transforms including moving average transform. Therefore, we believe that our work can be widely and practically used in many sort of transform based subsequence matching methods.

Improved Method of License Plate Detection and Recognition using Synthetic Number Plate (인조 번호판을 이용한 자동차 번호인식 성능 향상 기법)

  • Chang, Il-Sik;Park, Gooman
    • Journal of Broadcast Engineering
    • /
    • v.26 no.4
    • /
    • pp.453-462
    • /
    • 2021
  • A lot of license plate data is required for car number recognition. License plate data needs to be balanced from past license plates to the latest license plates. However, it is difficult to obtain data from the actual past license plate to the latest ones. In order to solve this problem, a license plate recognition study through deep learning is being conducted by creating a synthetic license plates. Since the synthetic data have differences from real data, and various data augmentation techniques are used to solve these problems. Existing data augmentation simply used methods such as brightness, rotation, affine transformation, blur, and noise. In this paper, we apply a style transformation method that transforms synthetic data into real-world data styles with data augmentation methods. In addition, real license plate data are noisy when it is captured from a distance and under the dark environment. If we simply recognize characters with input data, chances of misrecognition are high. To improve character recognition, in this paper, we applied the DeblurGANv2 method as a quality improvement method for character recognition, increasing the accuracy of license plate recognition. The method of deep learning for license plate detection and license plate number recognition used YOLO-V5. To determine the performance of the synthetic license plate data, we construct a test set by collecting our own secured license plates. License plate detection without style conversion recorded 0.614 mAP. As a result of applying the style transformation, we confirm that the license plate detection performance was improved by recording 0.679mAP. In addition, the successul detection rate without image enhancement was 0.872, and the detection rate was 0.915 after image enhancement, confirming that the performance improved.

Study on Standardization of the Environmental Impact Evaluation Method of Extremely Low Frequency Magnetic Fields near High Voltage Overhead Transmission Lines (고압 가공송전선로의 극저주파자기장 환경영향평가 방법 표준화에 관한 연구)

  • Park, Sung-Ae;Jung, Joonsig;Choi, Taebong;Jeong, Minjoo;Kim, Bu-Kyung;Lee, Jongchun
    • Journal of Environmental Impact Assessment
    • /
    • v.27 no.6
    • /
    • pp.658-673
    • /
    • 2018
  • Social conflicts with extremely low frequency magnetic field(ELF-MF) exposures are expected to exacerbate due to continued increase in electric power demand and construction of high voltage transmission lines(HVTL). However, in current environmental impact assessment(EIA) act, specific guidelines have not been included concretely about EIA of ELF-MF. Therefore, this study conducted a standardization study on EIA method through case analysis, field measurement, and expert consultation of the EIA for the ELF-MF near HVTL which is the main cause of exposures. The status of the EIA of the ELF-MF and the problem to be improved are derived and the EIA method which can solve it is suggested. The main contents of the study is that the physical characteristics of the ELF-MF affected by distance and powerload should be considered at all stages of EIA(survey of the current situation - Prediction of the impacts - preparation of mitigation plan ? post EIA planning). Based on this study, we also suggested the 'Measurement method for extremely low frequency magnetic field on transmission line' and 'Table for extremely low frequency magnetic field measurement record on transmission line'. The results of this study can be applied to the EIA that minimizes the damage and conflict to the construction of transmission line and derives rational measures at the present time when the human hazard to long term exposure of the ELF-MF is unclear.