• 제목/요약/키워드: missing data recovery

검색결과 23건 처리시간 0.025초

Deep learning-based recovery method for missing structural temperature data using LSTM network

  • Liu, Hao;Ding, You-Liang;Zhao, Han-Wei;Wang, Man-Ya;Geng, Fang-Fang
    • Structural Monitoring and Maintenance
    • /
    • 제7권2호
    • /
    • pp.109-124
    • /
    • 2020
  • Benefiting from the massive monitoring data collected by the Structural health monitoring (SHM) system, scholars can grasp the complex environmental effects and structural state during structure operation. However, the monitoring data is often missing due to sensor faults and other reasons. It is necessary to study the recovery method of missing monitoring data. Taking the structural temperature monitoring data of Nanjing Dashengguan Yangtze River Bridge as an example, the long short-term memory (LSTM) network-based recovery method for missing structural temperature data is proposed in this paper. Firstly, the prediction results of temperature data using LSTM network, support vector machine (SVM), and wavelet neural network (WNN) are compared to verify the accuracy advantage of LSTM network in predicting time series data (such as structural temperature). Secondly, the application of LSTM network in the recovery of missing structural temperature data is discussed in detail. The results show that: the LSTM network can effectively recover the missing structural temperature data; incorporating more intact sensor data as input will further improve the recovery effect of missing data; selecting the sensor data which has a higher correlation coefficient with the data we want to recover as the input can achieve higher accuracy.

Recovery the Missing Streamflow Data on River Basin Based on the Deep Neural Network Model

  • Le, Xuan-Hien;Lee, Giha
    • 한국수자원학회:학술대회논문집
    • /
    • 한국수자원학회 2019년도 학술발표회
    • /
    • pp.156-156
    • /
    • 2019
  • In this study, a gated recurrent unit (GRU) network is constructed based on a deep neural network (DNN) with the aim of restoring the missing daily flow data in river basins. Lai Chau hydrological station is located upstream of the Da river basin (Vietnam) is selected as the target station for this study. Input data of the model are data on observed daily flow for 24 years from 1961 to 1984 (before Hoa Binh dam was built) at 5 hydrological stations, in which 4 gauge stations in the basin downstream and restoring - target station (Lai Chau). The total available data is divided into sections for different purposes. The data set of 23 years (1961-1983) was employed for training and validation purposes, with corresponding rates of 80% for training and 20% for validation respectively. Another data set of one year (1984) was used for the testing purpose to objectively verify the performance and accuracy of the model. Though only a modest amount of input data is required and furthermore the Lai Chau hydrological station is located upstream of the Da River, the calculated results based on the suggested model are in satisfactory agreement with observed data, the Nash - Sutcliffe efficiency (NSE) is higher than 95%. The finding of this study illustrated the outstanding performance of the GRU network model in recovering the missing flow data at Lai Chau station. As a result, DNN models, as well as GRU network models, have great potential for application within the field of hydrology and hydraulics.

  • PDF

Simulation combined transfer learning model for missing data recovery of nonstationary wind speed

  • Qiushuang Lin;Xuming Bao;Ying Lei;Chunxiang Li
    • Wind and Structures
    • /
    • 제37권5호
    • /
    • pp.383-397
    • /
    • 2023
  • In the Structural Health Monitoring (SHM) system of civil engineering, data missing inevitably occurs during the data acquisition and transmission process, which brings great difficulties to data analysis and poses challenges to structural health monitoring. In this paper, Convolution Neural Network (CNN) is used to recover the nonstationary wind speed data missing randomly at sampling points. Given the technical constraints and financial implications, field monitoring data samples are often insufficient to train a deep learning model for the task at hand. Thus, simulation combined transfer learning strategy is proposed to address issues of overfitting and instability of the deep learning model caused by the paucity of training samples. According to a portion of target data samples, a substantial quantity of simulated data consistent with the characteristics of target data can be obtained by nonstationary wind-field simulation and are subsequently deployed for training an auxiliary CNN model. Afterwards, parameters of the pretrained auxiliary model are transferred to the target model as initial parameters, greatly enhancing training efficiency for the target task. Simulation synergy strategy effectively promotes the accuracy and stability of the target model to a great extent. Finally, the structural dynamic response analysis verifies the efficiency of the simulation synergy strategy.

Sparse 복원 알고리즘을 이용한 HRRP 및 ISAR 영상 형성에 관한 연구 (A Study on the Formulation of High Resolution Range Profile and ISAR Image Using Sparse Recovery Algorithm)

  • 배지훈;김경태;양은정
    • 한국전자파학회논문지
    • /
    • 제25권4호
    • /
    • pp.467-475
    • /
    • 2014
  • 본 논문에서는 1차원 레이더 특성(signature)인 고해상도 거리 측면도(HRRP)와 2차원 레이더 특성인 ISAR 영상을 형성하기 위하여 CS(Compressive Sensing) 기반의 레이더 신호 모델을 적용한 sparse 복원(sparse recovery) 알고리즘을 소개하고자 한다. 만약, 관측된 RCS(Radar Cross Section) 데이터 샘플에서 데이터 손실이 발생할 경우, 기존의 discrete Fourier transform(DFT) 방식으로는 올바른 고해상도의 레이더 특성들을 얻을 수 없다. 하지만, 데이터 손실이 존재하더라도 상기 sparse 복원 알고리즘을 적용하면 고해상도의 레이더 특성을 성공적으로 복원할 수 있고, 원래 광대역의 RCS 데이터를 이용한 레이더 특성과 동등하게 고해상도를 유지할 수 있다. 따라서, 본 논문에서 보여준 결과에서와 같이 원하지 않는 간섭신호나 전파 교란 신호에 의해 데이터 손실이 발생한 RCS 데이터를 수집하더라도, sparse 복원 알고리즘을 이용하면 기존 DFT 방식과 달리 고해상도의 레이더 특성을 성공적으로 복원할 수 있음을 관찰할 수 있었다.

Recovering missing data transmitted from a wireless sensor node for vibration-based bridge health monitoring

  • Kim, C.W.;Kawatani, M.;Ozaki, R.;Makihata, N.
    • Structural Engineering and Mechanics
    • /
    • 제38권4호
    • /
    • pp.417-428
    • /
    • 2011
  • This paper presents recovering of missing vibration data of a bridge transmitted from wireless sensors. Kalman filter algorithm is adopted to reconstruct the missing data analytically. Validity of the analytical approach is examined through a field experiment of a bridge. Observations demonstrate that, even a part of recovered acceleration responses is underestimated in comparison with those responses taken from cabled sensors, dominant frequencies taken from the reconstructed data are comparable with those from cabled sensors.

A mathematical model to recover missing monitoring data of foundation pit

  • Liu, Jiangang;Zhou, Dongdong;Liu, Kewen
    • Geomechanics and Engineering
    • /
    • 제9권3호
    • /
    • pp.275-286
    • /
    • 2015
  • A new method is presented to recover missing deformation data of lateral walls of foundation pit when the monitoring is interrupted; the method is called Dynamic Mathematical Model - Parameter Interpolation. The deformation of lateral walls of foundation pit is mainly affected by the type of supporting structure and the situation of constraints, therefore, this paper mainly studies the two different kinds of variation law of deep horizontal displacement when the lateral walls are constrained or not, proposes two dynamic curve models of normal distribution type and logarithmic type, deals with model parameters by interpolating and obtains the parameters of missing data, then missing monitoring data could be Figured out by these parameters. Compared with the result from the common average method which is used to recover missing data, in the upper 2/3 of the inclinometer tube, the result by using this method is closer to the actual monitoring data, in the lower 1/3 part of the inclinometer tube, the result from the common average method is closer to the actual monitoring data.

An Error Recovery Mechanism for Wireless Sensor Networks

  • Kim, Dong-Il
    • Journal of information and communication convergence engineering
    • /
    • 제10권3호
    • /
    • pp.237-241
    • /
    • 2012
  • In wireless sensor networks, the importance of transporting data correctly with reliability is increasing gradually along with the need to support communications between the nodes and sink. Data flow from the sink to the nodes requires reliability for control or management that is very sensitive and intolerant of error; however, data flow from the nodes to the sink is relatively tolerant. In this paper, with emphasis on the data flow from the sink to the nodes, we propose a mechanism that considers accurate transport with reliability hop-by-hop. During the process of sending the data, if errors occur or data is missing, the proposed mechanism supports error recovery using a fixed window with selective acknowledgment. In addition, this mechanism supports congestion control depending on the buffer condition. Through the simulation, we show that this mechanism is accurate, reliable, and proper for transport in wireless sensor networks.

효과적인 결측치 보완을 통한 다층 퍼셉트론 기반의 전력수요 예측 기법 (A Multilayer Perceptron-Based Electric Load Forecasting Scheme via Effective Recovering Missing Data)

  • 문지훈;박성우;황인준
    • 정보처리학회논문지:소프트웨어 및 데이터공학
    • /
    • 제8권2호
    • /
    • pp.67-78
    • /
    • 2019
  • 정확한 전력수요 예측은 스마트 그리드의 효율적인 운영에 있어 매우 중요하다. 최근 IT 기술이 획기적으로 발전되면서, 인공지능 기법을 이용한 빅 데이터 처리를 기반으로 정확한 전력수요를 예측하는 많은 연구가 진행되고 있다. 이러한 예측 모델은 주로 외부 요인과 과거 전력수요를 독립 변수로 사용한다. 하지만, 다양한 내부적 또는 외부적 원인으로 전력수요 데이터의 결측치가 발생하게 되면 정확한 예측 모델을 구성하기가 어렵다. 이에 본 논문에서는 랜덤 포레스트 기반의 결측치 데이터 보완 기법을 제안하고, 보완된 데이터를 기반으로 한 다층 퍼셉트론 기반의 전력수요 예측 모델을 구성한다. 다양한 실험을 통해 제안된 기법의 예측 성능을 입증한다.

Compressive sensing-based two-dimensional scattering-center extraction for incomplete RCS data

  • Bae, Ji-Hoon;Kim, Kyung-Tae
    • ETRI Journal
    • /
    • 제42권6호
    • /
    • pp.815-826
    • /
    • 2020
  • We propose a two-dimensional (2D) scattering-center-extraction (SCE) method using sparse recovery based on the compressive-sensing theory, even with data missing from the received radar cross-section (RCS) dataset. First, using the proposed method, we generate a 2D grid via adaptive discretization that has a considerably smaller size than a fully sampled fine grid. Subsequently, the coarse estimation of 2D scattering centers is performed using both the method of iteratively reweighted least square and a general peak-finding algorithm. Finally, the fine estimation of 2D scattering centers is performed using the orthogonal matching pursuit (OMP) procedure from an adaptively sampled Fourier dictionary. The measured RCS data, as well as simulation data using the point-scatterer model, are used to evaluate the 2D SCE accuracy of the proposed method. The results indicate that the proposed method can achieve higher SCE accuracy for an incomplete RCS dataset with missing data than that achieved by the conventional OMP, basis pursuit, smoothed L0, and existing discrete spectral estimation techniques.

The Influence of Service Recovery Justice on Intention to Recommend for Retailer

  • SHIN, Yongsun;KIM, Moonseop
    • 유통과학연구
    • /
    • 제18권2호
    • /
    • pp.91-98
    • /
    • 2020
  • Purpose: This research aimed to suggest retailing companies some ways to enhance customer satisfaction with service recovery and recommendation intention towards these companies. For this purpose, current study examined the relationships among service recovery justice, service failure severity, customer trust, recovery satisfaction and intention to recommend and the moderating role of ego-resilience. Research design, data and methodology: Current study developed a structural equation model in which perceived service recovery justice is a predictor, service failure severity, customer trust, recovery satisfaction are mediators, intention to recommend is a dependent variable and the ego-resilience is a moderator between the perceived service recovery justice and the customer trust and the recovery satisfaction. Data were collected from customers who experienced service failures from retailers. A total of 400 questionnaires were collected and 365 samples were used for analysis after deleting data having missing value. SPSS 25.0 and AMOS 24.0 were used to test the validity, reliability, and structural equation modeling. Results: Empirical results showed that the perceived service recovery justice had a negative influence on the perceived service failure severity and a positive influence on the customer trust and the recovery satisfaction. These results indicate that when customers perceive the service recovery justice more highly, they perceive the service failure less severe but they perceive the retailer more trustworthy and are satisfied with service recovery. In addition, the customer trust and the recovery satisfaction had a positive influence on the intention to recommend. These results indicate that when customers perceive the retailer more trustworthy and are satisfied with service recovery, they are more intend to recommend the retailer. Moreover, the influence of the perceived service recovery justice on the customer trust and the recovery satisfaction was moderated by the ego-resilience. Conclusions: This study contributed to the service recovery literature by proving the relationship among service recovery justice, service failure severity, customer trust, recovery satisfaction and intention to recommend. Moreover, current research introduced the ego-resilience into service recovery research area and revealed the moderation role of the ego-resilience. Managerially, this research suggested retailing companies some ways to effectively recover from service failure.