Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Convolutional neural network-based data anomaly detection considering class imbalance with limited data

  • Du, Yao (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Li, Ling-fang (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Hou, Rong-rong (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Wang, Xiao-you (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Tian, Wei (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Xia, Yong (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
  • Received : 2021.04.09
  • Accepted : 2021.06.21
  • Published : 2022.01.25
⟨ Previous Next ⟩

Abstract

The raw data collected by structural health monitoring (SHM) systems may suffer multiple patterns of anomalies, which pose a significant barrier for an automatic and accurate structural condition assessment. Therefore, the detection and classification of these anomalies is an essential pre-processing step for SHM systems. However, the heterogeneous data patterns, scarce anomalous samples and severe class imbalance make data anomaly detection difficult. In this regard, this study proposes a convolutional neural network-based data anomaly detection method. The time and frequency domains data are transferred as images and used as the input of the neural network for training. ResNet18 is adopted as the feature extractor to avoid training with massive labelled data. In addition, the focal loss function is adopted to soften the class imbalance-induced classification bias. The effectiveness of the proposed method is validated using acceleration data collected in a long-span cable-stayed bridge. The proposed approach detects and classifies data anomalies with high accuracy.

Keywords

Acknowledgement

The authors would like to thank the organisers of the 1st International Project Competition for SHM (IPC-SHM, 2020) for generously providing excellent opportunities during the COVID-19 and invaluable data from an actual structure. Special thanks go to Professor Hui Li and Professor Billie F. Spencer Jr., Co-Chairs of IPC-SHM, 2020. This research is also supported by the Key-Area Research and Development Program of Guangdong Province (Project No. 2019B111106001) and National Key Research and Development Program (Project No. 2019YFB1600700).

References

  1. An, Y.H., Chatzi, E., Sim, S.H., Laflamme, S., Blachowski, B. and Ou, J.P. (2019), "Recent progress and future trends on damage identification methods for bridge structures", Struct. Control Health Monitor., 26(10). https://doi.org/10.1002/stc.2416
  2. Bao, Y. and Li, H. (2020), "Machine learning paradigm for structural health monitoring", Struct. Health Monitor., 20(4), 1353-1372. https://doi.org/10.1177/1475921720972416
  3. Bao, Y., Tang, Z., Li, H. and Zhang, Y. (2019a), "Computer vision and deep learning-based data anomaly detection method for structural health monitoring", Struct. Health Monitor., 18(2), 401-421. https://doi.org/10.1177/1475921718757405
  4. Bao, Y.Q., Chen, Z.C., Wei, S.Y., Xu, Y., Tang, Z.Y. and Li, H. (2019b), "The State of the Art of Data Science and Engineering in Structural Health Monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027
  5. Bengio, Y. and Grandvalet, Y. (2004), "No unbiased estimator of the variance of k-fold cross-validation", J. Mach. Learn. Res., 5, 1089-1105.
  6. Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learning-based crack damage detection using convolutional neural networks", Comput.-Aided Civ. Infrastruct. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263
  7. Chang, P.C., Flatau, A. and Liu, S. (2003), "Health monitoring of civil infrastructure", Struct. Health Monitor., 2(3), 257-267. https://doi.org/10.1177/1475921703036169
  8. Fan, G., Li, J. and Hao, H. (2019), "Lost data recovery for structural health monitoring based on convolutional neural networks", Struct. Control Health Monitor., 26(10). https://doi.org/10.1002/stc.2433
  9. Fan, G., Li, J. and Hao, H. (2020), "Dynamic response reconstruction for structural health monitoring using densely connected convolutional networks", Struct. Health Monitor., 20(4), 1373-1391. https://doi.org/10.1177/1475921720916881
  10. Fu, Y.G., Peng, C., Gomez, F., Narazaki, Y. and Spencer, B.F. (2019), "Sensor fault management techniques for wireless smart sensor networks in structural health monitoring", Struct. Control Health Monitor., 26(7). https://doi.org/10.1002/stc.2362
  11. He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", IEEE Trans. Pattern Anal. Mach. Intell., 37(9), 1904-1916. https://doi.org/10.1109/TPAMI.2015.2389824
  12. He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  13. Hernandez-Garcia, M.R. and Masri, S.F. (2014), "Application of statistical monitoring using latent-variable techniques for detection of faults in sensor networks", J. Intell. Mater. Syst. Struct., 25(2), 121-136. https://doi.org/10.1177/1045389x13479182
  14. Hou, R. and Xia, Y. (2020), "Review on the new development of vibration-based damage identification for civil engineering structures: 2010-2019", J. Sound Vib., 491, 115741. https://doi.org/10.1016/j.jsv.2020.115741
  15. Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T. and Keutzer, K. (2014), "Densenet: Implementing efficient convnet descriptor pyramids", arXiv preprint arXiv:1404.1869.
  16. Ioffe, S. and Szegedy, C. (2015), "Batch normalisation: Accelerating deep network training by reducing internal covariate shift", Proceedings of International Conference on Machine Learning, pp. 448-456.
  17. Jeong, S., Ferguson, M., Hou, R., Lynch, J.P., Sohn, H. and Law, K.H. (2019), "Sensor data reconstruction using bidirectional recurrent neural network with application to bridge monitoring", Adv. Eng. Inf., 42, 1009911. https://doi.org/10.1016/j.aei.2019.100991
  18. Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012), "Imagenet classification with deep convolutional neural networks", Adv. Neural Inform. Process. Syst., 25, 1097-1105.
  19. Kullaa, J. (2011), "Distinguishing between sensor fault, structural damage, and environmental or operational effects in structural health monitoring", Mech. Syst. Sig. Process., 25(8), 2976-2989. https://doi.org/10.1016/j.ymssp.2011.05.017
  20. Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M. and Kolesnikov, A. (2020), "The open images dataset v4", Int. J. Comput. Vis., 1-26. https://doi.org/10.1007/s11263-020-01316-z
  21. LeCun, Y., Bengio, Y. and Hinton, G. (2015), "Deep learning", Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539
  22. Lei, X., Sun, L. and Xia, Y. (2020), "Lost data reconstruction for structural health monitoring using deep convolutional generative adversarial networks", Struct. Health Monitor., 20(4), 2069-2087. https://doi.org/10.1177/1475921720959226
  23. Li, J., Hao, H., Xia, Y. and Zhu, H.-P. (2014), "Damage detection of shear connectors in bridge structures with transmissibility in frequency domain", Int. J. Struct. Stab. Dyn., 14(02), 1350061. https://doi.org/10.1142/s0219455413500612
  24. Li, L.L., Liu, G., Zhang, L.L. and Li, Q. (2019), "Sensor fault detection with generalised likelihood ratio and correlation coefficient for bridge SHM", J. Sound Vib., 442, 445-458. https://doi.org/10.1016/j.jsv.2018.10.062
  25. Lin, T.-Y., Goyal, P., Girshick, R., He, K. and Dollar, P. (2017a), "Focal loss for dense object detection", Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988.
  26. Lin, Y.Z., Nie, Z.H. and Ma, H.W. (2017b), "Structural damage detection with automatic feature-extraction through deep learning", Comput.-Aided Civ. Infrastruct. Eng., 32(12), 1025-1046. https://doi.org/10.1111/mice.12313
  27. Liu, G., Li, L.L., Zhang, L.L., Li, Q. and Law, S.S. (2020), "Sensor faults classification for SHM systems using deep learning-based method with Tsfresh features", Smart Mater. Struct., 29(7). https://doi.org/10.1088/1361-665X/ab85a6
  28. Mao, J.X., Wang, H. and Spencer, B.F. (2020), "Toward data anomaly detection for automated structural health monitoring: Exploiting generative adversarial nets and autoencoders", Struct. Health Monitor, 20(4), 1609-1626. https://doi.org/10.1177/1475921720924601
  29. Mohtasham Khani, M., Vahidnia, S., Ghasemzadeh, L., Ozturk, Y.E., Yuvalaklioglu, M., Akin, S. and Ure, N.K. (2019), "Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines", Struct. Health Monitor., 19(5), 1440-1452. https://doi.org/10.1177/1475921719883202
  30. Ni, F.T., Zhang, J. and Noori, M.N. (2020), "Deep learning for data anomaly detection and data compression of a long-span suspension bridge", Comput.-Aided Civ. Infrastruct. Eng., 35(7), 685-700. https://doi.org/10.1111/mice.12528
  31. Oh, B.K., Glisic, B., Kim, Y. and Park, H.S. (2019), "Convolutional neural network-based wind-induced response estimation model for tall buildings", Comput.-Aided Civ. Infrastruct. Eng., 34(10), 843-858. https://doi.org/10.1111/mice.12476
  32. Pan, S.J. and Yang, Q.A. (2010), "A Survey on Transfer Learning", IEEE Trans. Knowl. Data. Eng., 22(10), 1345-1359. https://doi.org/10.1109/Tkde.2009.191
  33. Ramachandran, P., Zoph, B. and Le, Q.V. (2017), "Searching for activation functions", arXiv preprint arXiv:1710.05941.
  34. Rao, A.R.M., Kasireddy, V., Gopalakrishnan, N. and Lakshmi, K. (2015), "Sensor fault detection in structural health monitoring using null subspace-based approach", J. Intell. Mater. Syst. Struct., 26(2), 172-185. https://doi.org/10.1177/1045389x14522534
  35. Rawat, W. and Wang, Z. (2017), "Deep convolutional neural networks for image classification: A comprehensive review", Neural Comput., 29(9), 2352-2449. https://doi.org/10.1162/NECO_a_00990
  36. Rodriguez, J.D., Perez, A. and Lozano, J.A. (2009), "Sensitivity analysis of k-fold cross validation in prediction error estimation", IEEE Trans. Pattern Anal. Mach. Intell., 32(3), 569-575. https://doi.org/10.1109/TPAMI.2009.187
  37. Sharifi, R., Kim, Y. and Langari, R. (2010), "Sensor fault isolation and detection of smart structures", Smart Mater. Struct., 19(10), 105001. https://doi.org/10.1088/0964-1726/19/10/105001
  38. Smarsly, K. and Law, K.H. (2014), "Decentralised fault detection and isolation in wireless structural health monitoring systems using analytical redundancy", Adv. Eng. Software, 73, 1-10. https://doi.org/10.1016/j.advengsoft.2014.02.005
  39. Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Eng., 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  40. Sun, Z., Zou, Z.L. and Zhang, Y.F. (2017), "Utilisation of structural health monitoring in long-span bridges: Case studies", Struct. Control Health Monitor., 24(10). https://doi.org/10.1002/stc.1979
  41. Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A. (2017), "Inception-v4, inception-resnet and the impact of residual connections on learning", Proceedings of the AAAI Conference on Artificial Intelligence.
  42. Tang, Z.Y., Chen, Z.C., Bao, Y.Q. and Li, H. (2019), "Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring", Struct. Control Health Monitor., 26(1). https://doi.org/10.1002/stc.2296
  43. Wu, R.T. and Jahanshahi, M.R. (2019), "Deep convolutional neural network for structural dynamic response estimation and system identification", J. Eng. Mech., 145(1). https://doi.org/10.1061/(asce)em.1943-7889.0001556
  44. Xia, Y., Chen, B., Weng, S., Ni, Y.-Q. and Xu, Y.-L. (2012), "Temperature effect on vibration properties of civil structures: a literature review and case studies", J. Civil Struct. Health Monitor., 2(1), 29-46. https://doi.org/10.1007/s13349-011-0015-7
  45. Xia, Y., Lei, X., Wang, P., Liu, G. and Sun, L. (2020), "Long-term performance monitoring and assessment of concrete beam bridges using neutral axis indicator", Struct. Control Health Monitor., 27(12). https://doi.org/10.1002/stc.2637
  46. Yeum, C.M. and Dyke, S.J. (2015), "Vision-Based Automated Crack Detection for Bridge Inspection", Comput.-Aided Civ. Infrastruct. Eng., 30(10), 759-770. https://doi.org/10.1111/mice.12141
  47. Yi, T.H., Li, H.N., Song, G.B. and Guo, Q. (2016), "Detection of shifts in GPS measurements for a long-span bridge using CUSUM chart", Int. J. Struct. Stab. Dyn., 16(04), 1640024. https://doi.org/10.1142/S0219455416400241
  48. Yu, Y., Wang, C.Y., Gu, X.Y. and Li, J.C. (2018), "A novel deep learning-based method for damage identification of smart building structures", Struct. Health Monitor., 18(1), 143-163. https://doi.org/10.1177/1475921718804132
  49. Zhang, A., Wang, K.C.P., Li, B.X., Yang, E.H., Dai, X.X., Peng, Y., Fei, Y., Liu, Y., Li, J.Q. and Chen, C. (2017), "Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network", Comput.-Aided Civil Infrastruct. Eng., 32(10), 805-819. https://doi.org/10.1111/mice.12297
  50. Zhang, Y.Q., Miyamori, Y., Mikami, S. and Saito, T. (2019), "Vibration-based structural state identification by a 1-dimensional convolutional neural network", Comput.-Aided Civ. Infrastruct. Eng., 34(9), 822-839. https://doi.org/10.1111/mice.12447