Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

  • Tarutal Ghosh Mondal (Civil, Architectural and Environmental Engineering, Missouri University of Science and Technology) ;
  • Jau-Yu Chou (Department of Civil Engineering, National Taiwan University) ;
  • Yuguang Fu (School of Civil and Environmental Engineering, Nanyang Technological University) ;
  • Jianxiao Mao (Key Laboratory of C&PC Structures of Ministry of Education, Southeast University)
  • Received : 2022.11.30
  • Accepted : 2023.10.06
  • Published : 2023.09.25
⟨ Previous Next ⟩

Abstract

This study explores an alternative to the existing centralized process for data anomaly detection in modern Internet of Things (IoT)-based structural health monitoring (SHM) systems. An edge intelligence framework is proposed for the early detection and classification of various data anomalies facilitating quality enhancement of acquired data before transmitting to a central system. State-of-the-art deep neural network pruning techniques are investigated and compared aiming to significantly reduce the network size so that it can run efficiently on resource-constrained edge devices such as wireless smart sensors. Further, depthwise separable convolution (DSC) is invoked, the integration of which with advanced structural pruning methods exhibited superior compression capability. Last but not least, quantization-aware training (QAT) is adopted for faster processing and lower memory and power consumption. The proposed edge intelligence framework will eventually lead to reduced network overload and latency. This will enable intelligent self-adaptation strategies to be employed to timely deal with a faulty sensor, minimizing the wasteful use of power, memory, and other resources in wireless smart sensors, increasing efficiency, and reducing maintenance costs for modern smart SHM systems. This study presents a theoretical foundation for the proposed framework, the validation of which through actual field trials is a scope for future work.

Keywords

Acknowledgement

This research/project is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-TC-2021-001), the Ministry of Education Tier 1 Grants, Singapore (No. RG121/21), and the start-up grant at Nanyang Technological University, Singapore (03INS001210C120).

References

  1. Al-amri, R., Murugesan, R.K., Man, M., Abdulateef, A.F., Al-Sharafi, M.A. and Alkahtani, A.A. (2021), "A review of machine learning and deep learning techniques for anomaly detection in IoT data", Appl. Sci., 11(12), 5320. https://doi.org/10.3390/app11125320
  2. Alavi, A.H., Jiao, P., Buttlar, W.G. and Lajnef, N. (2018), "Internet of things-enabled smart cities: State-of-the-art and future trends", Measurement, 129, 589-606. https://doi.org/10.1016/j.measurement.2018.07.067
  3. Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z. and Li, H. (2019), "The state of the art of data science and engineering in structural health monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027
  4. Bao, Y., Li, J., Nagayama, T., Xu, Y., Spencer Jr, B.F. and Li, H. (2021), "The 1st international project competition for structural health monitoring (IPC-SHM, 2020): A summary and benchmark problem", Struct. Health Monitor., 20(4), 2229-2239. https://doi.org/10.1177/14759217211006485
  5. Bengio, Y., Leonard, N. and Courville, A. (2013), "Estimating or propagating gradients through stochastic neurons for conditional computation", arXiv preprint arXiv:1308.3432. https://doi.org/10.48550/arXiv.1308.3432
  6. Bisio, I., Garibotto, C., Lavagetto, F. and Sciarrone, A. (2022), "A novel IoT-based edge sensing platform for structure health monitoring", IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), NY, USA, May, pp. 1-6.
  7. Chatterjee, A. and Ahmed, B.S. (2022), "IoT anomaly detection methods and applications: A survey", Internet of Things, 19, 100568. https://doi.org/10.1016/j.iot.2022.100568
  8. Chollet, F. (2017), "Xception: Deep learning with depthwise separable convolutions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, July, pp. 1251-1258.
  9. Chou, J.Y., Fu, Y., Huang, S.K. and Chang, C.M. (2022), "SHM data anomaly classification using machine learning strategies: A comparative study", Smart Struct. Syst., Int. J., 29(1), 77-91. https://doi.org/10.12989/sss.2022.29.1.077
  10. Cook, A.A., Misirli, G. and Fan, Z. (2019), "Anomaly detection for IoT time-series data: A survey", IEEE Internet Things J., 7(7), 6481-6494. https://doi.org/10.1109/JIOT.2019.2958185
  11. Denil, M., Shakibi, B., Dinh, L., Ranzato, M. and De Freitas, N. (2013), "Predicting parameters in deep learning", Adv. Neural Inform. Process. Syst., 26.
  12. Ding, X., Zhou, X., Guo, Y., Han, J. and Liu, J. (2019), "Global sparse momentum SGD for pruning very deep neural networks", Adv. Neural Inform. Process. Syst., 32.
  13. Du, Y., Li, L.-f., Hou, R.-r., Wang, X.-y., Tian, W. and Xia, Y. (2022), "Convolutional neural network-based data anomaly detection considering class imbalance with limited data", Smart Struct. Syst., Int. J., 29(1), 63-75. https://doi.org/10.12989/sss.2022.29.1.063
  14. Frankle, J. and Carbin, M. (2018), "The lottery ticket hypothesis: Finding sparse, trainable neural networks", arXiv preprint arXiv:1803.03635. https://doi.org/10.48550/arXiv.1803.03635
  15. Frey, B.J. and Dueck, D. (2007), "Clustering by passing messages between data points", Science, 315(5814), 972-976. https://doi.org/10.1126/science.1136800
  16. Fu, Y., Zhu, L., Hoang, T., Mechitov, K. and Spencer Jr, B.F. (2018), "Demand-based wireless smart sensors for earthquake monitoring of civil infrastructure", In: Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2018, Vol. 10598, pp. 245-251. https://doi.org/10.1117/12.2296634
  17. Fu, Y., Peng, C., Gomez, F., Narazaki, Y. and Spencer Jr, B.F. (2019), "Sensor fault management techniques for wireless smart sensor networks in structural health monitoring", Struct. Control Health Monitor., 26(7), e2362. https://doi.org/10.1002/stc.2362
  18. Fu, Y., Zhu, Y., Hoang, T., Mechitov, K. and Spencer Jr, B.F. (2022), "xImpact: Intelligent Wireless System for Cost-Effective Rapid Condition Assessment of Bridges under Impacts", Sensors, 22(15), 5701. https://doi.org/10.3390/s22155701
  19. Gao, K., Chen, Z.-D., Weng, S., Zhu, H.-P. and Wu, L.-Y. (2022), "Detection of multi-type data anomaly for structural health monitoring using pattern recognition neural network", Smart Struct. Syst., Int. J., 29(1), 129-140. https://doi.org/10.12989/sss.2022.29.1.129
  20. Ghosh Mondal, T., Jahanshahi, M.R. and Wu, Z.Y. (2023), "Deep Learning-Based RGB-D Fusion for Multimodal Condition Assessment of Civil Infrastructure", J. Comput. Civil Eng., 37(4), 04023017. https://doi.org/10.1061/JCCEE5.CPENG-51
  21. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., WardeFarley, D., Ozair, S., Courville, A. and Bengio, Y. (2014), "Generative adversarial nets", Adv. Neural Inform. Process. Syst., 27. https://doi.org/10.1145/3422622
  22. Han, S., Mao, H. and Dally, W.J. (2015), "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding", arXiv preprint arXiv:1510.00149. https://doi.org/10.48550/arXiv.1510.00149
  23. Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M.A. and Dally, W.J. (2016), "EIE: efficient inference engine on compressed deep neural network", ACM SIGARCH Comput. Architect. News, 44(3), 243-254. https://doi.org/10.1145/3007787.3001163
  24. Haque, M.E., Asikuzzaman, M., Khan, I.U., Ra, I.-H., Hossain, M.S. and Shah, S.B.H. (2020), "Comparative study of IoT-based topology maintenance protocol in a wireless sensor network for structural health monitoring", Remote Sensing, 12(15), 2358. https://doi.org/10.3390/rs12152358
  25. Hassibi, B. and Stork, D. (1992), "Second order derivatives for network pruning: Optimal brain surgeon", Adv. Neural Inform. Process. Syst., 5.
  26. He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification", Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, December, pp. 1026-1034.
  27. He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA, June, pp. 770-778.
  28. He, Y., Zhang, X. and Sun, J. (2017), "Channel pruning for accelerating very deep neural networks", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October, pp. 1389-1397.
  29. Hou, S. and Wu, G. (2019), "A low-cost IoT-based wireless sensor system for bridge displacement monitoring", Smart Mater. Struct., 28(8), 085047. https://doi.org/10.1088/1361-665X/ab2a31
  30. Hu, H., Peng, R., Tai, Y.-W. and Tang, C.-K. (2016), "Network trimming: A data-driven neuron pruning approach towards efficient deep architectures", arXiv preprint arXiv:1607.03250. https://doi.org/10.48550/arXiv.1607.03250
  31. Huang, Z. and Wang, N. (2018), "Data-driven sparse structure selection for deep neural networks", Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, September, pp. 304-320.
  32. Huang, Y.T., Jahanshahi, M.R., Shen, F. and Mondal, T.G. (2023). "Deep Learning-Based Autonomous Road Condition Assessment Leveraging Inexpensive RGB and Depth Sensors and Heterogeneous Data Fusion: Pothole Detection and Quantification", J. Transport. Eng., Part B: Pavements, 149(2), 04023010. https://doi.org/10.1061/JPEODX.PVENG-1194
  33. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H. and Kalenichenko, D. (2018), "Quantization and training of neural networks for efficient integer-arithmetic-only inference", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June, pp. 2704-2713.
  34. LeCun, Y., Denker, J. and Solla, S. (1989), "Optimal brain damage", Adv. Neural Inform. Process. Syst., 2.
  35. Li, H., Kadav, A., Durdanovic, I., Samet, H. and Graf, H.P. (2016), "Pruning filters for efficient convnets", arXiv preprint arXiv:1608.08710. https://doi.org/10.48550/arXiv.1608.08710
  36. Liberty, E. (2013), "Simple and deterministic matrix sketching", Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, USA, August, pp. 581-588.
  37. Lin, S., Ji, R., Yan, C., Zhang, B., Cao, L., Ye, Q., Huang, F. and Doermann, D. (2019), "Towards optimal structured CNN pruning via generative adversarial learning", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, June, pp. 2790-2799.
  38. Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y. and Shao, L. (2020), "HRank: filter pruning using high-rank feature map", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June, pp. 1529-1538.
  39. Lin, M., Cao, L., Li, S., Ye, Q., Tian, Y., Liu, J., Tian, Q. and Ji, R. (2021a), "Filter sketch for network pruning", IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2021.3084206
  40. Lin, M., Ji, R., Li, S., Wang, Y., Wu, Y., Huang, F. and Ye, Q. (2021b), "Network pruning using adaptive exemplar filters", IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2021.3084856.
  41. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S. and Zhang, C. (2017), "Learning efficient convolutional networks through network slimming", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October, pp. 2736-2744.
  42. Liu, Z., Mu, H., Zhang, X., Guo, Z., Yang, X., Cheng, K.-T. and Sun, J. (2019), "Metapruning: Meta learning for automatic neural network channel pruning", Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, October, pp. 3296-3305.
  43. Liu, G., Niu, Y., Zhao, W., Duan, Y. and Shu, J. (2022), "Data anomaly detection for structural health monitoring using a combination network of GANomaly and CNN", Smart Struct. Syst., Int. J., 29(1), 53-62. https://doi.org/10.12989/sss.2022.29.1.053
  44. Luo, J.-H., Wu, J. and Lin, W. (2017), "ThiNet: a filter level pruning method for deep neural network compression", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October, pp. 5058-5066.
  45. Martakis, P., Movsessian, A., Reuland, Y., Pai, S.G., Quqa, S., Cava, D.G., Tcherniak, D. and Chatzi, E. (2021), "A semi-supervised interpretable machine learning framework for sensor fault detection", Smart Struct. Syst., Int. J., 29(1), 251-266. https://doi.org/10.12989/sss.2022.29.1.251
  46. Mishra, M., Lourenco, P.B. and Ramana, G.V. (2022), "Structural health monitoring of civil engineering structures by using the internet of things: A review", J. Build. Eng., 48, 103954. https://doi.org/10.1016/j.jobe.2021.103954
  47. Mondal, T.G. (2021), "Development of Multimodal Fusion-Based Visual Data Analytics for Robotic Inspection and Condition Assessment", Doctoral dissertation, Purdue University, West Lafayette, IN, USA.
  48. Nur, A.B.S. (2022), Knowledge distillation in deep neural network compression for artificial intelligent of things, Final Year Project Report, Nanyang Technological University.
  49. Park, J., Li, S., Wen, W., Tang, P.T.P., Li, H., Chen, Y. and Dubey, P. (2016), "Faster CNNs with direct sparse convolutions and guided pruning", arXiv preprint arXiv:1608.01409. https://doi.org/10.48550/arXiv.1608.01409
  50. Peng, C., Fu, Y. and Spencer Jr., B.F. (2017), "Sensor fault detection, identification, and recovery techniques for wireless sensor networks: A full-scale study", Proceedings of the 13th International Workshop on Advanced Smart Materials and Smart Structures Technology, Tokyo, Japan, July, pp. 22-23.
  51. Peralta Abadia, J., Fritz, H., Dragos, K. and Smarsly, K. (2022), "Sensor fault diagnosis coupling deep learning and wavelet transforms", Proceedings of the 13th International Workshop on Structural Health Monitoring, IWSHM 2021, Standford, CA, USA, September, pp. 779-785.
  52. Shajihan, S.A., Wang, S., Zhai, G. and Spencer Jr, B.F. (2022), "CNN based data anomaly detection using multi-channel imagery for structural health monitoring", Smart Struct. Syst., Int. J., 29(1), 181-193. https://doi.org/10.12989/sss.2022.29.1.181
  53. Singh, P., Verma, V.K., Rai, P. and Namboodiri, V.P. (2019), "Play and prune: Adaptive filter pruning for deep model compression", arXiv preprint arXiv:1905.04446. https://doi.org/10.48550/arXiv.1905.04446
  54. Sun, Y., Zheng, L., Deng, W. and Wang, S. (2017), "SVDNet for pedestrian retrieval", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October, pp. 3800-3808.
  55. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015), "Going deeper with convolutions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June, pp. 1-9.
  56. Tang, Z., Chen, Z., Bao, Y. and Li, H. (2019), "Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring", Struct. Control Health Monitor., 26(1), e2296. https://doi.org/10.1002/stc.2296
  57. Wen, W., Wu, C., Wang, Y., Chen, Y. and Li, H. (2016), "Learning structured sparsity in deep neural networks", Adv. Neural Inform. Process. Syst., 29.
  58. Wu, R.-T., Singla, A., Jahanshahi, M.R., Bertino, E., Ko, B.J. and Verma, D. (2019), "Pruning deep convolutional neural networks for efficient edge computing in condition assessment of infrastructures", Comput.-Aided Civil Infrastr. Eng., 34(9), 774-789. https://doi.org/10.1111/mice.12449
  59. Xu, J., Dang, D., Ma, Q., Liu, X. and Han, Q. (2022), "A novel and robust data anomaly detection framework using LAL-AdaBoost for structural health monitoring", J. Civil Struct. Health Monitor., 12(2), 305-321. https://doi.org/10.1007/s13349-021-00544-2
  60. Yang, K., Jiang, H., Diang, Y., Wang, M. and Wan, C. (2022), "Data abnormal detection using bidirectional long-short neural network combined with artificial experience", Smart Struct. Syst., Int. J., 29(1), 117-127. https://doi.org/10.12989/sss.2022.29.1.117
  61. Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V.I., Han, X., Gao, M., Lin, C.-Y. and Davis, L.S. (2018), "NISP: pruning networks using neuron importance score propagation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June, pp. 9194-9203.
  62. Zhang, Z., Yan, J., Li, L., Pan, H. and Dong, C. (2021), "Condition assessment of stay cables through enhanced time series classification using a deep learning approach", Smart Struct. Syst., Int. J., 29(1), 105-116. https://doi.org/10.48550/arXiv.2101.03701
  63. Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W. and Tian, Q. (2019), "Variational convolutional neural network pruning", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, June, pp. 2780-2789.