Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Optimal sensor placement for structural health monitoring based on deep reinforcement learning

  • Xianghao Meng (Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin Institute of Technology) ;
  • Haoyu Zhang (Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin Institute of Technology) ;
  • Kailiang Jia (Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin Institute of Technology) ;
  • Hui Li (Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin Institute of Technology) ;
  • Yong Huang (Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, School of Civil Engineering, Harbin Institute of Technology)
  • Received : 2022.03.11
  • Accepted : 2023.01.06
  • Published : 2023.03.25
⟨ Previous Next ⟩

Abstract

In structural health monitoring of large-scale structures, optimal sensor placement plays an important role because of the high cost of sensors and their supporting instruments, as well as the burden of data transmission and storage. In this study, a vibration sensor placement algorithm based on deep reinforcement learning (DRL) is proposed, which can effectively solve non-convex, high-dimensional, and discrete combinatorial sensor placement optimization problems. An objective function is constructed to estimate the quality of a specific vibration sensor placement scheme according to the modal assurance criterion (MAC). Using this objective function, a DRL-based algorithm is presented to determine the optimal vibration sensor placement scheme. Subsequently, we transform the sensor optimal placement process into a Markov decision process and employ a DRL-based optimization algorithm to maximize the objective function for optimal sensor placement. To illustrate the applicability of the proposed method, two examples are presented: a 10-story braced frame and a sea-crossing bridge model. A comparison study is also performed with a genetic algorithm and particle swarm algorithm. The proposed DRL-based algorithm can effectively solve the discrete combinatorial optimization problem for vibration sensor placements and can produce superior performance compared with the other two existing methods.

Keywords

Acknowledgement

The study was supported by National Key Research and Development Program of China under Grant No. 2021YFF0501003 and National Science Foundation of China under Grant Nos. U2139209 and 52078174.

References

  1. Akbarzadeh, V., Levesque, J.C., Gagne, C. and Parizeau, M. (2014), "Efficient sensor placement optimization using gradient descent and probabilistic coverage", Sensors, 14, 15525-15552. https://doi.org/10.3390/s140815525
  2. AlSaleh, R.J. and Clemente, F. (2020), "Combining GPS and accelerometers' records to capture torsional response of cylindrical tower", Smart Struct. Syst., Int. J., 25(1), 111-122. https://doi.org/10.12989/sss.2020.25.1.111
  3. Altunisik, A.C., Sevim, B., Sunca, F. and Okur, F.Y. (2021), "Optimal sensor placements for system identification of concrete arch dams", Adv. Concrete Constr., Int. J., 11(5), 397-407. https://doi.org/10.12989/acc.2021.11.5.397
  4. Borlenghi, P., Gentile, C. and Saisi, A. (2021), "Detecting and localizing anomalies on masonry towers from low-cost vibration monitoring", Smart Struct. Syst., Int. J., 27(2), 319-333. https://doi.org/10.12989/sss.2021.27.2.319
  5. Carne, T.G. and Dohrmann, C.R. (1994), A Modal Test Design Strategy for Model Correlation (No. SAND-94-2702C; CONF-950240-4), Sandia National Labs., Albuquerque, NM, USA.
  6. Chalioris, C.E., Voutetaki, M.E. and Liolios, A.A. (2020), "Structural health monitoring of seismically vulnerable RC frames under lateral cyclic loading", Earthq. Struct., Int. J., 19(1), 29-44. https://doi.org/10.12989/eas.2020.19.1.029
  7. Chang, C.C., Tsai, J., Lu, P.C. and Lai, C.A. (2020), "Accuracy Improvement of Autonomous Straight Take-off, Flying Forward, and Landing of a Drone with Deep Reinforcement Learning", Int. J. Computat. Intell. Syst., 13(1), 914-919. https://doi.org/10.2991/ijcis.d.200615.002
  8. Clegg, A., Erickson, Z., Grady, P., Turk, G., Kemp, C.C. and Liu, C.K. (2020), "Learning to collaborate from simulation for robot-assisted dressing", IEEE Robot. Automat. Lett., 5(2), 2746-2753. https://doi.org/10.1109/LRA.2020.2972852
  9. Ding, Z., Li, J. and Hao, H. (2020), "Structural damage identification by sparse deep belief network using uncertain and limited data", Struct. Control Health Monitor., 27(5), e2522. https://doi.org/10.1002/stc.2522
  10. Ewins, D.J. (1986), "Modal Testing: Theory and Practice", J. Vib. Acoust. Stress Reliabil. Des., 108(1), 109-110. https://doi.org/10.1115/1.3269294
  11. He, C., Xing, J., Li, J., Yang, Q., Wang, R. and Zhang, X. (2015), "A new optimal sensor placement strategy based on modified modal assurance criterion and improved adaptive genetic algorithm for structural health monitoring", Mathe. Probl. Eng., 11, 1-10. https://doi.org/10.1155/2015/626342
  12. Hosseini-Toudeshky, H. and Amjad, F.A. (2021), "Sensor placement optimization for guided wave-based structural health monitoring", Struct. Monitor. Maint., Int. J., 8(2), 125-150. https://doi.org/10.12989/SMM.2021.8.2.125
  13. Hou, R.R., Xia, Y. and Zhou, X.Q. (2019), "Genetic algorithm based optimal sensor placement for L1-regularized damage detection", Struct. Control Health Monitor., 26(1), e2274. https://doi.org/10.1002/stc.2274
  14. Huang, Y., Beck, J.L. and Li, H. (2017), "Bayesian system identification based on hierarchical sparse Bayesian learning and Gibbs sampling with application to structural damage assessment", Comput. Meth. Appl. Mech. Eng., 318 (2017), 382-411. https://doi.org/10.1016/j.cma.2017.01.030
  15. Huang, Y., Zhang, H., Li, H. and Wu, S. (2021), "Recovering compressed images for automatic crack segmentation using generative models", Mech. Syst. Signal Process., 146. https://doi.org/107061. 10.1016/j.ymssp.2020.107061
  16. Kalnoor, G. and Subrahmanyam, G. (2020), "A review on applications of Markov decision process model and energy efficiency in wireless sensor networks", Procedia Comput. Sci., 167, 2308-2317. https://doi.org/10.1016/j.procs.2020.03.283
  17. Kaloop, M.R., Elsharawy, M., Abdelwahed, B., Hu, J.W. and Kim, D. (2020), "Performance assessment of bridges using short-period structural health monitoring system: Sungsu bridge case study", Smart Struct. Syst., Int. J., 26(5), 667-680. https://doi.org/10.12989/sss.2020.26.5.667
  18. Kammer, D.C. (1991), "Sensor placement for on-orbit model identification and correlation of large space structures", J. Guid. Control Dyn., 14(2), 251-259. https://doi.org/10.2514/3.20635
  19. Krishna, A., Bartake, K., Niu, C., Wang, G., Lai, Y., Jia, X. and Mueller, K. (2021), "Image synthesis for data augmentation in medical ct using deep reinforcement learning", arXiv preprint arXiv:2103.10493. 2103.10493
  20. Lin, K., Gong, L., Li, X., Sun, T., Chen, B., Liu, C., Zhang, Z., Pu, J. and Zhang, J. (2020), "Exploration-efficient deep reinforcement learning with demonstration guidance for robot control", arXiv preprint arXiv:2002.12089.
  21. Liu, W., Gao, W.C., Sun, Y. and Xu, M.J. (2008), "Optimal sensor placement for spatial lattice structure based on genetic algorithms", J. Sound Vib., 317, 75-189. https://doi.org/10.1016/j.jsv.2008.03.026
  22. Liu, C.Y., Teng, J. and Zhen, P. (2020), "Optimal sensor placement for bridge damage detection using deflection influence line", Smart Struct. Syst., Int. J., 25(2), 169-181. https://doi.org/10.12989/sss.2020.25.2.169
  23. Lydon, D., Taylor, S.E., Lydon, M., del Rincon, J.M. and Hester, D. (2019), "Development and testing of a composite system for bridge health monitoring utilising computer vision and deep learning", Smart Struct. Syst., Int. J., 24(6), 723-732. https://doi.org/10.12989/sss.2019.24.6.723
  24. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M. (2013), "Playing atari with deep reinforcement learning", arXiv preprint, arXiv:1312.5602
  25. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G. and Petersen, S. (2015), "Human-level control through deep reinforcement learning", Nature, 518(7540), 529-533. https://doi.org/10.1038/nature14236
  26. Oh, J., Guo, X., Lee, H., Lewis, R.L. and Singh, S. (2015), "Action-conditional video prediction using deep networks in Atari games", Adv. Neural Inform. Process. Syst., 28.
  27. Qin, X., Zhan, P., Yu, C., Zhang, Q. and Sun, Y. (2021), "Health monitoring sensor placement optimization based on initial sensor layout using improved partheno-genetic algorithm", Adv. Struct. Eng., 24(2), 252-265. https://doi.org/10.1177/1369433220947198
  28. Quqa, S., Giordano, P.F., Limongelli, M.P., Landi, L. and Diotallevi, P.P. (2021), "Clump interpolation error for the identification of damage using decentralized sensor networks", Smart Struct. Syst., Int. J., 27(2), 351-363. https://doi.org/10.12989/sss.2021.27.2.351
  29. Rosafalco, L., Manzoni, A., Mariani, S. and Corigliano, A. (2020), "Fully convolutional networks for structural health monitoring through multivariate time series classification", Adv. Model. Simul. Eng. Sci., 7(1), 1-31. https://doi.org/10.1186/s40323-020-00174-1
  30. Shah, P. and Udwadia, F.E. (1978), "A methodology for optimal sensor locations for identification of dynamic systems", J. Appl. Mech. Transact. ASME, 45(1), 188-196. https://doi.org/10.1115/1.3424225
  31. Singh, Y., Sharma, S., Sutton, R. and Hatton, D. (2017), "Path planning of an autonomous surface vehicle based on artificial potential fields in a real time marine environment", Proceedings of the 16 International Conference on Computer and IT Applications in the Maritime Industries, May.
  32. Sun, H. and Buyukozturk, O. (2015), "Optimal sensor placement in structural health monitoring using discrete optimization", Smart Mater. Struct., 24(12), 125. 10.1088/0964-1726/24/12/125034
  33. Sutton, R.S. and Barto, A.G. (1998), "Reinforcement Learning: An Introduction", IEEE Transact. Neural Networks.
  34. Van Hasselt, H., Guez, A. and Silver, D. (2016), "Deep reinforcement learning with double q-learning", Proceedings of the AAAI Conference on Artificial Intelligence.
  35. Wang, Z., Li, H.X. and Chen, C.L. (2019), "Reinforcement learning-based optimal sensor placement for spatiotemporal modeling", IEEE Transact. Cybernet., 50(6), 2861-2871. https://doi.org/10.1109/TCYB.2019.2901897
  36. Wang, R., Li, J., An, S., Hao, H., Liu, W. and Li, L. (2021), "Densely connected convolutional networks for vibration based structural damage identification", Eng. Struct., 245, 112871. https://doi.org/10.1016/j.engstruct.2021.112871
  37. Wei, S.Y., Jin, X.W. and Li, H. (2019), "General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning", Computat. Mech., 64(5), 1361-1374. https://doi.org/10.1007/s00466-019-01715-1
  38. Wu, W.H., Jhou, J.W., Chen, C.C. and Lai, G. (2019), "A novel recursive stochastic subspace identification algorithm with its application in long-term structural health monitoring of office buildings", Smart Struct. Syst., Int. J., 24(4), 459-474. https://doi.org/10.12989/sss.2019.24.4.459
  39. Xu, C., Zhang, D., Chong, J., Chen, B. and Li, S. (2021), "Synthesis of gadolinium-enhanced liver tumors on nonenhanced liver MR images using pixel-level graph reinforcement learning", Med. Image Anal., 69, p. 101976. https://doi.org/10.1016/j.media.2021.101976
  40. Yang, C. and Lu, Z.X. (2017), "An interval effective independence method for optimal sensor placement based on non-probabilistic approach", Inst. Solid Mech., 60(2), 186-198. https://doi.org/10.1007/s11431-016-0526-9
  41. Yi, T.H., Li, H.N. and Zhang, X.D. (2015), "Health monitoring sensor placement optimization for Canton Tower using immune monkey algorithm", Struct. Control Health Monitor., 22(1), 123-138. https://doi.org/10.1002/stc.1664
  42. Yin, T., Yuen, K.V., Lam, H.F. and Zhu, H. (2017), "Entropy-based optimal sensor placement for model identification of periodic structures endowed with bolted joints", Comput.-Aided Civil Infrastr. Eng., 32(12), 1007-1024. https://doi.org/10.1111/mice.12309
  43. Yuen, K.V., Beck, J.L. and Katafygiotis, L.S. (2006), "Efficient model updating and health monitoring methodology using incomplete modal data without mode matching", Struct. Control Health Monitor., 13, 91-107. https://doi.org/10.1002/stc.144
  44. Yurtsever, E., Capito, L., Redmill, K. and Ozgune, U. (2020), "Information-driven distributed maximum likelihood estimation based on Gauss-Newton method in wireless sensor networks", In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1311-1316. https://doi.org/10.1109/IV47402.2020.9304735
  45. Zhao, T. and Nehorai, A. (2007), "Information-driven distributed maximum likelihood estimation based on Gauss-Newton method in wireless sensor networks", IEEE Transact. Signal Process., 55(9), 4669-4682. https://doi.org/10.1109/TSP.2007.896267
  46. Zhao, T., Wang, P. and Li, S. (2019), "Traffic signal control with deep reinforcement learning", Proceedings of 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China, December. https://doi.org/10.1109/ICICAS48597.2019.00164