Geophysics and Geophysical Exploration (지구물리와물리탐사)

Korean Society of Earth and Exploration Geophysicists (한국지구물리·물리탐사학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis on Strategies for Modeling the Wave Equation with Physics-Informed Neural Networks

물리정보신경망을 이용한 파동방정식 모델링 전략 분석

  • Sangin Cho (Department of Energy Resources Engineering, Inha University) ;
  • Woochang Choi (Department of Energy Resources Engineering, Inha University) ;
  • Jun Ji (Department of AI Application, Hansung University) ;
  • Sukjoon Pyun (Department of Energy Resources Engineering, Inha University)
  • 조상인 (인하대학교 에너지자원공학과) ;
  • 최우창 (인하대학교 에너지자원공학과) ;
  • 지준 (한성대학교 AI응용학과) ;
  • 편석준 (인하대학교 에너지자원공학과)
  • Received : 2023.07.03
  • Accepted : 2023.07.20
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The physics-informed neural network (PINN) has been proposed to overcome the limitations of various numerical methods used to solve partial differential equations (PDEs) and the drawbacks of purely data-driven machine learning. The PINN directly applies PDEs to the construction of the loss function, introducing physical constraints to machine learning training. This technique can also be applied to wave equation modeling. However, to solve the wave equation using the PINN, second-order differentiations with respect to input data must be performed during neural network training, and the resulting wavefields contain complex dynamical phenomena, requiring careful strategies. This tutorial elucidates the fundamental concepts of the PINN and discusses considerations for wave equation modeling using the PINN approach. These considerations include spatial coordinate normalization, the selection of activation functions, and strategies for incorporating physics loss. Our experimental results demonstrated that normalizing the spatial coordinates of the training data leads to a more accurate reflection of initial conditions in neural network training for wave equation modeling. Furthermore, the characteristics of various functions were compared to select an appropriate activation function for wavefield prediction using neural networks. These comparisons focused on their differentiation with respect to input data and their convergence properties. Finally, the results of two scenarios for incorporating physics loss into the loss function during neural network training were compared. Through numerical experiments, a curriculum-based learning strategy, applying physics loss after the initial training steps, was more effective than utilizing physics loss from the early training steps. In addition, the effectiveness of the PINN technique was confirmed by comparing these results with those of training without any use of physics loss.

편미분방정식의 해를 구하기 위한 여러 수치해법들의 한계와 순수 데이터 기반 기계학습의 단점을 극복하기 위해 물리정보신경망(physics-informed neural network, PINN)이 제안되었다. 물리정보신경망은 편미분방정식을 손실함수 구성에 직접 활용하여 기계학습 훈련에 물리적 제약을 주는 기법으로 파동방정식 모델링에도 활용될 수 있다. 그러나 물리정보신경망을 이용하여 파동방정식을 풀기 위해서는 신경망 훈련 시 입력에 대한 2차 미분이 수행되어야 하고, 그 결과로 출력되는 파동장은 복잡한 역학적 현상들을 포함하고 있어 섬세한 전략이 필요하다. 이 해설 논문에서는 물리정보신경망의 기본 개념을 설명하고 파동방정식 모델링에 활용하기 위한 고려사항들에 대해 고찰하였다. 이러한 고려사항에는 공간좌표 정규화, 활성함수 선정, 물리손실 추가 전략이 포함된다. 훈련자료의 공간좌표를 정규화한 후 사용하면 파동방정식 모델링을 위한 신경망 훈련에서 초기 조건이 더 정확하게 반영되는 것을 수치 실험을 통해 보였다. 또한 신경망을 통한 파동장 예측에 가장 적절한 활성함수를 선정하기 위해 여러 함수들의 특성을 비교했다. 특성 비교는 각 활성함수들의 입력자료에 대한 미분과 수렴성을 중심으로 이루어졌다. 마지막으로 신경망 훈련 중 손실함수에 물리손실을 추가하는 두가지 시나리오의 결과를 비교하였다. 수치 실험을 통해 훈련 초기부터 물리손실을 활용하는 전략보다 초기 훈련단계 이후부터 물리손실을 적용하는 커리큘럼 기반 학습전략이 효과적이라는 결과를 도출했다. 추가로 이 결과를 물리손실을 전혀 사용하지 않은 훈련 결과와 비교하여 PINN기법의 효과를 확인하였다.

Keywords

Acknowledgement

이 논문은 2023년도 정부(산업통상자원부)의 재원으로 한국 에너지기술평가원의 지원을 받아 수행된 연구임(20226A10100030, 고성능 해양 CO2 모니터링 기술개발).

References

  1. Alkhadhr, S., and Almekkawy, M., 2023, Wave Equation Modeling via Physics-Informed Neural Networks: Models of Soft and Hard Constraints for Initial and Boundary Conditions, Sensors, 23(5), 2792, doi: 10.3390/s23052792
  2. Arridge, S., Maass, P., Oktem, O., and Schonlieb, C. B., 2019, Solving inverse problems using data-driven models, Acta Numer., 28, 1-174, doi: 10.1017/S0962492919000059
  3. Baydin, A. G., Pearlmutter, B. A., Radul, A. A., and Siskind, J. M., 2018, Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18, 1-43. https://www.jmlr.org/papers/volume18/17-468/17-468.pdf
  4. Blechschmidt, J., and Ernst, O. G., 2021, Three ways to solve partial differential equations with neural networks-A review. GAMM-Mitteilungen, 44(2), e202100006, doi: 10.1002/gamm.202100006
  5. Brandolin, F., Ravasi, M., and Alkhalifah, T., 2022, PWD-PINN: Slope-assisted seismic interpolation with physics-informed neural networks, In Second International Meeting for Applied Geoscience & Energy (pp. 2646-2650). Society of Exploration Geophysicists and American Association of Petroleum Geologists. doi: 10.1190/image2022-3742422.1
  6. Cai, S., Mao, Z., Wang, Z., Yin, M., and Karniadakis, G. E., 2021, Physics-informed neural networks (PINNs) for fluid mechanics: A review, Acta Mech. Sin., 37(12), 1727-1738, doi: 10.48550/arXiv.2105.09506
  7. Chapra, S, 2011, EBOOK: Applied Numerical Methods with MATLAB for Engineers and Scientists, McGraw Hill.
  8. Chen, Y., Lu, L., Karniadakis, G. E., and Dal Negro, L., 2020, Physics-informed neural networks for inverse problems in nano-optics and metamaterials, Opt. Express, 28(8), 11618-11633, doi: 10.1364/OE.384875
  9. Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., and Piccialli, F., 2022, Scientific machine learning through physics-informed neural networks: where we are and what's next, J. Sci. Comput., 92(3), 88, doi: 10.1007/s10915-022-01939-z
  10. Dissanayake, M. W. M. G., and Phan-Thien, N., 1994, Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., 10(3), 195-201, doi: 10.1002/cnm.1640100303
  11. Erichson, N. B., Muehlebach, M., and Mahoney, M. W., 2019, Physics-informed autoencoders for Lyapunov-stable fluid flow prediction, arXiv preprint arXiv:1905.10866, doi: 10.48550/arXiv.1905.10866
  12. Garcia Ferrero, M. A., 2018, Global approximation theorems for partial differential equations and applications, Ph.D. thesis, Universidad Complutense de Madrid. https://www.icmat.es/Thesis/2018/Tesis_Ma_Angeles_Garcia_Ferrero.pdf
  13. Geron, A., 2022, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow, O'Reilly Media, Inc.
  14. Haghighat, E., Raissi, M., Moure, A., Gomez, H., and Juanes, R., 2021, A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics, Comput. Methods Appl. Mech. Eng., 379, 113741, doi: 10.1016/j.cma.2021.113741
  15. Heldmann, F., Berkhahn, S., Ehrhardt, M., and Klamroth, K., 2023, PINN training using biobjective optimization: The trade-off between data loss and residual loss, J. Comput. Phys., 488, 112211, doi: 10.1016/j.jcp.2023.112211
  16. Hornik, K., Stinchcombe, M., and White, H., 1989, Multilayer feedforward networks are universal approximators, Neural Netw., 2(5), 359-366, doi: 10.1016/0893-6080(89)90020-8
  17. Jagtap, A. D., Kawaguchi, K., and Karniadakis, G. E., 2020, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, J. Comput. Phys., 404, 109136, doi: 10.1016/j.jcp.2019.109136
  18. Junior, A. C. N., Almeida, J. D. S., Quinones, M. P., and de Albuquerque Martins, L. S., 2019, Physics-based machine learning inversion of subsurface elastic properties, In 81st EAGE Conference and Exhibition 2019 (Vol. 2019, No. 1, pp. 1-5). European Association of Geoscientists & Engineers, doi: 10.3997/2214-4609.201901147
  19. Karimpouli, S., and Tahmasebi, P., 2020, Physics informed machine learning: Seismic wave equation, Geosci. Front., 11(6), 1993-2001, doi: 10.1016/j.gsf.2020.07.007
  20. Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., and Yang, L., 2021, Physics-informed machine learning, Nat. Rev. Phys., 3(6), 422-440, doi: 10.1038/s42254-021-00314-5
  21. Kasim, M., Watson-Parris, D., Deaconu, L., Oliver, S., Hatfield, P., Froula, D., Gregori, G., Jarvis, M., Khatiwala, S., Korenaga, J., Topp-Mugglestone, J., Viezzer, E., and Vinko, S., 2020, Building high accuracy emulators for scientific simulations with deep neural architecture search, arXiv preprint arXiv:2001.08055, doi: 10.48550/arXiv.2001.08055
  22. Kreyszig, E., 2011, Advanced Engineering Mathematics, 10th Ed, Wiley.
  23. Kondor, R., and Trivedi, S., 2018, On the generalization of equivariance and convolution in neural networks to the action of compact groups, In International Conference on Machine Learning (pp. 2747-2755). https://proceedings.mlr.press/v80/kondor18a.html
  24. Lagaris, I. E., Likas, A., and Fotiadis, D. I., 1998, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9(5), 987-1000, doi: 10.1109/72.712178
  25. Lee, H., and Kang, I. S., 1990, Neural algorithm for solving differential equations, J. Comput. Phys., 91(1), 110-131, doi: 10.1016/0021-9991(90)90007-N
  26. Lim, K. L., Dutta, R., and Rotaru, M., 2022, Physics Informed Neural Network using Finite Difference Method, In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 1828-1833), doi: 10.1109/SMC53654.2022.9945171
  27. Mallat, S., 2016, Understanding deep convolutional networks, Philos. Trans. Royal Soc. A, 374(2065), 20150203, doi: 10.1098/rsta.2015.0203
  28. Mishra, S., and Molinaro, R., 2022, Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs, IMA J. Numer. Anal., 42(2), 981-1022, doi: 10.1093/imanum/drab032
  29. Moseley, B., Nissen-Meyer, T., and Markham, A., 2020a, Deep learning for fast simulation of seismic waves in complex media, Solid Earth, 11(4), 1527-1549, doi: 10.5194/se-11-1527-2020
  30. Moseley, B., Markham, A., and Nissen-Meyer, T., 2020b, Solving the wave equation with physics-informed deep learning, arXiv preprint arXiv:2006.11894, doi: 10.48550/arXiv.2006.11894
  31. Moseley, B., Markham, A., and Nissen-Meyer, T., 2021, Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations, arXiv preprint arXiv:2107.07871, doi: 10.48550/arXiv.2107.07871
  32. Paganini, M., de Oliveira, L., and Nachman, B., 2018, Accelerating science with generative adversarial networks: an application to 3D particle showers in multilayer calorimeters, Phys. Rev. Lett., 120(4), 042003, doi: 10.1103/PhysRevLett.120.042003
  33. Psichogios, D. C., and Ungar, L. H., 1992, A hybrid neural network-first principles approach to process modeling, AIChE J., 38(10), 1499-1511, doi: 10.1002/aic.690381003
  34. PyTorch, 2018, URL https://www.pytorch.org
  35. Raissi, M., Perdikaris, P., and Karniadakis, G. E., 2019, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707, doi: 10.1016/j.jcp.2018.10.045
  36. Raissi, M., Yazdani, A., and Karniadakis, G. E., 2020, Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Sci., 367(6481), 1026-1030, doi: 10.1126/science.aaw4741
  37. Rasht-Behesht, M., Huber, C., Shukla, K., and Karniadakis, G. E., 2022, Physics-Informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions. J. Geophys. Res. Solid Earth, 127(5), e2021JB023120, doi: 10.1029/2021JB023120
  38. Rasp, S., Pritchard, M. S., and Gentine, P., 2018, Deep learning to represent subgrid processes in climate models, Proceedings of the National Academy of Sciences, 115(39), 9684-9689, doi: 10.1073/pnas.1810286115
  39. Rumelhart, D. E., Hinton, G. E., and Williams, R. J., 1985, Learning internal representations by error propagation, California Univ San Diego La Jolla Inst for Cognitive Science. https://apps.dtic.mil/sti/citations/ADA164453
  40. Sahli Costabal, F., Yang, Y., Perdikaris, P., Hurtado, D. E., and Kuhl, E., 2020, Physics-informed neural networks for cardiac activation mapping, Front. Phys., 8, 42, doi: 10.3389/fphy.2020.00042
  41. Shin, Y., Darbon, J., and Karniadakis, G. E., 2020, On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs, Commun. Comput. Phys., 28(5), 2042-2074, doi: 10.4208/cicp.OA2020-0193
  42. Siahkoohi, A., Louboutin, M., and Herrmann, F. J., 2019, The importance of transfer learning in seismic modeling and imaging, Geophysics, 84(6), A47-A52, doi: 10.1190/geo2019-0056.1
  43. Sitzmann, V., Martel, J., Bergman, A., Lindell, D., and Wetzstein, G., 2020, Implicit neural representations with periodic activation functions, Adv. Neural Inf. Process. Syst., 33, 7462-7473, https://proceedings.neurips.cc/paper/2020/hash/53c04118df112c13a8c34b38343b9c10-Abstract.html 10-Abstract.html
  44. Smith, J. D., Azizzadenesheli, K., and Ross, Z. E., 2020, Eikonet: Solving the eikonal equation with deep neural networks, IEEE Trans. Geosci. Remote Sens., 59(12), 10685-10696, doi: 10.1109/TGRS.2020.3039165
  45. Song, C., Alkhalifah, T., and Waheed, U. B., 2021, Solving the frequency-domain acoustic VTI wave equation using physics-informed neural networks, Geophys. J. Int., 225(2), 846-859, doi: 10.1093/gji/ggab010
  46. Sun, L., Gao, H., Pan, S., and Wang, J. X., 2020, Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Comput. Methods Appl. Mech. Eng., 361, 112732, doi: 10.1016/j.cma.2019.112732
  47. TensorFlow, 2015, URL https://www.tensorflow.org
  48. Waheed, U. B., Haghighat, E., and Alkhalifah, T., 2020, Anisotropic eikonal solution using physics-informed neural networks, 90th Ann. Internat. Mtg., Soc. of Expl. Geophys., Expanded Abstracts, 1566-1570, doi: 10.1190/segam2020-3423159.1
  49. Waheed, U. B., Haghighat, E., Alkhalifah, T., Song, C., and Hao, Q., 2021, PINNeik: Eikonal solution using physics-informed neural networks. Computers & Geosciences, 155, 104833, doi: 10.1016/j.cageo.2021.104833
  50. Wang, S., Yu, X., and Perdikaris, P., 2022, When and why PINNs fail to train: A neural tangent kernel perspective, J. Comput. Phys., 449, 110768, doi: 10.1016/j.jcp.2021.110768
  51. Xu, Y., Li, J., and Chen, X., 2019, Physics informed neural networks for velocity inversion, 89th Ann. Internat. Mtg., Soc. Expl. Geophys., Expanded Abstracts, 2584-2588, doi: 10.1190/segam2019-3216823.1
  52. Yilmaz, O., 2001, Seismic data analysis: Processing, inversion, and interpretation of seismic data, Society of exploration geophysicists, doi: 10.1190/1.9781560801580
  53. Zhu, Y., Zabaras, N., Koutsourelakis, P. S., and Perdikaris, P., 2019, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, J. Comput. Phys., 394, 56-81, doi: 10.1016/j.jcp.2019.05.024