Geophysics and Geophysical Exploration (지구물리와물리탐사)

Korean Society of Earth and Exploration Geophysicists (한국지구물리·물리탐사학회)
권호 표지
DOI QR코드

DOI QR Code

Hyperparameter Search for Facies Classification with Bayesian Optimization

베이지안 최적화를 이용한 암상 분류 모델의 하이퍼 파라미터 탐색

  • Choi, Yonguk (Dept. Energy & Resources Engineering, Chonnam National University) ;
  • Yoon, Daeung (Dept. Energy & Resources Engineering, Chonnam National University) ;
  • Choi, Junhwan (Dept. of Earth Resources and Environmental Engineering, Hanyang University) ;
  • Byun, Joongmoo (Dept. of Earth Resources and Environmental Engineering, Hanyang University)
  • 최용욱 (전남대학교 에너지자원공학과) ;
  • 윤대웅 (전남대학교 에너지자원공학과) ;
  • 최준환 (한양대학교 자원환경공학과) ;
  • 변중무 (한양대학교 자원환경공학과)
  • Received : 2020.06.16
  • Accepted : 2020.07.30
  • Published : 2020.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the recent advancement of computer hardware and the contribution of open source libraries to facilitate access to artificial intelligence technology, the use of machine learning (ML) and deep learning (DL) technologies in various fields of exploration geophysics has increased. In addition, ML researchers have developed complex algorithms to improve the inference accuracy of various tasks such as image, video, voice, and natural language processing, and now they are expanding their interests into the field of automatic machine learning (AutoML). AutoML can be divided into three areas: feature engineering, architecture search, and hyperparameter search. Among them, this paper focuses on hyperparamter search with Bayesian optimization, and applies it to the problem of facies classification using seismic data and well logs. The effectiveness of the Bayesian optimization technique has been demonstrated using Vincent field data by comparing with the results of the random search technique.

최근 인공지능 기술의 발전과 함께 물리탐사의 다양한 분야에서도 인공지능의 핵심 기술인 머신러닝의 활용도가 증가하고 있다. 또한 머신러닝 및 딥러닝을 활용한 연구는 이미지, 비디오, 음성, 자연어 등 다양한 태스크의 추론 정확도를 높이기 위해 복잡한 알고리즘들이 개발되고 있고, 더 나아가 자료의 특성, 알고리즘 구조 및 하이퍼 파라미터의 최적화를 위한 자동 머신러닝(AutoML) 분야로 그 폭을 넓혀가고 있다. 본 연구에서는 AutoML 분야 중에서도 하이퍼 파라미터(hyperparameter) 자동 탐색을 위한 베이지안 최적화 기술에 중점을 두었으며, 본 기술을 물리탐사 분야에서도 암상 분류(facies classification) 문제에 적용했다. Vincent field의 현장 물리검층 및 탄성파 자료를 이용하여 암상 및 공극유체를 분류하는 지도학습 기반 모델에 적용하였고, 랜덤 탐색 기법의 결과와 비교하여 베이지안 최적화 기반 예측 프레임워크의 효율성을 검증하였다.

Keywords

References

  1. Araya-Polo, M., Jennings, J., Adler, A., and Dahlke, T., 2018, Deep-learning tomography, Lead Edge, 37(1), 58-66, doi:10.1190/tle37010058.1.
  2. Baldwin, J. L., Bateman, R. M., and Wheatley, C. L., 1990, Application of a neural network to the problem of mineral identification from well logs, The Log Analyst, 31(05), 279-293.
  3. Bergstra, J. S., Bardenet, R., Bengio, Y., and Kegl, B., 2011, Algorithms for hyper-parameter optimization, Adv. Neural. Inf. Process. Syst., 2546-2554.
  4. Choi, J., Yoon, D., Lee, S., and Byun, J., 2019, Petrofacies characterization using best combination of multiple elastic properties, J. Pet. Sci. Eng., 181, doi: 10.1016/j.petrol.2019.06.025.
  5. Choi, J., Kim, B., Kim, S., and Byun, J., 2017, Probabilistic facies analysis using 3D crossplot of stochastic forwardmodeling results, 87th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 3077-3081, doi: 10.1190/segam2017-17790996.1.
  6. Delfiner, P., Peyret, O., and Serra, O., 1987, Automatic determination of lithology from well logs, SPE Formation Evaluation, 2(03), 303-310, doi: 10.2118/13290-PA.
  7. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., and Greenspan, H., 2018, Synthetic data augmentation using GAN for improved liver lesion classification, 2018 IEEE 15th Int. Symp. Biomed. Imaging, 289-293, doi: 10.1109/ISBI.2018.8363576.
  8. Jones, D. R., 2001, A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization, 21(4), 345-383. https://doi.org/10.1023/A:1012771025575
  9. Kanter, J. M., and Veeramachaneni, K., 2015, Deep feature synthesis: Towards automating data science endeavors, 2015 IEEE Int. Conf. Data. Sci. Adv. Anal., 1-10, doi: 10.1109/DSAA.2015.7344858.
  10. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T. Y., 2017, LightGBM: A Highly Efficient Gradient Boosting Decision Tree, Adv. Neural Infor. Process. Syst., 30, 3149-3157.
  11. Klein, A., Falkner, S., Bartels, S., Hennig, P., and Hutter, F., 2017, Fast Bayesian optimization of machine learning hyperparameters on large datasets, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 528-536, doi: 10.1214/17-EJS1335SI.
  12. Lee, S., Choi, J., Yoon, D., and Byun, J., 2018, Automatic labeling strategy in semi-supervised seismic facies classification by integrating well logs and seismic data, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2998604.1.
  13. Li, H., Yang, W., and Yong, X., 2018, Deep learning for groundroll noise attenuation, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2981295.1.
  14. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A., 2018, Hyperband: A novel bandit-based approach to hyperparameter optimization, J. Mach. Learn. Res., 18, 1-52.
  15. Liu, H., Simonyan, K., and Yang, Y., 2018, Darts: Differentiable architecture search, arXiv preprint arXiv: 1806.09055.
  16. Mockus, J., 2012, Bayesian approach to global optimization: theory and applications, Springer Science & Business Media, 37.
  17. Nguyen, H. P., Liu, J., and Zio, E., 2020, A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Treestructured Parzen Estimator and applied to time-series data of NPP steam generators. Appl. Soft Comput., 89, 106116, doi:10.1016/j.asoc.2020.106116.
  18. Oh, S., Noh, K., Yoon, D., Seol, S. J., and Byun, J., 2018, Salt delineation from electromagnetic data using convolutional neural networks, IEEE Geosci. Remote Sens. Lett., 16(4), 519-523, doi: 10.1109/LGRS.2018.2877155.
  19. Park, J., Yoon, D., Seol, S. J., and Byun, J., 2019, Reconstruction of seismic field data with convolutional U-Net considering the optimal training input data, 89th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, doi: 10.1190/segam2019-3216017.1.
  20. Rashmi, K. V., and Gilad-Bachrach, R., 2015, DART: Dropouts meet Multiple Additive Regression Trees, Artificial Intelligence and Statistics, 489-497.
  21. Snoek, J., Larochelle, H., and Adams, R. P., 2012, Practical Bayesian optimization of machine learning algorithms, Adv. Neural Infor. Process. Syst., 2951-2959.
  22. Wolpert, D. H., and Macready, W. G., 1997, No free lunch theorems for optimization, IEEE Trans. Evol. Comput., 1(1), 67-82. https://doi.org/10.1109/4235.585893
  23. Wrona, T., Pan, I., Gawthorpe, R. L., and Fossen, H., 2018, Seismic facies analysis using machine learning, Geophysics, 83(5), O83-O95. https://doi.org/10.1190/geo2017-0595.1
  24. Yoon, D., Yeeh, Z., and Byun, J., 2020, Seismic Data Reconstruction Using Deep Bidirectional Long Short-Term Memory with Skip Connections, IEEE Geosci. Remote Sens. Lett., 1-5, doi: 10.1109/LGRS.2020.2993847.
  25. Yoon, D., Kim, S., Kim, J., Park, G., Park, H., Byun, J., Suh, J., Lee, C., Jang, I., Jo, S., and Choi, Y., 2018, Introduction of Resource Engineering with Machine Learning, CIR press, 377-396 (in Korean).
  26. Zoph, B., and Le, Q. V., 2016, Neural Architecture Search with Reinforcement Learning, arXiv preprint arXiv:1611.01578.