Hyperparameter Search for Facies Classification with Bayesian Optimization

베이지안 최적화를 이용한 암상 분류 모델의 하이퍼 파라미터 탐색

  • Choi, Yonguk (Dept. Energy & Resources Engineering, Chonnam National University) ;
  • Yoon, Daeung (Dept. Energy & Resources Engineering, Chonnam National University) ;
  • Choi, Junhwan (Dept. of Earth Resources and Environmental Engineering, Hanyang University) ;
  • Byun, Joongmoo (Dept. of Earth Resources and Environmental Engineering, Hanyang University)
  • 최용욱 (전남대학교 에너지자원공학과) ;
  • 윤대웅 (전남대학교 에너지자원공학과) ;
  • 최준환 (한양대학교 자원환경공학과) ;
  • 변중무 (한양대학교 자원환경공학과)
  • Received : 2020.06.16
  • Accepted : 2020.07.30
  • Published : 2020.08.31


With the recent advancement of computer hardware and the contribution of open source libraries to facilitate access to artificial intelligence technology, the use of machine learning (ML) and deep learning (DL) technologies in various fields of exploration geophysics has increased. In addition, ML researchers have developed complex algorithms to improve the inference accuracy of various tasks such as image, video, voice, and natural language processing, and now they are expanding their interests into the field of automatic machine learning (AutoML). AutoML can be divided into three areas: feature engineering, architecture search, and hyperparameter search. Among them, this paper focuses on hyperparamter search with Bayesian optimization, and applies it to the problem of facies classification using seismic data and well logs. The effectiveness of the Bayesian optimization technique has been demonstrated using Vincent field data by comparing with the results of the random search technique.


  1. Araya-Polo, M., Jennings, J., Adler, A., and Dahlke, T., 2018, Deep-learning tomography, Lead Edge, 37(1), 58-66, doi:10.1190/tle37010058.1.
  2. Baldwin, J. L., Bateman, R. M., and Wheatley, C. L., 1990, Application of a neural network to the problem of mineral identification from well logs, The Log Analyst, 31(05), 279-293.
  3. Bergstra, J. S., Bardenet, R., Bengio, Y., and Kegl, B., 2011, Algorithms for hyper-parameter optimization, Adv. Neural. Inf. Process. Syst., 2546-2554.
  4. Choi, J., Yoon, D., Lee, S., and Byun, J., 2019, Petrofacies characterization using best combination of multiple elastic properties, J. Pet. Sci. Eng., 181, doi: 10.1016/j.petrol.2019.06.025.
  5. Choi, J., Kim, B., Kim, S., and Byun, J., 2017, Probabilistic facies analysis using 3D crossplot of stochastic forwardmodeling results, 87th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 3077-3081, doi: 10.1190/segam2017-17790996.1.
  6. Delfiner, P., Peyret, O., and Serra, O., 1987, Automatic determination of lithology from well logs, SPE Formation Evaluation, 2(03), 303-310, doi: 10.2118/13290-PA.
  7. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., and Greenspan, H., 2018, Synthetic data augmentation using GAN for improved liver lesion classification, 2018 IEEE 15th Int. Symp. Biomed. Imaging, 289-293, doi: 10.1109/ISBI.2018.8363576.
  8. Jones, D. R., 2001, A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization, 21(4), 345-383.
  9. Kanter, J. M., and Veeramachaneni, K., 2015, Deep feature synthesis: Towards automating data science endeavors, 2015 IEEE Int. Conf. Data. Sci. Adv. Anal., 1-10, doi: 10.1109/DSAA.2015.7344858.
  10. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T. Y., 2017, LightGBM: A Highly Efficient Gradient Boosting Decision Tree, Adv. Neural Infor. Process. Syst., 30, 3149-3157.
  11. Klein, A., Falkner, S., Bartels, S., Hennig, P., and Hutter, F., 2017, Fast Bayesian optimization of machine learning hyperparameters on large datasets, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), 528-536, doi: 10.1214/17-EJS1335SI.
  12. Lee, S., Choi, J., Yoon, D., and Byun, J., 2018, Automatic labeling strategy in semi-supervised seismic facies classification by integrating well logs and seismic data, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2998604.1.
  13. Li, H., Yang, W., and Yong, X., 2018, Deep learning for groundroll noise attenuation, 88th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, 14-19, doi: 10.1190/segam2018-2981295.1.
  14. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A., 2018, Hyperband: A novel bandit-based approach to hyperparameter optimization, J. Mach. Learn. Res., 18, 1-52.
  15. Liu, H., Simonyan, K., and Yang, Y., 2018, Darts: Differentiable architecture search, arXiv preprint arXiv: 1806.09055.
  16. Mockus, J., 2012, Bayesian approach to global optimization: theory and applications, Springer Science & Business Media, 37.
  17. Nguyen, H. P., Liu, J., and Zio, E., 2020, A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Treestructured Parzen Estimator and applied to time-series data of NPP steam generators. Appl. Soft Comput., 89, 106116, doi:10.1016/j.asoc.2020.106116.
  18. Oh, S., Noh, K., Yoon, D., Seol, S. J., and Byun, J., 2018, Salt delineation from electromagnetic data using convolutional neural networks, IEEE Geosci. Remote Sens. Lett., 16(4), 519-523, doi: 10.1109/LGRS.2018.2877155.
  19. Park, J., Yoon, D., Seol, S. J., and Byun, J., 2019, Reconstruction of seismic field data with convolutional U-Net considering the optimal training input data, 89th Ann. Internat. Mtg. Soc. Expl. Geophys., Expanded Abstracts, doi: 10.1190/segam2019-3216017.1.
  20. Rashmi, K. V., and Gilad-Bachrach, R., 2015, DART: Dropouts meet Multiple Additive Regression Trees, Artificial Intelligence and Statistics, 489-497.
  21. Snoek, J., Larochelle, H., and Adams, R. P., 2012, Practical Bayesian optimization of machine learning algorithms, Adv. Neural Infor. Process. Syst., 2951-2959.
  22. Wolpert, D. H., and Macready, W. G., 1997, No free lunch theorems for optimization, IEEE Trans. Evol. Comput., 1(1), 67-82.
  23. Wrona, T., Pan, I., Gawthorpe, R. L., and Fossen, H., 2018, Seismic facies analysis using machine learning, Geophysics, 83(5), O83-O95.
  24. Yoon, D., Yeeh, Z., and Byun, J., 2020, Seismic Data Reconstruction Using Deep Bidirectional Long Short-Term Memory with Skip Connections, IEEE Geosci. Remote Sens. Lett., 1-5, doi: 10.1109/LGRS.2020.2993847.
  25. Yoon, D., Kim, S., Kim, J., Park, G., Park, H., Byun, J., Suh, J., Lee, C., Jang, I., Jo, S., and Choi, Y., 2018, Introduction of Resource Engineering with Machine Learning, CIR press, 377-396 (in Korean).
  26. Zoph, B., and Le, Q. V., 2016, Neural Architecture Search with Reinforcement Learning, arXiv preprint arXiv:1611.01578.