Scanline Based Metric for Evaluating the Accuracy of Automatic Fracture Survey Methods

자동 균열 조사기법의 정확도 평가를 위한 조사선 기반의 지표 제안

  • Kim, Jineon (Department of Energy Systems Engineering, Seoul National University) ;
  • Song, Jae-Joon (Department of Energy Systems Engineering, Seoul National University)
  • 김진언 (서울대학교 에너지시스템공학부) ;
  • 송재준 (서울대학교 에너지시스템공학부)
  • Received : 2019.07.31
  • Accepted : 2019.08.22
  • Published : 2019.08.31


While various automatic rock fracture survey methods have been researched, the evaluation of the accuracy of these methods raises issues due to the absence of a metric which fully expresses the similarity between automatic and manual fracture maps. Therefore, this paper proposes a geometry similarity metric which is especially designed to determine the overall similarity of fracture maps and to evaluate the accuracy of rock fracture survey methods by a single number. The proposed metric, Scanline Intersection Similarity (SIS), is derived by conducting a large number of scanline surveys upon two fracture maps using Python code. By comparing the frequency of intersections over a large number of scanlines, SIS is able to express the overall similarity between two fracture maps. The proposed metric was compared with Intersection Over Union (IoU) which is a widely used evaluation metric in computer vision. Results showed that IoU is inappropriate for evaluating the geometry similarity of fracture maps because it is overly sensitive to minor geometry differences of thin elongated objects. The proposed metric, on the other hand, reflected macro-geometry differences rather than micro-geometry differences, showing good agreement with human perception. The metric was further applied to evaluate the accuracy of a deep learning-based automatic fracture surveying method which resulted as 0.674 (SIS). However, the proposed metric is currently limited to 2D fracture maps and requires comparison with rock joint parameters such as RQD.


Supported by : 한국연구재단


  1. Barton, N., Lien, R. and Lunde, J., 1974, Engineering classification of rock masses for the design of tunnel support, Rock mechanics, 6(4), 189-236.
  2. Byun, H., Yoon, D., Kang, I., Kim, J. and Song, J., 2018, Automated Rock Fracture Detection Using a Convolutional Neural Network, Proceedings of 2018 Fall Joint Conference of KSMER.KSRM.MIRECO, Gangwon-do, South Korea.
  3. Csurka, G., Larlus, D. and Perronnin, F., 2013, What Is a Good Evaluation Measure for Semantic Segmentation?, Procedings of the 24th British Machine Vision Conference 2013,
  4. Deb, D., Hariharan, S., Rao, U.M. and Ryu, C.H., 2008, Automatic Detection and Analysis of Discontinuity Geometry of Rock Mass from Digital Images, Computers and Geosciences, 34(2), 115-26,
  5. Everingham, M., Gool, L., Williams, C., Winn, J. and Zisserman, A., 2010, The Pascal Visual Object Classes (VOC) Challenge, Int. J. Comput. Vision, 88(2), 303-38,
  6. Ferrero, A. M., Forlani, G., Roncella, R. and Voyat, H. I., 2009, Advanced Geostructural Survey Methods Applied to Rock Mass Characterization, Rock Mechanics and Rock Engineering, 42(4), 631-65,
  7. Han, J.H. and Song, J.J., 2007, Study on Applicability of Stereophotogrammetry to Rock Joint Survey, Tunnel and Underground Space, 17(2), 139-151.
  8. Han, J.H., Song, J.J. and Jo, Y.D., 2009, Effect of Photographing Light Intensity on Rock Joint Survey in Mine Tunnels Using Stereophotogrammetry, Tunnel and Underground Space, 19(6), 517-25.
  9. Hudson, J. A. and Priest, S. D., 1983, Discontinuity Frequency in Rock Masses, International Journal of Rock Mechanics and Mining Sciences & Geomechanics Abstracts, 20(2), 73-89,
  10. Hyun, S., Lee, J.S., Jeon, S., Kim, Y., Kim, K.Y. and Yun. T.S., 2019, Pixel-level Crack Detection in X-ray Computed Tomography Image of Granite using Deep Learning, Tunnel and Underground Space, 29(3), 184-196.
  11. Lee, H.C., Kwon, K.M., Moon, C.E. and Jo, Y.H., 2018, Measurement Equipment Development of Stability Evaluation for Joint Slope using Unmaned Aerial Vehicle, Tunnel and Underground Space, 28(3), 193-208.
  12. Lemy, F., and Hadjigeorgiou, J., 2003, Discontinuity Trace Map Construction Using Photographs of Rock Exposures, International Journal of Rock Mechanics and Mining Sciences, 40, 903-17,
  13. Li, X., Chen, J. and Zhu, H., 2016, A New Method for Automated Discontinuity Trace Mapping on Rock Mass 3D Surface Model, Computers and Geosciences, 89, 118-31,
  14. Mohebbi, M., Yarahmadi Bafghi, A. R., Fatehi Marji, M. and Gholamnejad, J., 2016, Rock Mass Structural Data Analysis Using Image Processing Techniques (Case Study: Choghart Iron Ore Mine Northern Slopes), Journal of Mining & Environment, 8(May), 61-74,
  15. Powers, D. M. W, 2011, Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation, Journal of Machine Learning Technologies, 2(1), 37-63,
  16. Shi, R., King, N. N. and Li, S., 2014, Jaccard Index Compensation for Object Segmentation Evaluation, 2014 IEEE International Conference on Image Processing (ICIP), Paris, 4457-61,