Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

Updating Smartphone's Exterior Orientation Parameters by Image-based Localization Method Using Geo-tagged Image Datasets and 3D Point Cloud as References

  • Wang, Ying Hsuan (Dept. of Civil and Environmental Engineering, Yonsei University) ;
  • Hong, Seunghwan (Stryx Inc.) ;
  • Bae, Junsu (Dept. of Civil and Environmental Engineering, Yonsei University) ;
  • Choi, Yoonjo (Dept. of Civil and Environmental Engineering, Yonsei University) ;
  • Sohn, Hong-Gyoo (Dept. of Civil and Environmental Engineering, Yonsei University)
  • Received : 2019.09.24
  • Accepted : 2019.10.11
  • Published : 2019.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the popularity of sensor-rich environments, smartphones have become one of the major platforms for obtaining and sharing information. Since it is difficult to utilize GNSS (Global Navigation Satellite System) inside the area with many buildings, the localization of smartphone in this case is considered as a challenging task. To resolve problem of localization using smartphone a four step image-based localization method and procedure is proposed. To improve the localization accuracy of smartphone datasets, MMS (Mobile Mapping System) and Google Street View were utilized. In our approach first, the searching for candidate matching image is performed by the query image of smartphone's using GNSS observation. Second, the SURF (Speed-Up Robust Features) image matching between the smartphone image and reference dataset is done and the wrong matching points are eliminated. Third, the geometric transformation is performed using the matching points with 2D affine transformation. Finally, the smartphone location and attitude estimation are done by PnP (Perspective-n-Point) algorithm. The location of smartphone GNSS observation is improved from the original 10.204m to a mean error of 3.575m. The attitude estimation is lower than 25 degrees from the 92.4% of the adjsuted images with an average of 5.1973 degrees.

Keywords

References

  1. Agarwal, P., Burgard, W., and Spinello, L. (2015), Metric localization using google street view, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3111-3118.
  2. Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. (2008), Speeded-up robust features (SURF), Computer vision and image understanding, Vol. 110, No. 3, pp. 346-359. https://doi.org/10.1016/j.cviu.2007.09.014
  3. Fischler, M.A., and Bolles, R.C. (1981), Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, Vol. 24, No. 6, pp. 381-395. https://doi.org/10.1145/358669.358692
  4. Fuentes-Pacheco, J., Ruiz-Ascencio, J., and Rendon-Mancha, J.M. (2015), Visual simultaneous localization and mapping: a survey, Artificial Intelligence Review, Vol. 43, No. 1, pp. 55-81. https://doi.org/10.1007/s10462-012-9365-8
  5. Google. (2019). Street View API - Developer Guide, https://developers.google.com/maps/documentation/streetview/intro (last date accessed : May 22, 2019)
  6. Jegou, H., Douze, M., and Schmid, C. (2008), Hamming embedding and weak geometry consistency for large scale image search-extended version, European Conference on Computer Vision, pp. 304-317.
  7. Kim, H.J., Dunn, E., and Frahm, J.M. (2017), Learned contextual feature reweighting for image geo-localization, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3251-3260.
  8. Leica Geosystems. (2019). Leica Pegasus: Two Mobile Sensor Platform $\mid$ Leica Geosystems, https://leica-geosystems.com/products/mobile-sensor-platforms/capture-platforms/leicapegasus_two (last date accessed : March 29, 2019)
  9. Li, Y., Snavely, N., Huttenlocher, D., and Fua, P. (2012), Worldwide pose estimation using 3d point clouds, European conference on computer vision, Berlin, Heidelberg, pp. 15-29.
  10. Liu, H., Mei, T., Luo, J., Li, H., and Li, S. (2012), Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing, Proceedings of the 20th ACM international conference on Multimedia, pp. 9-18.
  11. Mathworks (2019), Single Camera Calibrator App, https://kr.mathworks.com/help/vision/ug/single-cameracalibrator-app.html (last date accessed: August 10, 2019)
  12. Sadeghi, H., Valaee, S., and Shirani, S. (2016), 2DTriPnP: A robust two-dimensional method for fine visual localization using Google streetview database, IEEE Transactions on Vehicular Technology, Vol. 66, No. 6, pp. 4678-4690. https://doi.org/10.1109/TVT.2016.2615630
  13. Salarian, M., Manavella, A., and Ansari, R. (2015), Accurate localization in dense urban area using google street view images, 2015 SAI intelligent systems conference (IntelliSys), pp. 485-490.
  14. Samsung US. (2019). Samsung Galaxy S9 & S9+ Specifications - S9 Specs & Features $\mid$ Samsung US., https://www.samsung.com/us/smartphones/galaxy-s9/specs/ (last date accessed: June 10, 2019)
  15. Sattler, T., Havlena, M., Radenovic, F., Schindler, K., and Pollefeys, M. (2015), Hyperpoints and fine vocabularies for large-scale location recognition, Proceedings of the IEEE International Conference on Computer Vision, pp. 2102-2110.
  16. Sattler, T., Havlena, M., Schindler, K., and Pollefeys, M. (2016), Large-scale location recognition and the geometric burstiness problem, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1582-1590.
  17. Schonberger, J.L., Pollefeys, M., Geiger, A., and Sattler, T. (2018), Semantic visual localization, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6896-6906.
  18. Verstockt, S., Gerke, M., and Kerle, N. (2015), Geolocalization of Crowdsourced Images for 3-D Modeling of City Points of Interest, IEEE geoscience and remote sensing letters, Vol. 12, No. 8, pp. 1670-1674. https://doi.org/10.1109/LGRS.2015.2418816
  19. Wang, Y.H. (2019), Estimation of Object Location from Smartphone Images and Positioning Sensors Using MMS Datasets as Reference, Master's thesis, Yonsei University, Seoul, Korea, 32p.
  20. Wu, T., Liu, J., Li, Z., Liu, K., and Xu, B. (2018), Accurate smartphone indoor visual positioning based on a highprecision 3D photorealistic map, Sensors, Vol. 18, No. 6, pp. 1974. https://doi.org/10.3390/s18061974
  21. Zamir, A.R., and Shah, M. (2010), Accurate image localization based on google maps street view, European Conference on Computer Vision, Berlin, Heidelberg, pp. 255-268.
  22. Zandbergen, P.A., and Barbeau, S.J. (2011), Positional accuracy of assisted GPS data from high-sensitivity GPSenabled mobile phones, The Journal of Navigation, Vol. 64, No. 3, pp. 381-399. https://doi.org/10.1017/S0373463311000051