The Journal of The Korea Institute of Intelligent Transport Systems (한국ITS학회 논문지)

The Korea Institute of Intelligent Transport Systems (한국ITS학회)
권호 표지
DOI QR코드

DOI QR Code

Predicting Accident Vulnerable Situation and Extracting Scenarios of Automated Vehicleusing Vision Transformer Method Based on Vision Data

Vision Transformer를 활용한 비전 데이터 기반 자율주행자동차 사고 취약상황 예측 및 시나리오 도출

  • 이우섭 (서울특별시 도시교통실 교통정책과) ;
  • 강민희 (홍익대학교 일반대학원 산업융합협동과정 스마트도시전공) ;
  • 윤영 (홍익대학교 컴퓨터공학과) ;
  • 황기연 (홍익대학교 도시공학과)
  • Received : 2022.08.16
  • Accepted : 2022.10.17
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, various studies have been conducted to improve automated vehicle (AV) safety for AVs commercialization. In particular, the scenario method is directly related to essential safety assessments. However, the existing scenario do not have objectivity and explanability due to lack of data and experts' interventions. Therefore, this paper presents the AVs safety assessment extended scenario using real traffic accident data and vision transformer (ViT), which is explainable artificial intelligence (XAI). The optimal ViT showed 94% accuracy, and the scenario was presented with Attention Map. This work provides a new framework for an AVs safety assessment method to alleviate the lack of existing scenarios.

자율주행자동차 상용화를 위해 자율주행자동차 안전성 제고를 위한 다양한 연구가 수행되고 있으며, 그 중 시나리오 연구가 안전성 평가에 직접적으로 연관되어 필수적으로 고려되고 있다. 그러나 기존 시나리오 제시의 경우 데이터 부재 및 전문가 개입으로 인해 객관성 및 설명력이 보완될 필요가 있다는 의견이 제시되고 있다. 이에 본 연구에서는 실제 사고 데이터 및 설명력 있는 인공지능 방법론인 ViT 모델을 활용하여 확장된 자율주행자동차 안전성 평가 시나리오를 제시한다. 활용 데이터에 최적화시킨 ViT 모델 학습 결과, 94% 정확도가 확인되었으며 Attention Map을 추가적으로 활용하여 설명력 있는 시나리오를 제시하였다. 본 연구를 통해 기존 시나리오 접근법의 한계를 보완하고 인공지능을 활용하여 새로운 안전성 평가 시나리오 수립 프레임워크를 제시할 수 있을 것으로 기대된다.

Keywords

Acknowledgement

본 연구는 이우섭 석사 학위 논문 「Vision Transformer를 활용한 실주행 데이터 기반 자율주행자동차 사고 취약상황 예측 및 시나리오 도출 연구」를 수정·보완하여 작성되었으며, 국토교통부 자율주행기술개발혁신사업 '주행 및 충돌상황 대응 안전성 평가기술개발(22AMDP-C161754-02)' 과제 지원으로 수행되었습니다.

References

  1. Abbas, H., O'Kelly, M., Rodionova, A. and Mangharam, R.(2017), "Safe at any speed: A simulation-based test harness for autonomous vehicles", In International Workshop on Design, Modeling, and Evaluation of Cyber Physical Systems, Springer, Cham, October, pp.94-106.
  2. Alambeigi, H., McDonald, A. D. and Tankasala, S. R.(2020), Crash themes in automated vehicles: A topic modeling analysis of the California Department of Motor Vehicles automated vehicle crash database, arXiv preprint arXiv:2001.11087.
  3. Arvin, R., Khattak, A. J. and Qi, H.(2021), "Safety critical event prediction through unified analysis of driver and vehicle volatilities: Application of deep learning methods", Accident Analysis & Prevention, vol. 151, 105949. https://doi.org/10.1016/j.aap.2020.105949
  4. Audi, A. G. and Volkswagen, A. G.(2019), https://www.pegasusprojekt.de/files/tmpl/PegasusAbschlussveranstaltung/PEGASUS-Gesamtmethode.pdf, 2022.06.13.
  5. Bartels, A., Eberle, U. and Knapp, A.(2015), "Deliverable D2.1. System Classification and Glossary", Adaptive Consortium, Wolfsburg, Germany, Feb. 6, p.63.
  6. Chae, H., Kim, S., Yi, K., Lee, M. and Min, K.(2019), "Development and Implementation of Safety Evaluation Scenarios for Automated Driving Vehicles on Test Bed", In 26th International Technical Conference on the Enhanced Safety of Vehicles (ESV): Technology: Enabling a Safer Tomorrow National Highway Traffic Safety Administration (No. 19-0061), pp.1-13.
  7. Das, A. and Rad, P.(2020), Opportunities and challenges in explainable artificial intelligence (xai): A survey, arXiv preprint arXiv:2006.11371.
  8. De Gelder, E., Den Camp, O. O. and De Boer, N.(2020), https://cetran.sg/wp-content/uploads/2020/01/REP200121_Scenario_Categories_v1.7.pdf, 2022.06.13.
  9. Demetriou, A., Allsvag, H., Rahrovani, S. and Chehreghani, M. H.(2020), "Generation of driving scenario trajectories with generative adversarial networks", In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), IEEE, September, pp.1-6.
  10. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkorit, J. and Houlsby, N.(2020), An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929.
  11. ENABLE-S3 Consortium(2019), https://www.tugraz.at/fileadmin/user_upload/Institute/IHF/Projekte/ENABLE-S3_SummaryofResults_May2019.pdf, 2022.06.13.
  12. Erdogan, A., Ugranli, B., Adali, E., Sentas, A., Mungan, E., Kaplan, E. and Leitner, A.(2019), "Real-world maneuver extraction for autonomous vehicle validation: A comparative study", In 2019 IEEE Intelligent Vehicles Symposium (IV), IEEE, June, pp.267-272.
  13. Fremont, D. J., Kim, E., Pant, Y. V., Seshia, S. A., Acharya, A., Bruso, X., Paul, W., Lemke, S., Lu Q. and Mehta, S.(2020), "Formal scenario-based testing of autonomous vehicles: From simulation to the real world", In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), IEEE, September, pp.1-8.
  14. Goodall, N. J.(2014), "Ethical decision making during automated vehicle crashes", Transportation Research Record, vol. 2424, no. 1, pp.58-65. https://doi.org/10.3141/2424-07
  15. Hiller, J., Svanberg, E., Koskinen, S., Bellotti, F. and Osman, N.(2019), "The L3Pilot Common Data Format-Enabling efficient automated driving data analysis", In Proceedings of the 26th International Technical Conference on the Enhanced Safety of Vehicles, Eindhoven, The Netherlands, June, pp.10-13.
  16. Jenkins, I. R., Gee, L. O., Knauss, A., Yin, H. and Schroeder, J.(2018), "Accident scenario generation with recurrent neural networks", In 2018 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE, November, pp.3340-3345.
  17. Kang, M., Im, I. J., Song, J. and Hwang, K.(2022a), "Is Only the Dedicated Lane for Automated Vehicles Essential in the Future? The Dedicated Lanes Optimal Operating System Evaluation", Sustainability, vol. 14, no. 18, p.11490. https://doi.org/10.3390/su141811490
  18. Kang, M., Song, J. and Hwang, K.(2020), "For Preventative Automated Driving System (PADS): Traffic Accident Context Analysis Based on Deep Neural Networks", Electronics, vol. 9, no. 11, p.1829. https://doi.org/10.3390/electronics9111829
  19. Kang, M., Song, J. and Hwang, K.(2022b), "The Extraction of Automated Vehicles Traffic Accident Factors and Scenarios Using Real-World Data", In Congress on Intelligent Systems, Springer, Singapore, pp.1-15.
  20. Karim, M. M., Li, Y., Qin, R. and Yin, Z.(2021), A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention, arXiv preprint arXiv:2106.10319. 2106
  21. Kim, D. Y., Lee, S. Y., Lee, H. K., Cho, I. S., Shin, J. K. and Park, K. H.(2019), "Development of Quantitative Methods for Evaluating Failure Safety of Level 3 Autonomous Vehicles", The Journal of The Korea Institute of Intelligent Transport Systems, vol. 18, no. 1, pp.91-102.
  22. Kim, K., Kim, B., Lee, K., Ko, B. and Yi, K.(2017), "Design of integrated risk management-based dynamic driving control of automated vehicles", IEEE Intelligent Transportation Systems Magazine, vol. 9, no. 1, pp.57-73. https://doi.org/10.1109/MITS.2016.2580714
  23. Ko, W., Park, S., Yun, J., Park, S. and Yun, I.(2022a), "Development of a framework for generating driving safety assessment scenarios for automated vehicles", Sensors, vol. 22, no. 16, p.6031. https://doi.org/10.3390/s22166031
  24. Ko, W. R., Yun, I. S., Park, S. M., Jeong, H. M. and Park, S. H.(2022b), "Derivation of Assessment Scenario Elements for Automated Vehicles in the Expressway Mainline Section", The Journal of The Korea Institute of Intelligent Transport Systems, vol. 21, no. 1, pp.221-239. https://doi.org/10.12815/kits.2022.21.1.221
  25. Koopman, P., Ferrell, U., Fratrik, F. and Wagner, M.(2019), "A safety standard approach for fully autonomous vehicles", In International Conference on Computer Safety, Reliability, and Security, Springer, Cham, pp.326-332.
  26. Lee, J. M., Jung, U. I. and Song, B. S.(2020), "Critical Scenario Generation for Collision Avoidance of Automated Vehicles Based on Traffic Accident Analysis and Machine Learning", Transactions of the Korean Society of Automotive Engineers, vol. 28, no. 11, pp.817-826. https://doi.org/10.7467/KSAE.2020.28.11.817
  27. Lee, W., Kang, M. H., Song, J. and Hwang, K.(2021), "The Design of Preventive Automated Driving Systems Based on Convolutional Neural Network", Electronics, vol. 10, no. 14, p.1737. https://doi.org/10.3390/electronics10141737
  28. Lim, H. H., Chae, H. S., Lee, M. S. and Lee, K. S.(2017), "Development and Validation of Safety Performance Evaluation Scenarios of Autonomous Vehicle based on Driving Data", Journal of Auto-Vehicle Safety Association, vol. 9, no. 4, pp.7-13. https://doi.org/10.22680/KASA.2017.9.4.007
  29. Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J. and Han, J.(2019), On the variance of the adaptive learning rate and beyond, arXiv preprint arXiv:1908.03265.
  30. Luettel, T., Himmelsbach, M. and Wuensche, H. J.(2012), "Autonomous ground vehicles-Concepts and a path to the future", Proceedings of the IEEE, pp.1831-1839.
  31. Masmoudi, M., Ghazzai, H., Frikha, M. and Massoud, Y.(2019), "Object detection learning techniques for autonomous vehicle applications", In 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES), IEEE, pp.1-5.
  32. Norden, J., O'Kelly, M. and Sinha, A.(2019), Efficient black-box assessment of autonomous vehicle safety, arXiv preprint arXiv:1912.03618.
  33. Osman, O. A., Hajij, M., Bakhit, P. R. and Ishak, S.(2019), "Prediction of near-crashes from observed vehicle kinematics using machine learning", Transportation Research Record, vol. 2673, no. 12, pp.463-473.
  34. Park, S. M., So, J. H., Ko, H. G., Jeong, H. R. and Yun, I. S.(2019a), "Development of Safety Evaluation Scenarios for Autonomous Vehicle Tests Using 5-Layer Format (Case of the Community Road)", The Journal of the Korea Institute of Intelligent Transport Systems, vol. 18, no. 2, pp.114-128. https://doi.org/10.12815/kits.2019.18.2.114
  35. Park, S. H., Jeong, H. R., Kwon, C. W., Kim, J. H. and Yun, I. S.(2019b), "Analysis of Take-over Time and Stabilization of Autonomous Vehicle Using a Driving Simulator", The Journal of the Korea Institute of Intelligent Transport Systems, vol. 18, no. 4, pp.31-43.
  36. Remmen, F., Cara, I., De Gelder, E. and Willemsen, D.(2018), "Cut-in scenario prediction for automated vehicles", In 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), IEEE, pp.1-7.
  37. Riedmaier, S., Ponn, T., Ludwig, D., Schick, B. and Diermeyer, F.(2020), "Survey on scenario-based safety assessment of automated vehicles", IEEE Access, vol. 8, pp.87456-87477. https://doi.org/10.1109/ACCESS.2020.2993730
  38. Strickland, M., Fainekos, G. and Amor, H. B.(2018), "Deep predictive models for collision risk assessment in autonomous driving", In 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp.4685-4692.
  39. Sui, B., Lubbe, N. and Bargman, J.(2019), "A clustering approach to developing car-to-two-wheeler test scenarios for the assessment of Automated Emergency Braking in China using in-depth Chinese crash data", Accident Analysis & Prevention, vol. 132, 105242. https://doi.org/10.1016/j.aap.2019.07.018
  40. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z.(2016), "Rethinking the inception architecture for computer vision", In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp.2818-2826.
  41. Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F. and Maurer, M.(2015), "Defining and substantiating the terms scene, situation, and scenario for automated driving", In 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, pp.982-988.
  42. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. and Polosukhin, I.(2017), "Attention is all you need", Advances in Neural Information Processing Systems, vol. 30.
  43. Virdi, J.(2018), Using deep learning to predict obstacle trajectories for collision avoidance in autonomous vehicles, University of California, San Diego.
  44. Yu, R., Ai, H. and Gao, Z.(2020), "Identifying High Risk Driving Scenarios Utilizing a CNN-LSTM Analysis Approach", In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), IEEE, pp.1-6.
  45. Zhao, X., Robu, V., Flynn, D., Salako, K. and Strigini, L.(2019), "Assessing the safety and reliability of autonomous vehicles from road testing", In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE), IEEE, pp.13-23.