Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

The Preliminary Study on Driver's Brain Activation during Take Over Request of Conditional Autonomous Vehicle

조건부 자율주행에서 제어권 전환 시 운전자의 뇌 활성도에 관한 예비연구

  • Hong, Daye (Dept. of Computer Science, Hanyang University) ;
  • Kim, Somin (Dept. of Computer Science, Hanyang University) ;
  • Kim, Kwanguk (Dept. of Computer Science, Hanyang University)
  • 홍다예 (한양대학교 컴퓨터.소프트웨어 학과) ;
  • 김소민 (한양대학교 컴퓨터.소프트웨어 학과) ;
  • 김광욱 (한양대학교 컴퓨터.소프트웨어 학과)
  • Received : 2022.06.11
  • Accepted : 2022.07.06
  • Published : 2022.07.26
Download PDF
⟨ Previous Next ⟩

Abstract

Conditional autonomous vehicles should hand over control to the driver according on driving situations. However, if the driver is immersed in a non-driving task, the driver may not be able to make suitable decisions. Previous studies have confirmed that the cues enhance take-over performance with a directional information on driving. However, studies on the effect of take-over cues on the driver's brain activities are rigorously investigated yet. Therefore, this study we evaluates the driver's brain activity according to the take-over cue. A total of 25 participants evaluated the take-over performance using a driving simulator. Brain activity was evaluated by functional near-infrared spectroscopy, which measures brain activity through changes in oxidized hemoglobin concentration in the blood. It evaluates the activation of the prefrontal cortex (PFC) in the brain region. As a result, it was confirmed that the driver's PFC was activated in the presence of the cue so that the driver could stably control the vehicle. Since this study results confirmed that the effect of the cue on the driver's brain activity, and it is expected to contribute to the study of take-over performance on biomakers in conditional autonomous driving in future.

조건부 자율주행 차량은 주행 상황에 따라 운전자에게 제어권을 인계해야 한다. 그러나 운전자가 비운전 과제에 몰입해 있다면, 상황에 맞는 의사결정을 하지 못할 수 있다. 선행연구에서는 주행 방향정보를 제공하는 단서 자극 (Cue)이 제어권 전환 성능을 높인다는 것을 확인했으나, 이러한 방법론이 실제 운전자의 뇌 활성도에 미치는 영향에 관한 연구는 매우 제한적이다. 따라서 본 연구는 조건부 자율주행에서 제어권 전환 시, Cue에 따른 운전자의 뇌 활동을 평가한다. 총 25명의 피험자가 운전 시뮬레이터를 활용한 제어권 전환 성능에 평가를 수행하였다. 뇌 활동의 평가를 위해서는 혈액 내 산화 헤모글로빈 농도 변화를 통해 뇌 활성화를 측정하는 기능적 근적외 분광법이 사용되었으며 뇌 영역 중 전전두피질 (Prefrontal Cortex; PFC)의 활성화를 평가했다. 실험결과, Cue가 존재하는 경우 운전자의 PFC가 활성화되어 안정적인 제어권 전환 성능이 형성됨을 확인하였다. 본 연구는 Cue가 운전자의 뇌 활성도에 미치는 영향을 정량적으로 확인하였다는 점에서, 향후 조건부 자율주행에서 생체반응을 활용한 제어권 전환 성능 평가 연구에 기여할 수 있을 것으로 예상된다.

Keywords

Acknowledgement

이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. 2021R1A2C2013479). 이 논문은 2018년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No.2018R1A5A7059549). Correspondence to K. Kim (kenny@hanyang.ac.kr)

References

  1. Sae Mobilus. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, 2018.
  2. Tesla Motors, "Model S software version 7.0," 2016.
  3. Tesla Motors, "Autopilot," 2017.
  4. W. Morales-Alvarez, O. Sipele, R. Leberon, H. H. Tadjine, and C. Olaverri-Monreal, "Automated driving: A literature review of the take over request in conditional automation," Electronics, 9(12), pp. 2087, 2020. https://doi.org/10.3390/electronics9122087
  5. S. Li, P. Blythe, W. Guo, and A. Namdeo, "Investigation of older driver's takeover performance in highly automated vehicles in adverse weather conditions," IET Intelligent Transport Systems, 12(9), pp. 1157-1165, 2018. https://doi.org/10.1049/iet-its.2018.0104
  6. L. Kalb, L. Streit, and K. Bengler, "Multimodal priming of drivers for a cooperative take-over," In 2018 21st International Conference on Intelligent Transportation Systems (ITSC) pp. 1029-1034, 2018
  7. A. Eriksson, S. M. Petermeijer, M. Zimmermann, J. C. De Winter, K. J. Bengler, and N. A. Stanton, "Rolling out the red (and green) carpet: supporting driver decision making in automation-to-manual transitions," IEEE Transactions on Human-Machine Systems, 49(1), pp. 20-31, 2018. https://doi.org/10.1109/thms.2018.2883862
  8. S. M. Petermeijer, S. Cieler, and J. C. De Winter, "Comparing spatially static and dynamic vibrotactile take-over requests in the driver seat," Accident analysis & prevention, 99, pp. 218-227, 2017. https://doi.org/10.1016/j.aap.2016.12.001
  9. S. Petermeijer, P. Bazilinskyy, K. Bengler, and J. De Winter, "Take-over again: Investigating multimodal and directional TORs to get the driver back into the loop," Applied ergonomics, 62, pp. 204-215, 2017. https://doi.org/10.1016/j.apergo.2017.02.023
  10. M. Schwalk, N. Kalogerakis, and T. Maier, "Driver support by a vibrotactile seat matrix-Recognition, adequacy and workload of tactile patterns in take-over scenarios during automated driving," Procedia Manufacturing, 3, pp. 2466-2473, 2015. https://doi.org/10.1016/j.promfg.2015.07.507
  11. S. S. Borojeni, L. Chuang, W. Heuten, and S. Boll, "Assisting drivers with ambient take-over requests in highly automated driving," In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 237-244, 2016.
  12. C. Gold, D. Dambock, L. Lorenz, and K. Bengler, ""Take over!" How long does it take to get the driver back into the loop?," In Proceedings of the human factors and ergonomics society annual meeting, Vol. 57, No. 1, pp. 1938-1942, 2013. https://doi.org/10.1177/1541931213571433
  13. L. Lorenz, P. Kerschbaum, and J. Schumann, "Designing take over scenarios for automated driving: How does augmented reality support the driver to get back into the loop?," In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 58, No. 1, pp. 1681-1685, 2014.
  14. J. Heo, H. Lee, S. Yoon, and K. Kim, "Responses to Take-Over Request in Autonomous Vehicles: Effects of Environmental Conditions and Cues," IEEE Transaction on Intelligent Transportation System, 2022 (Accepted).
  15. S. Li, P. Blythe, W. Guo, and A. Namdeo, "Investigation of older driver's takeover performance in highly automated vehicles in adverse weather conditions," IET Intell. Transp. Syst., vol. 12, no. 9, pp. 1157-1165, 2018. https://doi.org/10.1049/iet-its.2018.0104
  16. R. V. D. Horst, "Time-to-collision as a cue for decision-making in braking," Vision in Vehicles-III, 1991.
  17. S. Balters, J. M. Baker, J. W. Geeseman, and A. L. Reiss, "A methodological review of fNIRS in driving research: Relevance to the future of autonomous vehicles," Frontiers in human neuroscience, 2021.
  18. D. Badre, and A. D. Wagner, "Left ventrolateral prefrontal cortex and the cognitive control of memory," Neuropsychologia, 45(13), pp. 2883-2901, 2007. https://doi.org/10.1016/j.neuropsychologia.2007.06.015
  19. S. Jahani, A. L. Fantana, D. Harper, J. M. Ellison, D. A. Boas, B. P. Forester, and M. A. Yucel, "fNIRS can robustly measure brain activity during memory encoding and retrieval in healthy subjects," Scientific reports, 7(1), pp. 1-14, 2017. https://doi.org/10.1038/s41598-016-0028-x
  20. A. Nissen, "Why we love blue hues on websites: a fNIRS investigation of color and its impact on the neural processing of ecommerce websites," In NeuroIS Retreat, pp. 1-15, 2020
  21. E. K. Miller, and J. D. Cohen, "An integrative theory of prefrontal cortex function," Annual review of neuroscience, 24(1), pp. 167-202, 2001. https://doi.org/10.1146/annurev.neuro.24.1.167
  22. S. Sibi, H. Ayaz, D. P. Kuhns, D. M. Sirkin, and W. Ju, "Monitoring driver cognitive load using functional near infrared spectroscopy in partially autonomous cars," In 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 419-425, 2016.
  23. T. Shimizu, S. Hirose, H. Obara, K. Yanagisawa, H. Tsunashima, Y. Marumo, ... and M. Taira, "Measurement of frontal cortex brain activity attributable to the driving workload and increased attention," SAE International Journal of Passenger Cars-Mechanical Systems, 2(1), pp. 736-744, 2009. https://doi.org/10.4271/2009-01-0545
  24. A. Unni, K. Ihme, H. Surm, L. Weber, A. Ludtke, D. Nicklas, ... and J. W. Rieger, "Brain activity measured with fNIRS for the prediction of cognitive workload," In 2015 6th IEEE International Conference on Cognitive Infocommunications, pp. 349-354, 2015.
  25. L. R. Derogatis, and R. Unger, "Symptom checklist-90-revised," The Corsini encyclopedia of psychology, pp. 1-2, 2010.
  26. D. Shinar, "Actual versus estimated night-time pedestrian visibility," Ergonomics, vol. 27, no. 8, pp. 863-871, 1984. https://doi.org/10.1080/00140138408963560
  27. S. Samuel, A. Borowsky, S. Zilberstein, and D. L. Fisher, "Minimum time to situation awareness in scenarios involving transfer of control from an automated driving suite," Transportation research record, pp. 115-120, 2016
  28. S. G. Hart, "Nasa-task load index (NASA-TLX); 20 years later," Proc. Hum. Factors Ergon. Soc. Annu. Meet., vol. 50, no. 9, pp. 904-908, 2006.
  29. H. J. Foy, P. Runham, and P. Chapman, "Prefrontal cortex activation and young driver behaviour: a fNIRS study," PLoS one, 11(5), pp. e0156512, 2016. https://doi.org/10.1371/journal.pone.0156512
  30. T. Nguyen, S. Ahn, H. Jang, S. C. Jun, and J. G. Kim, "Utilization of a combined EEG/NIRS system to predict driver drowsiness," Scientific reports, 7(1), pp. 1-10, 2017. https://doi.org/10.1038/s41598-016-0028-x
  31. T. H. Cho, Y. Nah, S. H. Park, and S. Han, "Prefrontal cortical activation in Internet Gaming Disorder Scale high scorers during actual real-time internet gaming: A preliminary study using fNIRS," Journal of Behavioral Addictions, 2022.
  32. N. Eshel, E. E. Nelson, R. J. Blair, D. S. Pine, and M. Ernst, "Neural substrates of choice selection in adults and adolescents: development of the ventrolateral prefrontal and anterior cingulate cortices," Neuropsychologia, 45(6), pp. 1270-1279.9, 2007. https://doi.org/10.1016/j.neuropsychologia.2006.10.004
  33. L. Steinberg, "A social neuroscience perspective on adolescent risk-taking," In Biosocial Theories of Crime, pp. 435-463, 2017.
  34. Y. Nakano, T. Kojima, H. Kawanaka, and K. Oguri. "Study of improving the cognitive ability of elderly drivers. In 16th International" IEEE Conference on Intelligent Transportation Systems pp. 547-551, 2013.
  35. C. G. Coutlee, and S. A. Huettel, "The functional neuroanatomy of decision making: prefrontal control of thought and action," Brain research, 1428, pp. 3-12, 2012. https://doi.org/10.1016/j.brainres.2011.05.053
  36. J. Kim, D. Park, P. Lee, J. Cho, S.-H. Yoon and S. Park. "Development of Management and Evaluation System for Realistic Virtual Reality Field Training Exercise Contents : A Case Study," Korea Computer Graphics Society, pp. 111-121, 2020.