Journal of the Korean Society for Library and Information Science (한국문헌정보학회지)

Korean Society For Library And Information Science (한국문헌정보학회)
권호 표지
DOI QR코드

DOI QR Code

Toward a Key-frame Extraction Framework for Video Storyboard Surrogates Based on Users' EEG Signals

이용자 기반의 비디오 키프레임 자동 추출을 위한 뇌파측정기술(EEG) 적용

  • Received : 2015.01.26
  • Accepted : 2015.02.16
  • Published : 2015.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

This study examined the feasibility of using EEG signals and ERP P3b for extracting video key-frames based on users' cognitive responses. Twenty participants were used to collect EEG signals. This research found that the average amplitude of right parietal lobe is higher than that of left parietal lobe when relevant images were shown to participants; there is a significant difference between the average amplitudes of both parietal lobes. On the other hand, the average amplitude of left parietal lobe in the case of non-relevant images is lower than that in the case of relevant images. Moreover, there is no significant difference between the average amplitudes of both parietal lobes in the case of non-relevant images. Additionally, the latency of MGFP1 and channel coherence can be also used as criteria to extract key-frames.

본 연구는 뇌파측정기술(EEG)과 사건관련유발전위 P3b를 활용하여 이용자의 인지적 반응을 측정한 후 비디오 키프레임을 자동으로 추출할 수 있는지의 가능성을 조사해 보았다. 20명의 피험자들을 대상으로 뇌파를 측정하고 분석한 결과, 적합 이미지 자극 시 좌측 두정엽 영역이 우측 두정엽 영역보다 더 활성화되며, 좌우측간 두정엽 영역의 활성화 정도가 유의한 차이를 보였다. 비적합 이미지 자극 시에는 좌측 두정엽 영역이 적합 이미지보다 덜 활성화되고, 두정엽 영역의 좌우간 활성화도 유의한 차이가 없는 것으로 나타났다. 이외에, 모든 채널의 평균값(MGFP1)의 잠재기, 채널 동시성 패턴 등에서도 두 자극간에 차이를 보여 뇌파측정기술에 기반한 키프레임 자동 추출이 가능한 것으로 확인되었다.

Keywords

References

  1. 고유빈 외. 2012. 뇌 전도를 이용한 집중력 정량화 연구. HCI 2012 학술대회, 23-25.(Koh, Y. et al. 2012. "Study of quantifying concentration index using electroencephalography (EEG)." HCI Society of Korea, 23-25.)
  2. 권준수. 2000. 인지기능연구에서의 사건관련전위의 이용. 인지과학작업, 1(1): 79-98.(Kwon, J. 2000. "The Use of event-related potentials in the study of cognitive functions." Journal of Cognitive Science, 1(1): 79-98.)
  3. 권형규. 2011. 컴퓨터 덧셈학습의 인지적 개인차에 대한 암묵적 연합검사를 적용한 뇌파 분석. 정보교육학회논문지, 15(4): 635-644.(Kwon, H. 2011. "Brainwave activities of the cognitive individual differences in computerized arithmetic addition by implicit association test." Journal of the Korean Association of Information Education, 15(4): 635-644.)
  4. 김종화. 2013. 사회감성 인식을 위한 EEG 코히런스를 사용한 뇌 기능 연결성 분석. 박사학위논문, 상명대학교 감성공학과.(Kim, J. 2013. Recognition of social emotion using EEG coherence analysis. Ph.D Dissertation, Department of Emotion Engineering, Sangmyung University.)
  5. 김용호. 2006. 뇌파측정기술(EEG)을 이용한 TV 영상 감성반응의 실험 연구. 한국방송학보, 20(1): 7-49.(Kim, Y. 2006. "A study on measuring the lateral specification of brain activity using brainwave measurement(EEG)." Korean Journal of Broadcasting and Telecommunication Studies, 20(1): 7-49.)
  6. 김현희. 2008. 영상 초록 구현을 위한 키프레임 추출 알고리즘의 설계와 성능 평가. 정보관리학회지, 25(4): 131-148.(Kim, H. 2008. "Design and evaluation of the key-frame extraction algorithm for constructing the Virtual Storyboard Surrogates." Journal of the Korean Society for Information Management, 25(4): 131-148.) https://doi.org/10.3743/KOSIM.2008.25.4.131
  7. 박두흠, 권준수, 정도언. 2001. 기면병에서 전산화 뇌파 지도화 기법으로 측정한 뇌파 동시성 변화. 수면.정신생리, 8(2): 121-128.(Park, D., Kwon, J. and Jeong, D. 2001. "Changes of EEG coherence in narcolepsy measured with computerized EEG mapping technique." Sleep medicine and psychophysiology, 8(2): 121-128.)
  8. 이성은. 2010. 간접화행의 인지적 이해모델. 독일어 문화권 연구, 19: 133-159.(Lee, S. E. 2010. "Das kognitive Verstehensmodell indirekter Sprechakte." Zeitschrift fur Deutschsprachige Kultur & Literaturen, 19: 133-159.)
  9. 이지영. 2006. 뇌 연구방법론을 통해 살펴본 음악 처리과정 연구: 음악과 언어, 음악과 정서를 중심으로. 낭만음악, 18(3): 69-146.(Lee, J. 2006. "Neurophysiology and brain-imaging study of music -music & language, music & emotion-." Nang Man Music, 18(3): 69-146.)
  10. 이충연, 장병탁. 2014. 뇌의 기억 인출에 대한 유효 EEG 연결성 분석. 컴퓨팅의 실제 및 레터, 20(4): 257-261.(Lee, C. and Zhang, B. 2014. "Analysis on effective EEG connectivity of memory retrieval in the brain." Journal of KIISE: Computing Practices and Letters, 20(4): 257-261.)
  11. 이충연 외. 2011. EEG 기반 뇌기능 분석을 이용한 영화 장면-대사 기억 게임에서의 인지 학습 특성. 한국정보과학회, 38(1): 210-213.(Lee, C. et al. 2011. "Properties of human cognitive learning in a movie scene-dialogue memory game using EEG-based brain function analysis." Journal of Computing Science and Engineering, 38(1): 210-213.)
  12. 임용수, 이승환, 홍석인. 2010. 정신분열병 환자에서 생물학적 지표로서 N100, P300과 정량화뇌파의 적용. 대한정신약물학회지, 21(2): 78-86.(Lim, Y., Lee, S. and Hong, S. 2010. "Application of N100, P300 and QEEG as a biological marker in patients with schizophrenia." The Korean Journal of Psychopharmacology, 21(2): 78-86.)
  13. 장윤석, 한재웅. 2014. 시각자극 과제에 의한 집중 시의 뇌파분석. 한국전자통신학회 논문지, 9(5): 589-594.(Jang, Y. and Han, J. 2014. "Analysis of EEG generated from concentration by visual stimulus task." The Journal of the Korea institute of electronic communication sciences, 9(5): 589-594.)
  14. Behneman, A. et al. 2009. Enhancing text-based analysis using neurophysiological measures. In D. D. Schmorrow et al. (Eds.), Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience. Vol. 5638, 449-458. Berlin, Heidelberg: Springer.
  15. Browne, P. and Smeaton, A. F. 2005. "Video retrieval using dialogue, keyframe similarity and video objects." ICIP 2005 - International Conference on Image Processing, Genova, Italy, 11-14.
  16. Davidson, R. J. and Irwin, W. 1999. "The functional neuroanatomy of emotion and affective style." Trends in Cognitive Science, 3: 11-21. https://doi.org/10.1016/S1364-6613(98)01265-0
  17. DeFrance, J. F. 1997. "Age-related changes in cognitive ERPs of attenuation." Brain Topography, 9(4): 283-293. https://doi.org/10.1007/BF01464483
  18. Galletti, C. et al. 2003. "Role of the medial parieto-occipital cortex in the control of reaching and grasping movements." Exp Brain Res, 153: 158-170. https://doi.org/10.1007/s00221-003-1589-z
  19. Gwizdka, J. et al. 2013. "Applications of neuroimaging in information science: Challenges and opportunities." Proceedings of the American Society for Information Science and Technology, 50(1): 1-4.
  20. Hillyard, S. A. and Woods, D. L. 1979. "Electrophysiological analysis of human brain function." In M. S. Gazzaniga (Ed.), Handbook of behavioral neurobiology: Vol 2. Neuropsychology (pp. 345-378). New York: Plenum Press.
  21. Iyer, H. and Lewis, C.D. 2007. "Prioritization strategies for video storyboard keyframes." Journal of the American Society for Information Science and Technology, 58(5): 629-644. https://doi.org/10.1002/asi.20554
  22. Jung, H. et al. 2012. "Reduced source activity of event-related potentials for affective facial pictures in schizophrenia patients." Schizophrenia Research, 136: 150-159. https://doi.org/10.1016/j.schres.2011.10.023
  23. Kim, H. and Kim, Y. 2010. "Toward a conceptual framework of key-frame extraction and storyboard display for video summarization." Journal of the American Society for Information Science and Technology, 61(5): 927-939. https://doi.org/10.1002/asi.21317
  24. Koelstra, S., Muehl, C. and Patras, I. 2009. "EEG analysis for implicit tagging of video data." Proceeding of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009, 27-32, Los Alamitos, 2009. IEEE Computer Society Press.
  25. Moshfeghi, Y., Pinto, L. R., Pollick, F. R. and Jose, J. M. 2013. "Understanding relevance: An fMRI study." In P. Serdyukov et al., eds. Advances in Information Retrieval. Springer Berlin Heidelberg.
  26. Naghavi, H. and Nyberg, L. 2005. "Common fronto-parietal activity in attention, memory, and consciousness: Shared demands on integration?" Consciousness and Cognition, 14(2): 390-425. https://doi.org/10.1016/j.concog.2004.10.003
  27. Song, Y., Marchionini, G. and Oh, C. 2010. "What are the most eye-catching and ear-catching features in the video?" Implications for video summarization. WWW 2010, April 26-30, 2010, Raleigh, North Carolina.
  28. Thatcher, R. W., Krause, P. J. and Hrybyk, M. 1986. "Cortico-cortical associations and EEG coherence: a two-compartmental model." Electroencephalogr Clin Neurophysiol, 64: 123-143. https://doi.org/10.1016/0013-4694(86)90107-0
  29. Tulving, E. et al. 1994. "Hemispheric encoding / retrieval asymmetry in episodic memory: position emission tomography findings." Proceedings of the National Academy of Sciences of the United States of America, 91(6): 2016-2020. https://doi.org/10.1073/pnas.91.6.2016
  30. Wang, S., Zhu, Y., Wu, G. and Ji, Q. 2013. "Hybrid video emotional tagging using users' EEG and video content." Multimedia Tools and Applications, 72(2): 1257-1283. https://doi.org/10.1007/s11042-013-1450-8
  31. Yang, M. 2005. An exploration of users' video relevance criteria. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.
  32. Yang, M. and Marchionini, G. 2004. Exploring users' video relevance criteria: A pilot study. Proceedings of the ASIST Annual Meeting, 41: 229-238. Medford, NJ: Information Today.