Journal of Internet of Things and Convergence (사물인터넷융복합논문지)

The Korea Internet of Things Society (한국사물인터넷학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Brain-machine Interface for MindPong using Internet of Things

마인드 퐁 제어를 위한 사물인터넷을 이용하는 뇌-기계 인터페이스 개발

  • Hoon-Hee Kim (Department of Computer Engineering and Artificial Intelligence, Pukyong National University)
  • 김훈희 (국립부경대학교 컴퓨터.인공지능공학부)
  • Received : 2023.10.07
  • Accepted : 2023.11.29
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Brain-Machine Interfaces(BMI) are interfaces that control machines by decoding brainwaves, which are electrical signals generated from neural activities. Although BMIs can be applied in various fields, their widespread usage is hindered by the low portability of the hardware required for brainwave measurement and decoding. To address this issue, previous research proposed a brain-machine interface system based on the Internet of Things (IoT) using cloud computing. In this study, we developed and tested an application that uses brainwaves to control the Pong game, demonstrating the real-time usability of the system. The results showed that users of the proposed BMI achieved scores comparable to optimal control artificial intelligence in real-time Pong game matches. Thus, this research suggests that IoT-based brain-machine interfaces can be utilized in a variety of real-time applications in everyday life.

뇌-기계 인터페이스(BMI)는 신경활동을 통해 발생하는 전기 신호인 뇌파를 해석하여 기계를 제어하는 인터페이스이다. BMI는 다양한 분야에 적용될 수 있으나 뇌파 측정 및 해석을 위한 하드웨어의 휴대성이 낮아 대중적으로 사용되기에 어렵다는 단점이 있다. 이런 문제점을 해결하기 위해 이전 연구에서는 클라우드 컴퓨팅을 이용한 사물인터넷 기반 뇌-기계 인터페이스 시스템을 제안하였다. 본 연구에서는 위 시스템의 실시간 사용성을 증명하기 위하여 뇌파로 퐁(Pong) 게임을 조종하는 애플리케이션을 개발하여 테스트하였다. 그 결과 제안된 BMI 사용자가 최적 제어 인공지능과의 실시간 퐁 게임 대결에서 대등한 스코어를 보였다. 따라서 본 연구 결과는 사물인터넷 기반 뇌-기계 인터페이스가 일상생활 속 다양항 실시간 애플리케이션으로 활용될 수 있음을 시사한다.

Keywords

Acknowledgement

본 논문은 2022학년도 부경대학교(202212510001), 과학치안 공공연구성과 실용화 촉진 시범사업 연구과제(No.1711174175), 한국연구재단(No.2022H1D8A3038663), 한국연구재단(RS-2023-00242528), Startup Growth Technology Development Project(Strategic) (No.1425174568) 지원을 받아 수행됨.

References

  1. P. R. Roelfsema, D. Denys, and P. C. Klink, "Mind Reading and Writing: The Future of NeurotechNology," Trends in Cognitive Sciences, Vol.22, No.7, pp.598-610, 2018. https://doi.org/10.1016/j.tics.2018.04.001
  2. M. A. Lebedev and M. A. L. Nicolelis, "Brain?machine interfaces: past, present and future," Trends in Neurosciences, Vol.29, No.9, pp.536-546, 2006. https://doi.org/10.1016/j.tins.2006.07.004
  3. D. J. McFarland, W. A. Sarnacki, and J. R. Wolpaw, "Electroencephalographic (EEG) control of three-dimensional movement," Journal of Neural Engineering, Vol.7, No.3, pp.36007, 2010.
  4. Y. Chae, J. Jeong, and S. Jo, "Toward brain-actuated humaNoid robots: AsynchroNous direct control using an EEG-Based BCI," IEEE Transactions on Robotics, Vol.28, No.5, pp.1131-1144, 2012. https://doi.org/10.1109/TRO.2012.2201310
  5. K. Lafleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He, "Quadcopter control in three-dimensional space using a Noninvasive motor imagery-based brain-computer interface," Journal of Neural Engineering, Vol.10, No.4, p.46003, 2013.
  6. O. Al Zoubi, M. Awad, and N. K. Kasabov, "Anytime multipurpose emotion recognition from EEG data using a Liquid State Machine based framework," Artificial Intelligence in Medicine, Vol.86, pp.1-8, 2018. https://doi.org/10.1016/j.artmed.2018.01.001
  7. E. Tidoni, P. Gergondet, G. Fusco, A. Kheddar, and S. M. Aglioti, "The Role of Audio-Visual Feedback in a Thought-Based Control of a HumaNoid Robot: A BCI Study in Healthy and Spinal Cord Injured People," IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol.25, No.6, pp.772-781, 2017. https://doi.org/10.1109/TNSRE.2016.2597863
  8. F. Lopes da Silva, "EEG and MEG: Relevance to Neuroscience," Neuron, Vol.80, No.5, pp.1112-1128, 2013. https://doi.org/10.1016/j.neuron.2013.10.017
  9. K. Volkova, M. A. Lebedev, A. Kaplan, and A. Ossadtchi, "Decoding Movement From Electrocorticographic Activity: A Review," Frontiers in Neuroinformatics, Vol.13, pp.74, 2019.
  10. Y. W. Kim and H. B. Kang, "An Analysis of Visual Distraction and Cognitive Distraction using EEG," Journal of Korea Multimedia Society, Vol.21, No.2, pp.166-172, 2018. https://doi.org/10.9717/KMMS.2018.21.2.166
  11. H. H. Kim and J. Jeong, "An electrocorticographic decoder for arm movement for brain-machine interface using an echo state network and Gaussian readout," Applied Soft Computing, Vol.117, pp.1-13, 2022. https://doi.org/10.1016/j.asoc.2021.108393
  12. H. S. Wi and B. M. Lee, "Classification Method of Sleep Induction Sounds in Sleep Care Service based on Brain Wave," Journal of Korea Multimedia Society, Vol.23, No.11, pp.1406-1417, 2020. https://doi.org/10.9717/KMMS.2020.23.11.1406
  13. H. Han and B. M. Lee, "Customized Eyelid Warming Control Technique Using EEG Data in a Warming Mask for Sleep Induction," Journal of Korea Multimedia Society, Vol.24, No.8, pp.1149-1160, 2021. https://doi.org/10.9717/KMMS.2021.24.8.1149
  14. H. H. Kim, "Implementation of Brain-machine Interface System using Cloud IoT," Journal of Internet of Things and Convergence, Vol.9, No.1, pp.25-31, 2023. https://doi.org/10.20465/KIOTS.2023.9.1.025
  15. Neuralink Homepage. https://neuralink.com/
  16. A. MogNon, J. Jovicich, L. Bruzzone, and M. Buiatti, "ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features," Psychophysiology, Vol.48, No.2, pp.229-240, 2011. https://doi.org/10.1111/j.1469-8986.2010.01061.x
  17. L. Bozhkov, P. Koprinkova-Hristova, and P. Georgieva, "Reservoir computing for emotion valence discrimination from EEG signals," Neurocomputing, Vol.231, pp.28-40, 2017. https://doi.org/10.1016/j.neucom.2016.03.108