Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Trends and Future Directions on Extended Reality based Human Digital Augmentation Technology

확장현실 기반 휴먼 디지털 증강 기술 동향과 발전방향

  • 신춘성 (전남대학교 문화전문대학원) ;
  • 이영호 (국립목포대학교 컴퓨터공학과) ;
  • 윤효석 (한신대학교 컴퓨터공학부)
  • Received : 2020.07.09
  • Accepted : 2020.09.08
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the advent of the 4th industrial revolution, technologies for improving human ability are being developed. In particular, human augmentation is developing in the form of not only detecting the ability to understand internal and external conditions of humans, but also detects one's deficiencies and enhances their abilities. With the explosive growth and proliferation of extended reality, human augmentation technology has the potential to be utilized in various aspects of our lives. To analyze recent human augmentation technology in forms of digital enhancements, this paper examines the history and concept of human augmentation, which began with the trans-humanism. Furthermore, we identify core characteristics of human augmentation as user state quantification, cognitive ability measurement and analysis. Then we present how these characteristics should be used in extended reality based human digital augmentation for enhancing cognitive abilities. Lastly, we discuss future directions on feasible applications and development direction of digital human augmentation.

4차 산업혁명의 도래와 함께 인간의 능력을 향상하기 위한 기술이 진화하고 있다. 특히 휴먼 디지털 증강은 컴퓨팅 기술의 발전을 통해 인간의 내외적 상태를 정확하게 이해하고 부족한 능력을 인지할 뿐만 아니라 확장 및 향상하는 형태로 발전하고 있다. 여기에 확장현실의 폭발적인 성장과 확산을 통해 휴먼 증강 기술은 우리 생활의 다양한 분야에서 활용 가능한 잠재력을 가지고 있다. 본 논문은 주목을 받고 있는 휴먼 디지털 증강을 살펴보기 위해 트렌스 휴머니즘으로 시작된 휴먼 증강의 역사와 개념을 살펴보고 휴먼 디지털 증강의 개념을 정리한다. 또한 휴먼 디지털 증강을 실현하기 위한 주요 기술로 사용자 상태 정량화와 인지 측정 및 분석 기술을 살펴보고, 이를 기반으로 한 확장현실 기반 휴먼 디지털 인지능력 증강기술을 소개한다. 더 나아가 향후 휴먼 디지털 증강기술의 적용분야 및 발전 방향을 논의한다.

Keywords

References

  1. Bhattacharyya, R., Coffman, B., Choe, J., and Phillips, M. (2017). Does Neurotechnology Produce a Better Brain?, Computer, 50(2), 48-58. https://doi.org/10.1109/MC.2017.49.
  2. Bourne, R., Flaxman, S. R., Braithwaite, T., Cicinelli, M. V., Das, A., Jonas, J. B., Keeffe, J., Kempen, J. H., Leasher, J., Limburg, H., Naidoo, K., Pesudovs, K., Resnikoff, S., Silvester, A., Stevens, G. A., Tahhan, N., Wong, T. Y., and Taylor, H. R. (2017). Magnitude, Temporal Trends, and Projections of the Global Prevalence of Blindness and Distance and Near Vision Impairment: A Systematic Review and Meta-analysis, The Lancet Global Health, 5(9). https://doi.org/10.1016/S2214-109X(17)30293-0.
  3. Cao, Y., Xu, B., Ye, Z., Yang, J., Cao, Y. Tisse, C., and Li, X. (2018). Depth and Thermal Sensor Fusion to Enhance 3D Thermographic Reconstruction, Optics Express, 26(7), 8179-8193. https://doi.org/10.1364/OE.26.008179.
  4. Chiu, C., Thelwell, M., Senior, T., Choppin, S.,Hart, J., and Wheat, J. (2019). Comparison of Depth Cameras for Three Dimensional Reconstruction in Medicine, Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 233(9), 938-947. https://doi.org/10.1177/0954411919859922.
  5. Cho, J., and Choi, A. (2020). Cuffless Blood Pressure Estimation Based on a Convolutional Neural Network using PPG and ECG Signals for Portable or Wearable Blood Pressure Devices, 25(3), 1-10. https://doi.org/10.9723/jksiis.2020.25.3.001.
  6. Chu, T. (2014). Human Purpose and Transhuman Potential: A Cosmic Vision of Our Future Evolution, Origin Press.
  7. Coffey, C., Ratcliff, G., Saxton, J., Bryan, R., Fried, L., and Lucke, J. (2001). Cognitive Correlates of Human Brain Aging: A Quantitative Magnetic Resonance Imaging Investigation, The Journal of Neuropsychiatry and Clinical Neurosciences, 13(4), 471-485. https://doi.org/10.1176/jnp.13.4.471.
  8. Engelbart, D. C., and English, W. K. (1968). A Research Center for Augmenting Human Intellect, Proceedings of the 1968 Fall Joint Computer Conference, Part I (AFIPS '68 (Fall, part I)) , Dec. 9-11, San Francisco, California, USA, pp. 395-410.
  9. Fan, K., Huber, J. Nanayakkara, S., and Inami, M. (2014). SpiderVision: Extending the Human Field of View for Augmented Awareness, Proceedings of the 5th Augmented Human International Conference, Kobe, Japan, pp. 1-8.
  10. Farooq, U., and Grudin, J. (2016). Human-computer Integration, Interactions, 23(6), 26-32. https://doi.org/10.1145/3001896.
  11. Grady, C. L. (2000). Functional Brain Imaging and Age-related Changes in Cognition, Biological Psychology, 54(1-3), 259-281. https://doi.org/10.1016/s0301-0511(00)00059-4.
  12. Hamanishi, N., and Rekimoto, J. (2017). Body Cursor: Supporting Sports Training with the Out-of-Body Sense, Proceedings of CHI 2017 Workshop on Amplification and Augmentation of Human Perception, May 7, Denver, CO, USA.
  13. Held, B., Aljuneidi, S., Pham, V. T.-V., and Joseph, A. (2019). Helon 360: A Smart Fire Fighters' Helmet Integrated Augmented Reality and $360^{\circ}$ Thermal Image Data Streaming, https://doi.org/10.13140/RG.2.2.28158.41287.
  14. Kendall, A., Matthew K. G., and Roberto C. (2015). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization, IEEE International Conference on Computer Vision (ICCV) , Dec. 7-13, Santiago, Chile, pp. 2938-2946.
  15. Kim, S., and Dey, A. K. (2009). Simulated Augmented Reality Windshield Display as a Cognitive Mapping Aid for Elder Driver Navigation, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Apr 4-9, Boston, Massachusetts, USA, pp. 133-142.
  16. Koch Dandolo, C. L., Cosentino, A., and Jepsen, P. U. (2015). Inspection of Panel Paintings Beneath Gilded Finishes Using Terahertz Time-domain Imaging, Studies in Conservation, Supplement 1, pp. S159-S166. https://doi.org/10.1179/0039363015Z.000000000220.
  17. Kwon, M.-R., Hong, K.-J., and Jung, K.-C. (2016). Customized Search System using Real-time Contexts of User, Journal of the Korea Industrial Information Systems Research, 21(5), 19-30. https://doi.org/10.9723/jksiis.2016.21.5.019.
  18. Lee, J., Kim, E., Yu, J., Kim, J., and Woo, W. (2018). Holistic Quantified Self Framework for Augmented Human, International Conference on Distributed, Ambient, and Pervasive Interactions, Jul. 15-20, Las Vegas, NV, USA, pp. 188-201.
  19. Licklider, J. C. R., (1960). Man-Computer Symbiosis, IRE Transactions on Human Factors in Electronics, HFE-1(1), 4-11. https://doi.org/10.1109/THFE2.1960.4503259.
  20. Mann, S., Furness, T., Yuan, Y., Lorio, J., and Wang, Z. (2018). All Reality: Virtual, Augmented, Mixed (X), Mediated (X,Y), and Multimediated Reality, Retrieved August 2, 2018, from Cornell University, arXiv website: https://arxiv.org/abs/1804.08386.
  21. Min, J.-K., Doryab, A., Wiese, J., Amini, S., Zimmerman, J., and Hong, J. (2014). Toss 'N' turn: Smartphone as Sleep and Sleep Quality Detector, CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Apr. 26-May 1, Toronto, Canada, pp. 477-486.
  22. Min, S., and Oh, Y. (2020). Survey on Obstacle Detection Features of Smart Technologies to Help Visually Impaired Poepl Walk, Journal of the Korea Industrial Information Systems Research, 25(3), 31-38. https://doi.org/10.9723/jksiis.2020.25.3.031.
  23. Oleksy, T., and Wnuk, A. (2016). Augmented Places: An Impact of Embodied Historical Experience on Attitudes Towards Places, Computers in Human Behavior, 57, 11-16. https://doi.org/10.1016/j.chb.2015.12.014.
  24. Padmanaban, N., Konrad, R., and Wetzstein, G. (2019). Autofocals: Evaluating Gaze-contingent Eyeglasses for Presbyopes, Science Advances, 5(6), eaav6187. https://doi.org/10.1126/sciadv.aav6187.
  25. Ryu, W.-S., Kim, H., and Chung, S.-T. (2010). An Empirical Study on Quantitative Evaluation of Cognitive Function, Progress in Medical Physics, 21(1), 42-51.
  26. Redmon, J., and Farhadi, A. (2016). YOLO9000: Better, Faster, Stronger, Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 21-26, Honolulu, HI, USA, pp. 6517-6525.
  27. Schmidt, A. (2017). Augmenting Human Intellect and Amplifying Perception and Cognition, IEEE Pervasive Computing Systems, 16(1), 6-10. https://doi.org/10.1109/MPRV.2017.8
  28. Shin, C., Hong, J-H., and Dey, A. K. (2012). Understanding and Prediction of Mobile Application Usage for Smart Phones, Proceedings of the UbiComp, Sep. 5-8, Pittsburgh, Pennsylvania, USA, pp 173-182.
  29. Starner, T., Mann, S., Rhodes, B., Levine, J., Healey, J., Kirsch, D., Picard, R. W., and Pentland, A. (1997). Augmented Reality Through Wearable Computing, Presence: Teleoperators and Virtual Environments, 6(4), 386-398. https://doi.org/10.1162/pres.1997.6.4.386.
  30. Stephanidis, C., Salvendy, G., Antona, M., Chen, J. Y. C., Dong, J., Duffy, V. G., Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L. P., Guo, Y., Harris, D., Ioannou, A., Jeong, K., Konomi, S., Kromker, H., Kurosu, M., Lewis, J. R., Marcus, A., Meiselwitz, G., Moallem, A., Mori, H., Nah, F. F.-H., Ntoa, S., Rau, P.-L. P., Schmorrow, D., Siau, K., Streitz, N., Wang, W., Yamamoto, S., Zaphiris, P., and Zhou, J. (2019). Seven HCI Grand Challenges, International Journal of Human-Computer Interaction, 35(14), 1229-1269, https://doi.org/10.1080/10447318.2019.1619259.
  31. Tag, B., Mannschreck, R., Sugiura, K., Chernyshov, G., Ohta, N., and Kunze, K. (2017). Facial Thermography for Attention Tracking on Smart Eyewear: An Initial Study, CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing, May 6-11, Denver, Colorado, USA, pp. 2959-2966.
  32. Tatarian, K., Couceiro, M., Ribeiro, E., and Faria, D. (2018). Stepping-stones to Transhumanism: An EMG-controlled Low-cost Prosthetic Hand for Academia, 2018 International Conference on Intelligent Systems (IS), Sep. 25-27, Funchal - Madeira, Portugal, pp. 807-812.
  33. Weiser, M. (1991). The Computer for the 21st Century, Scientific American, 265(3), 94-105. https://doi.org/10.1038/scientificamerican0991-94
  34. Willis, K. D. D., and Wilson, A. D. (2013). InfraStructs: Fabricating Information Inside Physical Objects for Imaging in the Terahertz Region, ACM Transactions on Graphics, 32(4), 138. https://doi.org/10.1145/2461912.2461936.
  35. Wolf, K. (2017). Augmenting Interface Perception through Sensory Illusion, Proceedings of CHI 2017 Workshop on Amplification and Augmentation of Human Perception, May 6-11, Denver, Colorado, USA.
  36. World Economic Forum. (2016). World Economic Forum Annual Meeting 2016: Mastering the Fourth Industrial Revolution, http://www3.weforum.org/docs/WEF_AM16_Report.pdf (Accessed on Aug. 21st, 2020)
  37. Xu, W. (2019). Toward human-centered AI: a Perspective from Human-computer Interaction, Interactions, 26(4), 42-46. https://doi.org/10.1145/3328485.
  38. Yang, Z., Shi, J., Jiang, W., Sui, Y., Wu, Y., Ma, S., Kang, C., and Li, H. (2019). Influences of Augmented Reality Assistance on Performance and Cognitive Loads in Different Stages of Assembly Task, Frontiers in Psychology, 10, 1703. https://doi.org/10.3389/fpsyg.2019.01703.
  39. Yoon, H., Doh, Y. Y., Yi, M. Y., and Woo, W. (2014). A Conceptual Framework for Augmented Smart Coach based on Quantified Holistic Self, International Conference on Distributed, Ambient, and Pervasive Interactions, Jun. 22-27, Heraklion, Crete, Greece, pp. 498-508.
  40. Yoon, H., and Shin, C. (2019). Cross-Device Computation Coordination for Mobile Collocated Interactions with Wearables, Sensors, 19(4), 796. https://doi.org/10.3390/s19040796.
  41. Zientara, P., Lee, S., Smith, G. H., Brenner, R., Itti, L., Rosson, M. B., Carroll, J., Irick, K., and Narayanan, V. (2017). Third Eye: A Shopping Assistant for the Visually Impaired, Computer, 50(2). 16-24. https://doi.org/10.1109/MC.2017.36.