Current Optics and Photonics

Optical Society of Korea (한국광학회)
권호 표지
DOI QR코드

DOI QR Code

3D Augmented Reality Streaming System Based on a Lamina Display

  • Baek, Hogil (Department of Information Display, Kyung Hee University) ;
  • Park, Jinwoo (Department of Information Display, Kyung Hee University) ;
  • Kim, Youngrok (Department of Information Display, Kyung Hee University) ;
  • Park, Sungwoong (Department of Information Display, Kyung Hee University) ;
  • Choi, Hee-Jin (Department of Physics and Astronomy, Sejong University) ;
  • Min, Sung-Wook (Department of Information Display, Kyung Hee University)
  • Received : 2020.11.27
  • Accepted : 2020.12.17
  • Published : 2021.02.25
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a three-dimensional (3D) streaming system based on a lamina display that can convey field information in real-time by creating floating 3D images that can satisfy the accommodation cue. The proposed system is mainly composed of three parts, namely: a 3D vision camera unit to obtain and provide RGB and depth data in real-time, a 3D image engine unit to realize the 3D volume with a fast response time by using the RGB and depth data, and an optical floating unit to bring the implemented 3D image out of the system and consequently increase the sense of presence. Furthermore, we devise the streaming method required for implementing augmented reality (AR) images by using a multilayered image, and the proposed method for implementing AR 3D video in real-time non-face-to-face communication has been experimentally verified.

Keywords

References

  1. H. Hua, "Enabling focus cues in head-mounted displays," Proc. IEEE 105, 805-824 (2017). https://doi.org/10.1109/JPROC.2017.2648796
  2. M. von Waldkirch, P. Lukowicz, and G. Troster, "Defocusing simulations on a retinal scanning display for quasi accommodation-free viewing," Opt. Express 11, 3220-3233 (2003). https://doi.org/10.1364/OE.11.003220
  3. H. Hua and B. Javidi, "A 3D integral imaging optical seethrough head-mounted display," Opt. Express 22, 13484-13491 (2014). https://doi.org/10.1364/OE.22.013484
  4. A. Yuuki, K. Itoga, and T. Satake, "A new Maxwellian view display for trouble-free accommodation: a new Maxwellian view display," J. Soc. Inf. Disp. 20, 581-588 (2012). https://doi.org/10.1002/jsid.122
  5. F.-C. Huang, D. Luebke, and G. Wetzstein, "The light field stereoscope," in Proc. ACM Emerging Technologies on - SIGGRAPH '15 (LA, USA, July. 2015), Article no. 24.
  6. D. Lanman and D. Luebke, "Near-eye light field displays," ACM Trans. Graph. 32, 220 (2013).
  7. A. Maimone, D. Lanman, K. Rathinavel, K. Keller, D. Luebke, and H. Fuchs, "Pinlight displays: wide field of view augmented reality eyeglasses using defocused point light sources," ACM Trans. Graph. 33, 20 (2014).
  8. H.-J. Yeom, H.-J. Kim, S.-B. Kim, H. Zhang, B. Li, Y.-M. Ji, S.-H. Kim, and J.-H. Park, "3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation," Opt. Express 23, 32025-32034 (2015). https://doi.org/10.1364/OE.23.032025
  9. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, "Holographic head-mounted display with RGB light emitting diode light source," Opt. Express 22, 6526-6534 (2014). https://doi.org/10.1364/OE.22.006526
  10. T. Ando, K. Yamasaki, M. Okamoto, T. Matsumoto, and E. Shimizu, "Retinal projection display using holographic optical element," Proc. SPIE 3956, 211-216 (2000).
  11. K. J. MacKenzie, D. M. Hoffman, and S. J. Watt, "Accommodation to multiple-focal-plane displays: implications for improving stereoscopic displays and for accommodation control," J. Vis. 10, 22 (2010).
  12. S. Zhu, P. Jin, W. Qiao, and L. Gao, "High-resolution head mounted display using stacked LCDs and birefringent lens," Proc. SPIE 10676, 106761B (2018).
  13. P. V. Johnson, J. AQ. Parnell, J. Kim, C. D. Saunter, G. D. Love, and M. S. Banks, "Dynamic lens and monovision 3D displays to improve viewer comfort," Opt. Express 24, 11808-11827 (2016). https://doi.org/10.1364/OE.24.011808
  14. G. D. Love, D. M. Hoffman, P. J. W. Hands, J. Gao, A. K. Kirby, and M. S. Banks, "High-speed switchable lens enables the development of a volumetric stereoscopic display," Opt. Express 17, 15716-15725 (2009). https://doi.org/10.1364/OE.17.015716
  15. D. Kim, S. Lee, S. Moon, J. Cho, Y. Jo, and B. Lee, "Hybrid multi-layer displays providing accommodation cues," Opt. Express 26, 17170-17184 (2018). https://doi.org/10.1364/OE.26.017170
  16. W. Cui and L. Gao, "Optical mapping near-eye three-dimensional display with correct focus cues," Opt. Lett. 42, 2475-2478 (2017). https://doi.org/10.1364/OL.42.002475
  17. M. von Waldkirch, P. Lukowicz, and G. Troster, "Oscillating fluid lens in coherent retinal projection displays for extending depth of focus," Opt. Commun. 253, 407-418 (2005). https://doi.org/10.1016/j.optcom.2005.05.006
  18. S.-G. Park, S. Yoon, J. Yeom, H. Baek, S.-W. Min, and B. Lee, "Lamina 3D display: projection-type depth-fused display using polarization-encoded depth information," Opt. Express 22, 26162-26172 (2014). https://doi.org/10.1364/OE.22.026162
  19. S. Yoon, H. Baek, S.-W. Min, S.-G. Park, M.-K. Park, S.-H. Yoo, H.-R. Kim, and B. Lee, "Implementation of active-type Lamina 3D display system," Opt. Express 23, 15848-15856 (2015). https://doi.org/10.1364/OE.23.015848
  20. S.-G. Park, J.-H. Kim, and S.-W. Min, "Polarization distributed depth map for depth-fused three-dimensional display," Opt. Express 19, 4316-4323 (2011). https://doi.org/10.1364/OE.19.004316
  21. S. Ravikumar, K. Akeley, and M. S. Banks, "Creating effective focus cues in multi-plane 3D displays," Opt. Express 19, 20940-20952 (2011). https://doi.org/10.1364/OE.19.020940
  22. S.-G. Park, Y. Yamaguchi, J. Nakamura, B. Lee, and Y. Takaki, "Long-range 3D display using a collimated multi-layer display," Opt. Express 24, 23052-23062 (2016). https://doi.org/10.1364/OE.24.023052
  23. S. Liu and H. Hua, "A systematic method for designing depthfused multi-focal plane three-dimensional displays," Opt. Express 18, 11562-11573 (2010). https://doi.org/10.1364/OE.18.011562
  24. A. Tsunakawa, T. Soumiya, H. Yamamoto, and S. Suyama, "Perceived depth change of depth-fused 3-D display with changing distance between front and rear planes," IEICE Trans. Electron. E96.C, 1378-1383 (2013). https://doi.org/10.1587/transele.E96.C.1378
  25. S.-G. Park, J.-H. Jung, Y. Jeong, and B. Lee, "Depth-fused display with improved viewing characteristics," Opt. Express 21, 28758-28770 (2013). https://doi.org/10.1364/OE.21.028758
  26. S. Liu, D. Cheng, and H. Hua, "An optical see-through head mounted display with addressable focal planes," in Proc. 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (Cambridge, UK, September. 2008), pp. 33-42.
  27. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, "Vergence-accommodation conflicts hinder visual performance and cause visual fatigue," J. Vis. 8, 33 (2008).
  28. G.-A. Koulieris, B. Bui, M. S. Banks, and G. Drettakis, "Accommodation and comfort in head-mounted displays," ACM Trans. Graph. 36, 87 (2017).
  29. M. Lambooij, M. Fortuin, W. Ijsselsteijn, B. Evans, and I. Heynderickx, "Measuring visual fatigue and visual discomfort associated with 3-D displays," J. Soc. Inf. Disp. 18, 931-943 (2010). https://doi.org/10.1889/JSID18.11.931
  30. M. Lambooij, W. IJsselsteijn, M. Fortuin, and I. Heynderickx, "Visual discomfort and visual fatigue of stereoscopic displays: a review," J. Imaging Sci. Technol. 53, 030201-1-30201-14 (2009). https://doi.org/10.2352/J.ImagingSci.Technol.2009.53.3.030201
  31. T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks, "The zone of comfort: Predicting visual discomfort with stereo displays," J. Vision 11, 11 (2011).
  32. H. Urey, K. V. Chellappan, E. Erden, and P. Surman, "State of the Art in Stereoscopic and Autostereoscopic Displays," Proc. IEEE 99, 540-555 (2011). https://doi.org/10.1109/JPROC.2010.2098351
  33. Y. Takaki and Y. Yamaguchi, "Flat-panel see-through three-dimensional display based on integral imaging," Opt. Lett. 40, 1873-1876 (2015). https://doi.org/10.1364/OL.40.001873
  34. X. Shen and B. Javidi, "Large depth of focus dynamic micro integral imaging for optical see-through augmented reality display using a focus-tunable lens," Appl. Opt. 57, B184-B189 (2018). https://doi.org/10.1364/AO.57.00B184
  35. X. Hu and H. Hua, "High-resolution optical see-through multifocal-plane head-mounted display using freeform optics," Opt. Express 22, 13896-13903 (2014). https://doi.org/10.1364/OE.22.013896
  36. G. E. Favalora, J. Napoli, D. M. Hall, R. K. Dorval, M. Giovinco, M. J. Richmond, and W. S. Chun, "100-million-voxel volumetric display," Proc. SPIE 4712, 300-312 (2000)
  37. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, "Rendering for an interactive 360° light field display," ACM Trans. Graph. 26, 40 (2007). https://doi.org/10.1145/1276377.1276427
  38. D. Zhao, L. Ma, C. Ma, J. Tang, and H. Liao, "Floating autostereoscopic 3D display with multidimensional images for telesurgical visualization," Int J. CARS 11, 207-215 (2016). https://doi.org/10.1007/s11548-015-1289-8
  39. H. Choi, Y. Kim, J.-H. Park, S. Jung, and B. Lee, "Improved analysis on the viewing angle of integral imaging," Appl. Opt. 44, 2311-2317 (2005). https://doi.org/10.1364/AO.44.002311
  40. H. Kakeya, "MOEVision: simple multiview display with clear floating image," Proc. SPIE 6490, 64900J (2007).
  41. H. Yamamoto, H. Bando, R. Kujime, and S. Suyama, "Design of crossed-mirror array to form floating 3D LED signs," Proc. SPIE 8288, 828820 (2012).
  42. Y. Yoshimizu and E. Iwase, "Radially arranged dihedral corner reflector array for wide viewing angle of floating image without virtual image," Opt. Express 27, 918-927 (2019). https://doi.org/10.1364/OE.27.000918
  43. S. Choi, S. Park, and S.-W. Min, "Design of ghost-free floating 3D display with narrow thickness using offset lens and dihedral corner reflector arrays," Opt. Express 28, 15691-15705 (2020). https://doi.org/10.1364/oe.392036
  44. D. Miyazaki, N. Hirano, Y. Maeda, S. Yamamoto, T. Mukai, and S. Maekawa, "Floating volumetric image formation using a dihedral corner reflector array device," Appl. Opt. 52, A281-A289 (2013). https://doi.org/10.1364/AO.52.00A281
  45. D. Guffanti, A. Brunete, M. Hernando, J. Rueda, and E. Navarro Cabello, "The accuracy of the Microsoft Kinect V2 sensor for human gait analysis. A different approach for comparison with the ground truth," Sensors 20, 4405 (2020). https://doi.org/10.3390/s20164405