Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지

Sound Researches in Computer Graphics Community: Part I. Sound Synthesis and Spatialization

컴퓨터 그래픽스 커뮤니티에 소개된 사운드 관련 연구들: Part I. 사운드 합성과 공간화

  • Published : 2009.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

Sound is very important element to enhance and reinforce reality and immersion of users in virtual reality and computer animation. Recently, significant researches about sound modeling are presented in computer graphics community. In this article, main subjects are explained and major researches are reviewed based on the sound papers presented in computer graphics community. Specially, several papers about following two subjects are reviewed in this paper: 1) synthesing sound using physically-based laws and generating sound synchronized with graphics. 2) spatializing sound and modeling sonic environment. Many research about sound modeling have been focused on more efficient modeling of real physical law and generate realistic sound with limited resources. Based on this concept, various papers are introduced and the relationship between researches about sound and graphics is discussed.

사운드는 가상현실 및 컴퓨터 애니메이션에서 사용자의 현실감 및 집중도를 높이기 위한 중요한 요소이다. 최근에 컴퓨터 그래픽스 분야에서도 연구자들이 사운드 관련 연구들에 관심을 가지고 여러 컴퓨터 그래픽스 학술지와 학회에 주목할만한 연구결과들이 발표되고 있다. 본 논문에서는 컴퓨터 그래픽스 분야에서 소개된 논문들을 중심으로, 컴퓨터 그래픽스에서 관심을 가지는 사운드 연구의 주제에 대하여 알아보고 중요성을 가지는 논문들을 소개한다. 특히 본 논문에서는 다음 두 가지 분야의 논문들을 소개한다. 첫째, 사운드를 물리적으로 합성함으로써 그래픽스의 물리 법칙에 동기화되는 사운드를 생성하는 것이다. 둘째, 음원에서 나오는 사운드가 사람에게 들리는 음향적 환경을 모델링하는 것이다. 사운드에 대한 연구에서 주된 관심은 이러한 음향학적인 물리법칙을 더욱 효율적으로 모델링하여 적은 자원으로도 실제적인 사운드 효과를 발생할 수 있는지에 대한 것이다. 이러한 관점을 중점으로 다양한 사운드 관련 논문들을 살펴보고 컴퓨터 그래픽스와의 관계를 살펴보도록 한다.

Keywords

References

  1. N. I. Durlach and A. S. Mavor, Virtual reality scientific and technological challenges. National Academy Press, 1995.
  2. R. L. Storms, “Auditory-visual cross-modal perception phenomena,” Ph.D. dissertation, Naval Postgraduate School, Monterey, California, 1998.
  3. D. Terzopoulos and K. Fleischer, “Deformable models,” The Visual Computer, vol. 4, no. 6, pp. 306–331, 1988.
  4. P. R. Cook, Real sound synthesis for interactive applications. Natick, MA: A.K.Peters. Ltd., 2002.
  5. I. Bork, A. Chaigne, L.-C. Trebuchet, M. Kosfelder, and D. Pillot, “Comparison between modal analysis and finite element modeling of a marimba bar,” Acoustica united with Acta Acustica, vol. 85, no. 2, pp. 258–266, 1999.
  6. J. Bretos, C. Santamaria, and J. A. Moral, “Finite element analysis and experimental measurements of natural eigenmodes and random responses of wooden bars used in musical instruments,” Applied Acoustics, vol. 56, pp. 141–156, 1999.
  7. F. Orduna-Bustamante, “Nonuniform beams with harmonically related overtones for use in percussion instruments,” Journal of the Acoustical Society of America, vol. 90, no. 6, pp. 3582–3583, 1991.
  8. J. F. O’Brien, P. R. Cook, and G. Essl, “Synthesizing sounds from physically based motion,” in Proc. ACM SIGGRAPH, 2001, pp. 529–536.
  9. S. S. An, T. Kim, and D. L. James, “Optimizing cubature for efficient integration of subspace deformations,” in Proc. ACM SIGGRAPH Asia, 2009.
  10. K. van denDoel and D. K. Pai, “Synthesis of shape dependent sounds with physical modeling,” in Proc. the International Conference on Auditory Displays, 1996.
  11. K. van denDoel and D. K. Pai, “The sound of physical shapes,” Presence, vol. 7, no. 4, pp. 382-395, 1998.
  12. K. van denDoel and D. K. Pai, “Foley automatic: physicallybased sound effects for interactive simulation and animation,” in Proc. ACM SIGGRAPH, 2001, pp. 537–544.
  13. D. K. Pai, K. van denDoel, K. James, J. Lang, J. E. Lloyd, J. L. Richmond, and S. H. Yau, “Scanning physical interaction behavior of 3d objects,” in Proc. ACM SIGGRAPH, 2001, pp. 87–96.
  14. J. L. Richmond and D. K. Pai, “D.k. robotic measurement and modeling of contact sounds,” in Proc. the International Conference on Auditory Displays, 2000.
  15. J. F. O’Brien, C. Shen, and C. M. Gatchalian, “Synthesizing sounds from rigid-body simulations,” in Proc. ACM SIGGRAPH Symposium on Computer Animation, 2002, pp. 175-181.
  16. N. Raghuvanshi and M. C. Lin, “Interactive sound synthesis for large scale environments,” in Proc. ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, 2006, pp. 101-108.
  17. A. Sek and B. C. Moore, “Frequency discrimination as a function of frequency, measured in several ways,” in Audio Anecdotes, 2003.
  18. N. Raghuvanshi and M. C. Lin, “Physically based sound synthesis for large-scale virtual environments,” IEEE Computer Graphics and Application, vol. 27, no. 1, pp. 14-18, 2007.
  19. K. van denDoel, D. Knott, and D. K. Pai, “Measurements of perceptual quality of contact sound models,” in Proc. the International Conference on Auditory Displays, 2002, pp. 345-349.
  20. K. van denDoel, D. Knott, and D. K. Pai, “Interactive simulation of complex audiovisual scenes,” Presence: Teleoperators and Virtual Environments, vol. 13, no. 1, pp. 99-111, 2004. https://doi.org/10.1162/105474604774048252
  21. E. M. Painter and A. S. Spanias, “A review of algorithms for perceptual coding of digital audio signals,” in Proc. Digital Signal Proceesings, 1997, pp. 179-208.
  22. N. Bonneel, D. Drettakis, N. Tsingos, I. Viaud-Delmon, and D. James, “Fast modal sounds with scalable frequencydomain synthesis,” in Proc. ACM SIGGRAPH, 2008.
  23. K. van denDoel and D. K. Pai, “Modal synthesis for vibrating objects,” in Audio Anecdotes, 2003.
  24. Y. Dobashi, T. Yamamoto, and T. Nishita, “Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics,” in Proc. ACM SIGGRAPH, 2003, pp. 732-740.
  25. P. Larsson, D. vastfjall, and M. Kleiner, “Better presence and performance in virtual environments by improved binaural sound rendering,” in Proc. AES 22nd International Conference on virtual, synthetic and entertainment audio, 2002, pp. 31-38.
  26. H. Kuttruff, Room Acoustics, 3rd Edition. London: Elsevier Science, 1991.
  27. T. Funkhouser, I. Carlbon, G. Elko, G. Pingali, M. Sondhi, and J. West, “A beam tracking approach to acoustic modeling for interactive virtual environments,” in Proc. ACM SIGGRAPH, 1998, pp. 21-32.
  28. T. Funkhouser, P. Min, and I. Carlbon, “Real-time acoustic modeling for distributed virtual environments,” in Proc. ACM SIGGRAPH, 1999, pp. 365-374.
  29. P. Min and T. Funkhouser, “Priority-driven acoustic modeling for virtual environments,” Computer Graphics Forum, vol. 19, no. 3, 2000.
  30. N. Tsingos, T. Funkhouser, T. Ngan, and I. Carlbon, “Modeling acoustics in virtual environments using the uniform theory of diffraction,” in Proc. ACM SIGGRAPH, 2001, pp. 545- 552.
  31. D. A. McNamara, C. W. I. Pistorius, and J. A. G. Malherbe, Introduction to the uniform geometrical theory of diffraction. Artech Houser, 1990.
  32. P. P. Sloan, J. Kautz, and J. Snyder, “Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments,” in Proc. ACM SIGGRAPH, 2002, pp. 527-536.
  33. D. L. James, J. Barbic, and D. K. Pai, “Precomputed acoustic transfer: Output-sensitive, accurate sound generation for geometrically complex vibration sources,” in Proc. ACM SIGGRAPH, 2006, pp. 987-995.
  34. T. Takala and J. Hahn, “Sound rendering,” in Proc. ACM SIGGRAPH, 1992, pp. 111-220.
  35. W. Martens, “Principal components analysis and resynthesis of spectral cues to perceived direction,” in Proc. International Computer Music Conference, 1987, pp. 274-281.
  36. J. Chen, B. V. Veen, and K. Hecox, “A spatial feature extraction and regularization model for the head-related transfer function,” Journal of the Acoustical Society of America, vol. 97, pp. 439-452, 1995. https://doi.org/10.1121/1.413110
  37. H. Fouad, J. Hahn, and J. Ballas, “Perceptually based scheduling algorithms for real-time synthesis of complex sonic environments,” in Proc. the International Conference on Auditory Displays, 1997.
  38. J. Herder, “Optimization of sound spatialization resource management through clustering,” Journal of Three Dimensional Images, 3D-Forum Society, vol. 13, no. 3, pp. 59-65, 1999.
  39. E. Paquette, P. Poulin, and G. Drettakis, “A light hierarchy for fast rendering of scenes with many lights,” Computer Graphics Forum, vol. 17, pp. 63-74, 1998. https://doi.org/10.1111/1467-8659.00254
  40. B. C. Moore, An introduction to the psychology of hearning. Academic Press, 4th edition, 1997.
  41. J. Fouad, J. Ballas, and D. Brock, “An extensible toolkit for creating virtual sonic environments,” in Proc. the International Conference on Auditory Displays, 2000.
  42. E. Wenzel, J. Miller, and J. Abel, “A software-based system for interactive spatial sound synthesis,” in Proc. the International Conference on Auditory Displays, 2000.
  43. L. Savioja, J. Huopaniemi, T. Lokki, and R. Vaananen, “Creating interactive virtual acoustic environments,” Journal of the Audio Engineering Society, vol. 47, no. 8.
  44. N. Tsingos, E. Gallo, and G. Drettakis, “Perceptual audio rendering of complex virtual environments,” in Proc. ACM SIGGRAPH, 2004, pp. 249-258.
  45. T. Funkhouser and C. Sequin, “Adaptive display algorithms for interactive frame rates during visualization of complex virtual environments,” in Proc. ACM SIGGRAPH, 1993, pp. 247-254.
  46. M. Wand and W. Strasser, “Multi-resolution sound rendering,” in Proc. Symposium on Point-Based Graphics, 2004.
  47. T. Moeck, N. Bonneel, N. Tsingos, G. Drettakis, I. Viaud- Delmon, and D. Aloza, “Progressive perceptual audio rendering of complex scenes,” in Proc. ACM SIGGRAPH Symposium on interactive 3D Graphics and Games, 2007, pp. 189-196.
  48. C. Kayser, C. Petkov, M. Lippert, and N. Logothetis, “Mechanisms for allocating auditory attention: An auditory saliency map,” Current Biology, vol. 15, pp. 1943-1947, 2005. https://doi.org/10.1016/j.cub.2005.09.040
  49. R. Guski and N. Troje, “Audiovisual phenomenal causality,” Perception and Psuchophysics, vol. 65, no. 5, pp. 789-800, 2003. https://doi.org/10.3758/BF03194815
  50. D. Grelaud, N. Bonneel, M. Wimmer, M. Asselot, and G. Drettakis, “Efficient and practical audio-visual rendering for games using crossmodal perception,” in Proc. ACM SIGGRAPH Symposium on interactive 3D Graphics and Games, 2009.
  51. N. Bonneel, C. Suied, I. Viaud-Delmon, and G. Drettakis, “Bimodal perception of audio-visual material properties for virtual environments,” ACM Transactions on Applied Perception (Accepted with minor revisions), 2009.
  52. W. Hairston, M. Wallace, J. V. Stein, J. Norris, and J. Schirillo, “Visual localization ability influences cross-modal bias,” Journal of Cognition Neroscience, vol. 15, pp. 20-29, 2003. https://doi.org/10.1162/089892903321107792
  53. D. Alais and D. Burr, “The ventriloquism effect results from near-optimal bimodal integration,” Current Biology, vol. 14, pp. 257-262, 2004. https://doi.org/10.1016/j.cub.2004.01.029