Journal of Computing Science and Engineering

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Registration of 3D CT Data to 2D Endoscopic Image using a Gradient Mutual Information based Viewpoint Matching for Image-Guided Medialization Laryngoplasty

  • Yim, Yeny (Department of Computer Science, The George Washington University) ;
  • Wakid, Mike (Department of Computer Science, The George Washington University) ;
  • Kirmizibayrak, Can (Department of Computer Science, The George Washington University) ;
  • Bielamowicz, Steven (The George Washington University Medical Center) ;
  • Hahn, James (Department of Computer Science, The George Washington University)
  • Received : 2010.10.25
  • Accepted : 2010.12.10
  • Published : 2010.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a novel method for the registration of 3D CT scans to 2D endoscopic images during the image-guided medialization laryngoplasty. This study aims to allow the surgeon to find the precise configuration of the implant and place it into the desired location by employing accurate registration methods of the 3D CT data to intra-operative patient and interactive visualization tools for the registered images. In this study, the proposed registration methods enable the surgeon to compare the outcome of the procedure to the pre-planned shape by matching the vocal folds in the CT rendered images to the endoscopic images. The 3D image fusion provides an interactive and intuitive guidance for surgeon by visualizing a combined and correlated relationship of the multiple imaging modalities. The 3D Magic Lens helps to effectively visualize laryngeal anatomical structures by applying different transparencies and transfer functions to the region of interest. The preliminary results of the study demonstrated that the proposed method can be readily extended for image-guided surgery of real patients.

Keywords

References

  1. ANDERSON, T. D., J. R. SPIEGEL, and R. T. SATALOFF. 2003. Thyroplasty revisions: frequency and predictive factors. J Voice. 17, 3, 442-8. https://doi.org/10.1067/S0892-1997(03)00080-8
  2. ANON, J. B., S. P. LIPMAN, D. OPPENHEIM, AND R. A. HALT. 1994. Computer-assisted endoscopic sinus surgery. Laryngoscope. 104, 7, 901-5.
  3. AZUMA, R. 1997. Survey of Augmented Reality. Presence: teleoperators and virtual environments. 6, 4, 355-385.
  4. BERGSTROM, M. AND T. GREITZ. 1976. Stereotaxic computed tomography. AJR Am J Roentgenol. 127, 1, 167-70. https://doi.org/10.2214/ajr.127.1.167
  5. BETTEGA, G., P. CINQUIN, J. LEBEAU, AND B. RAPHAEL. 2002. Computer-assisted orthognathic surgery: clinical evaluation of a mandibular condyle repositioning system. J Oral Maxillofac Surg. 60, 1, 27-34; discussion 34-5. https://doi.org/10.1053/joms.2002.29069
  6. BIELAMOWICZ, S. 2004. Perspectives on medialization laryngoplasty. Otolaryngol Clin North Am. 37, 1, 139-60, vii. https://doi.org/10.1016/S0030-6665(03)00166-X
  7. BIRKFELLNER, W., P. SOLAR, A. GAHLEITNER, K. HUBER, F. KAINBERGER, ET AL. 2001. In-vitro assessment of a registration protocol for image guided implant dentistry. Clin Oral Implants Res. 12, 1, 69-78. https://doi.org/10.1034/j.1600-0501.2001.012001069.x
  8. BROWN, R. A. 1979. A computerized tomography-computer graphics approach to stereotaxic localization. J Neurosurg. 50, 6, 715-20. https://doi.org/10.3171/jns.1979.50.6.0715
  9. BURNS, R. P. 1997. Image-guided breast biopsy. Am J Surg. 173, 1, 9-11; discussion 12-3. https://doi.org/10.1016/S0002-9610(96)00374-1
  10. DARABI, K., P. GRUNERT, AND A. PERNECZKY. 1997. Accuracy of intraoperative navigation using skin markers. Computer Assisted Radiology and Surgery: 920-924.
  11. DEGUCHI, D., K. AKIYAMA, K. MORI, T. KITASAKA, Y. SUENAGA, ET AL. 2006. A method for bronchoscope tracking by combining a position sensor and image registration. Comput Aided Surg. 11, 3, 109-17. https://doi.org/10.3109/10929080600751159
  12. DELIGIANNI, F., A. J. CHUNG, AND G. Z. YANG. 2006. Nonrigid 2-D/3-D registration for patient specific bronchoscopy simulation with statistical shape modeling: phantom validation. IEEE Trans Med Imaging. 25, 11, 1462-71. https://doi.org/10.1109/TMI.2006.883452
  13. DEY, D., D. G. GOBBI, P. J. SLOMKA, K. J. M. SURRY, AND T. M. PETERS. 2002. Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: Creating stereoscopic panoramas. IEEE Transactions on Medical Imaging. 21, 1, 23-30. https://doi.org/10.1109/42.981231
  14. DEY, J. AND S. NAPEL. 2006. Targeted 2D/3D registration using ray normalization and a hybrid optimizer. Med Phys. 33, 12, 4730-8. https://doi.org/10.1118/1.2388156
  15. ERIC, A., E. A. BIER, M. C. STONE, K. PIER, W. BUXTON, ET AL. 1993. Toolglass and Magic Lenses: The See-Through Interface. Proc. of the 20th Ann.Conf. on Computer Graphics and Interactive Techniques 1, 73-80.
  16. FU, D. AND G. KUDUVALLI. 2008. A fast, accurate, and automatic 2D-3D image registration for image-guided cranial radiosurgery. Med Phys. 35, 5, 2180-94. https://doi.org/10.1118/1.2903431
  17. GONZALEZ, R. C. AND R. E. WOODS. 1992. Digital image processing.
  18. HAUTMANN, H., A. SCHNEIDER, T. PINKAU, F. PELTZ, AND H. FEUSSNER. 2005. Electromagnetic catheter navigation during bronchoscop : validation of a novel method by conventional fluoroscopy. Chest. 128, 1, 382-7. https://doi.org/10.1378/chest.128.1.382
  19. HELFERTY, J. P., A. J. SHERBONDY, A. P. KIRALY, AND W. E. HIGGINS. 2007. Computer-based system for the virtual-endoscopic guidance of bronchoscopy. Comput Vis Image Underst. 108, 1-2, 171-187. https://doi.org/10.1016/j.cviu.2006.10.010
  20. HORSLEY, V. AND R. CLARK. 1908. The structure and functions of the cerebellum examined by a new method. Brain. 31, 1, 45-124. https://doi.org/10.1093/brain/31.1.45
  21. Image-guided surgery toolkit (IGSTK). Available from: http://www.igstk.org/.
  22. JIN, G., N. BAEK, J. K. HAHN, S. BIELAMOWICZ, R. MITTAL, ET AL. 2009. Image guided medialization laryngoplasty. Comput Animat Virtual Worlds. 20, 1, 67-77. https://doi.org/10.1002/cav.271
  23. JIN, G., S. J. LEE, J. K. HAHN, S. BIELAMOWICZ, R. MITTAL, ET AL. 2006. 3D surface reconstruction and registration for image guided medialization laryngoplasty. Lect Notes Comput Sci. 4291, 1, 761-770. https://doi.org/10.1007/11919476_76
  24. JOSHI, A., D. SCHEINOST, K. VIVES, D. SPENCER, L. STAIB, ET AL. 2008. Novel interaction techniques for neurosurgical planning and stereotactic IEEE Trans on Visualization and Computer Graphics. 14, 6.
  25. KELLY, P. J. 1986. Computer-assisted stereotaxis: new approaches for the management of intracranial intra-axial tumors. Neurology. 36, 4, 535-41. https://doi.org/10.1212/WNL.36.4.535
  26. KIRMIZIBAYRAK, C., M. WAKID, J. K. HAHN, AND S. BIELAMOWICZ. 2010. Interactive visualization for image guided medialization laryngoplasty. Proc. of Computer Graphics International.
  27. KLEIN, J., O. FRIMAN, M. HADWIGER, B. PREIM, F. RITTER, ET AL. 2009. Visual computing for medical diagnosis and treatment. Computers & Graphics 33, 554-565. https://doi.org/10.1016/j.cag.2009.04.006
  28. KRUGER, J. AND R. WESTERMANN. 2003. Acceleration techniques for gpu-based volume rendering. 14th IEEE Visualization: 287-292.
  29. LAVALLEE, S., P. SAUTOT, J. TROCCAZ, P. CINQUIN, AND P. MERLOZ. 1995. Computer-assisted spine surgery: a technique for accurate transpedicular screw fixation using CT data and a 3-D optical localizer. J Image Guid Surg. 1, 1, 65-73. https://doi.org/10.1002/(SICI)1522-712X(1995)1:1<65::AID-IGS10>3.0.CO;2-7
  30. LEE, T. H., J. LEE, H. LEE, H. KYE, Y. G. SHIN, ET AL. 2009. Fast perspective volume ray casting method using GPU-based acceleration techniques for translucency rendering in 3D endoluminal CT colonography. Comput Biol Med. 39, 8, 657-66. https://doi.org/10.1016/j.compbiomed.2009.04.007
  31. LEKSELL, L. 1951. The stereotaxic method and radiosurgery of the brain. Acta Chir Scand. 102, 4, 316-9.
  32. LEVOY, M. 1988. Direct visualization of surfaces from computed tomography data Proceedings of the SPIE. 914, 828-841.
  33. LORENSEN, W. E. AND H. E. CLINE. 1987. Marching cubes: a high resolution 3D surface construction algorithm. Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. 21, 163-169.
  34. MAES, F., D. VANDERMEULEN, AND P. SUETENS. 1999. Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information. Med Image Anal. 3, 4, 373-86 https://doi.org/10.1016/S1361-8415(99)80030-9
  35. MAURER, C. R., J. J. MCCRORY, AND J. M. FITZPATRICK. 1993. Estimation of accuracy in localizing externally attached markers in multimodal volume head images. SPIE Medical imaging: image processing. 1898, 43-54.
  36. MAURER, C. R., JR., J. M. FITZPATRICK, M. Y. WANG, R. L. GALLOWAY, JR., R. J. MACIUNAS, ET AL. 1997. Registration of head volume images using implantable fiducial markers. IEEE Trans Med Imaging. 16, 4, 447-62. https://doi.org/10.1109/42.611354
  37. MONTAGNOLI, A. N., J. B. RUBERT, R. CAPOBIANCO, AND J. C. PEREIRA. 2006. Vocal folds vibrations with a three-dimensional deformable model. The eighth IEEE International Symposium on Multimedia (ISM'06): 674-678.
  38. MUNBODH, R., H. D. TAGARE, Z. CHEN, D. A. JAFFRAY, D. J. MOSELEY, ET AL. 2009. 2D-3D registration for prostate radiation therapy based on a statistical model of transmission images. Med Phys. 36, 10, 4555-68. https://doi.org/10.1118/1.3213531
  39. MUNBODH, R., Z. CHEN, D. A. JAFFRAY, D. J. MOSELEY, J. P. KNISELY, ET AL. 2008. Automated 2D-3D registration of portal images and CT data using line-segment enhancement. Med Phys. 35, 10, 4352-61. https://doi.org/10.1118/1.2975143
  40. ONIK, G. M., D. B. DOWNEY, AND A. FENSTER. 1996. Three-dimensional sonographically monitored cryosurgery in a prostate phantom. J Ultrasound Med. 15, 3, 267-70. https://doi.org/10.7863/jum.1996.15.3.267
  41. PAUL, P., O. FLEIG, AND P. JANNIN. 2005. Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: Methods and performance evaluation. IEEE Transactions on Medical Imaging. 24, 11, 1500-1511. https://doi.org/10.1109/TMI.2005.857029
  42. PERLIN, K. AND D. FOX. 1993. Pad An Alternative Approach to the Computer Interface. The 20th Ann. Conf. on Computer graphics and Interactive Techniques: 57-64.
  43. PETERS, T. M. 2006. Image-guidance for surgical procedures. Phys Med Biol. 51, 14, R505-40. https://doi.org/10.1088/0031-9155/51/14/R01
  44. PRESS, W. H. 1992. Numerical recipes in C: the art of scientific computing. xxvi, 994 p.
  45. RIEDER, C., F. RITTER, M. RASPE, AND H. O. PEITGEN. 2008. Interactive Visualization of Multimodal Volume Data for Neurosurgical Tumor Treatment. Computer Graphics Forum. 27, 3, 1055-1062. https://doi.org/10.1111/j.1467-8659.2008.01242.x
  46. ROBERTS, D. W., J. W. STROHBEHN, J. F. HATCH, W. MURRAY, AND H. KETTENBERGER. 1986. A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. J Neurosurg. 65, 4, 545-9. https://doi.org/10.3171/jns.1986.65.4.0545
  47. SATO, Y., M. NAKAMOTO, Y. TAMAKI, T. SASAMA, I. SAKITA, ET AL. 1998. Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging. 17, 5, 681-93. https://doi.org/10.1109/42.736019
  48. SCHROEDER, W., K. MARTIN, AND B. LORENSEN. 2000. Visualization toolkit: an object-oriented approach to 3D graphics.
  49. SOLOMON, S. B., P. WHITE, JR., C. M. WIENER, J. B. ORENS, AND K. P. WANG. 2000. Threedimensional CT-guided bronchoscopy with a real-time electromagnetic position sensor: a comparison of two image registration methods. Chest. 118, 6, 1783-7. https://doi.org/10.1378/chest.118.6.1783
  50. SPIEGEL, E. A., H. T. WYCIS, M. MARKS, AND A. J. LEE. 1947. Stereotaxic apparatus for operations on the human brain. Science. 106, 2754, 349-50. https://doi.org/10.1126/science.106.2754.349
  51. Statistics on voice, speech, and language. Available from: http://www.nidcd.nih.gov/health/statistics/vsl.asp.
  52. STEFANSIC, J. D., A. J. HERLINE, Y. SHYR, W. C. CHAPMAN, J. M. FITZPATRICK, ET AL. 2000. Registration of physical space to laparoscopic image space for use in minimally invasive hepatic surgery. IEEE Trans Med Imaging. 19, 10, 1012-23. https://doi.org/10.1109/42.887616
  53. STUDHOLME, C., D. L. G. HILL, AND D. J. HAWKES. 1999. An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognition. 32, 1, 71-86. https://doi.org/10.1016/S0031-3203(98)00091-0
  54. TOMAZEVIC, D., B. LIKAR, AND F. PERNUS. 2006. 3-D/2-D Registration by Integrating 2-D Information in 3-D. IEEE Transactions on Medical Imaging. 25, 1.
  55. VIEGA, J., M. J. CONWAY, G. WILLIAMS, AND R. PAUSCH. 1996. 3D Magic Lenses. The 9th Annual ACM Symposium on UI Software and Technology. 1, 1, 51-58.
  56. VIOLA, P. AND P. E. WELLS III. 1997. Alignment by maximization of mutual information. International Journal of Computer Vision. 24, 2, 137-154. https://doi.org/10.1023/A:1007958904918
  57. VIOLA, P., WELLS III, P. E. 1997. Alignment by maximization of mutual information. International Journal of Computer Vision. 24, 2, 137-154. https://doi.org/10.1023/A:1007958904918
  58. WANG, L., Y. ZHAO, K. MUELLER, AND A. KAUFMAN. 2005. The Magic Volume Lens: An Interactive Focus+Context Technique for Volume Rendering. IEEE Visualization 05: 367-374.
  59. WANG, X., Q. ZHANG, Q. HAN, R. YANG, M. CARSWELL, ET AL. 2009. Endoscopic video texture mapping on pre-built 3-D anatomical objects without camera tracking. IEEE Trans Med Imaging. 29, 6, 1213-23.
  60. WELLS, W. M., 3RD, P. VIOLA, H. ATSUMI, S. NAKAJIMA, AND R. KIKINIS. 1996. Multi-modal volume registration by maximization of mutual information. Med Image Anal. 1, 1, 35-51. https://doi.org/10.1016/S1361-8415(01)80004-9
  61. YIM, Y. AND H. HONG. 2008. Correction of segmented lung boundary for inclusion of pleural nodules and pulmonary vessels in chest CT images. Comput Biol Med. 38, 8, 845-57. https://doi.org/10.1016/j.compbiomed.2008.04.012

Cited by

  1. Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery vol.35, pp.1, 2016, https://doi.org/10.1109/TMI.2015.2452907
  2. Fast and Accurate Rigid Registration of 3D CT Images by Combining Feature and Intensity vol.6, pp.1, 2012, https://doi.org/10.5626/JCSE.2012.6.1.1
  3. Surgical planning of Isshiki type I thyroplasty using an open-source Digital Imaging and Communication in Medicine viewer OsiriX vol.134, pp.6, 2014, https://doi.org/10.3109/00016489.2014.880796
  4. Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors vol.11, pp.8, 2016, https://doi.org/10.1007/s11548-015-1331-x