Journal of the Institute of Convergence Signal Processing (융합신호처리학회논문지)

The Korea Institute of Convergence Signal Processing (한국융합신호처리학회)
권호 표지
DOI QR코드

DOI QR Code

The Novel Label Free Staining Algorithm in Digital Pathology

차세대 디지털 병리를 위한 Label Free 디지털염색 알고리즘 비교 연구

  • Seok-Min Hwang (Dept. of Biomedical Engineering, School of Medicine, Keimyung University) ;
  • Yeun-Woo Jung (Dept. of Biomedical Engineering, School of Medicine, Keimyung University) ;
  • Dong-Bum Kim (Dept. of Biomedical Engineering, School of Medicine, Keimyung University) ;
  • Seung Ah Lee (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Nam Hoon Cho (Dept. of Pathology, College of Medicine, Yonsei University) ;
  • Jong-Ha Lee (Dept. of Biomedical Engineering, School of Medicine, Keimyung University)
  • 황석민 (계명대학교 공과대학 의용공학과) ;
  • 정연우 (계명대학교 공과대학 의용공학과) ;
  • 김동범 (계명대학교 공과대학 의용공학과) ;
  • 이승아 (연세대학교 공과대학 전기전자공학부) ;
  • 조남훈 (연세대학교 의과대학 병리학교실) ;
  • 이종하 (계명대학교 공과대학 의용공학과)
  • Received : 2023.02.06
  • Accepted : 2023.03.30
  • Published : 2023.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

To distinguish cancer cells from normal cells, H&E (Hematoxylin & Eosin) staining is required. Pathological staining requires a lot of money and time. Recently, a digital dyeing method has been introduced to reduce such cost and time. In this paper, we propose a novel digital pathology algorithms. The first algorithm is the Pair method. This method learns the dyed phase image and unstained amplitude image taken by FPM (Fourier Ptychographic Microscopy) and converts it into a dyed amplitude image. The second algorithm is the unpair method. This method use the stained and unstained fluorescence microscopic images for modeling. In this study, digital staining was performed using a generative adversarial network (GAN). From the experimental results, we noticed that both the pair and unpair algorithms shows the excellent performance.

암세포와 정상세포를 구분하기 위해서는 H&E(Hematoxylin&Eosin) 염색이 필요하다. 병리 염색은 많은 비용과 시간이 필요하다. 최근 이러한 비용과 시간을 줄이고자 디지털 염색 방법이 소개되고 있다. 본 연구에서는 병리 H&E 염색의 디지털 변환 방법에 대한 새로운 알고리즘을 제안한다. 첫 번째 알고리즘은 Pair방법이다. 본 방법은 FPM(Fourier Ptychographic Microscopy)으로 촬영된 염색된 Phase 영상과 염색되지 않은 Amplitude 영상을 학습하여 염색된 Amplitude 영상으로 변환한다. 두 번째 알고리즘은 Unpair방법이다. 본 방법은 염색된 형광현미경 영상과 염색되지 않은 형광현미경 영상을 학습하여 모델링하여 디지털 염색을 수행한다. 본 연구에서는 GAN(generative Adversarial Network)를 활용하여 디지털 염색을 진행하였다. 연구 결과 Pair방법과 Unpair방법 모두 우수한 성능의 디지털 염색 결과를 확보하였다.

Keywords

Acknowledgement

This research is financial supported by "The digital pathology-based AI analysis solution development project" through the Ministry of Health and Welfare, Republic of Korea (HI21C0977) and Korea Basic Science Institute (National research Facilities and Equipment Center) grant funded by the Ministry of Education.(grant No.2020R1A6C101B189).

References

  1. D. Mayerich, M. J. Walsh, A. Kadjacsy-Balla, P. S. Ray, S. M. Hewitt, and R. Bhargava, "Stain-less staining for computed histopathology,"  Technology, vol. 3, no. 01, pp. 27-31, 2015 https://doi.org/10.1142/S2339547815200010
  2. F. Yi, J. Huang, L. Yang, Y. Xie, and G. Xiao, "Automatic extraction of cell nuclei from H&E-stained histopathological images," Journal of Medical Imaging, vol. 4, no. 2, p. 027502, 2017.
  3. K. de Haan et al., "Deep learning-based transformation of H&E stained tissues into special stains," Nature communications, vol. 12, no. 1, pp. 1-13, 2021. https://doi.org/10.1038/s41467-020-20314-w
  4. G. Avwioro, "Histochemical uses of haematoxylina review," Jpcs, vol. 1, no. 5, pp. 24-34, 2011.
  5. P. Marshall and R. Horobin, "The oxidation products of haematoxylin and their role in biological staining," The Histochemical Journal, vol. 4, no. 6, pp. 493-503, 1972. https://doi.org/10.1007/BF01011129
  6. G. Clark, "Comparison of various oxidants for alum hematoxylin," Stain Technology, vol. 49, no. 4, pp. 225-227, 1974 https://doi.org/10.3109/10520297409116982
  7. P. M. Kulkarni et al., "Deep Learning Based on Standard H&E Images of Primary Melanoma Tumors Identifies Patients at Risk for Visceral Recurrence and DeathDeep Learning-based Prognostic Biomarker for Melanoma," Clinical Cancer Research, vol. 26, no. 5, pp. 1126-1134, 2020. https://doi.org/10.1158/1078-0432.CCR-19-1495
  8. K. Kim and J. Guck, "The relative densities of cytoplasm and nuclear compartments are robust against strong perturbation," Biophysical Journal, vol. 119, no. 10, pp. 1946-1957, 2020. https://doi.org/10.1016/j.bpj.2020.08.044
  9. M. Mir, B. Bhaduri, R. Wang, R. Zhu, and G. Popescu, "Quantitative phase imaging," Progress in optics, vol. 57, no. 133-37, p. 217, 2012
  10. K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
  11. F. Zernike, "Phase contrast, a new method for the microscopic observation of transparent objects part II," Physica, vol. 9, no. 10, pp. 974-986, 1942.vol. 3, no. 01, pp. 27-31, 2015 https://doi.org/10.1016/S0031-8914(42)80079-8
  12. J. Hosang, R. Benenson, P. Dollar, and B. Schiele, "What makes for effective detection proposals?," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 4, pp. 814-830, 2015. https://doi.org/10.1109/TPAMI.2015.2465908
  13. G. Zheng, R. Horstmeyer, and C. Yang, "Wide-field, high-resolution Fourier ptychographic microscopy," Nature photonics, vol. 7, no. 9, pp. 739-745, 2013. https://doi.org/10.1038/nphoton.2013.187
  14. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, "Light field microscopy," in ACM SIGGRAPH 2006 Papers, 2006, pp. 924-934.
  15. M. Levoy, Z. Zhang, and I. McDowall, "Recording and controlling the 4D light field in a microscope using microlens arrays," Journal of microscopy, vol. 235, no. 2, pp. 144-162, 2009 https://doi.org/10.1111/j.1365-2818.2009.03195.x
  16. B. R. Masters, "Quantitative phase imaging of cells and tissue," Journal of Biomedical Optics, vol. 17, no. 2,sp. 029901, 2012.