• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제14권3호
• /
• pp.1167-1187
• /
• 2020

With the development of medical imaging technology, image registration has been widely used in the field of disease diagnosis. The registration between different modal images of brain magnetic resonance (MR) is particularly important for the diagnosis of brain diseases. However, previous registration methods don't take advantage of the prior knowledge of bilateral brain symmetry. Moreover, the difference in gray scale information of different modal images increases the difficulty of registration. In this paper, a multimodal medical image registration method based on image segmentation and symmetric self-similarity is proposed. This method uses modal independent self-similar information and modal consistency information to register images. More particularly, we propose two novel symmetric self-similarity constraint operators to constrain the segmented medical images and convert each modal medical image into a unified modal for multimodal image registration. The experimental results show that the proposed method can effectively reduce the error rate of brain MR multimodal medical image registration with rotation and translation transformations (average 0.43mm and 0.60mm) respectively, whose accuracy is better compared to state-of-the-art image registration methods.

• ETRI Journal
• /
• 제39권2호
• /
• pp.173-180
• /
• 2017

Multimodal medical image fusion is the process of retrieving valuable information from medical images. The primary goal of medical image fusion is to combine several images obtained from various sources into a distinct image suitable for improved diagnosis. Complexity in medical images is higher, and many soft computing methods are applied by researchers to process them. Intuitionistic fuzzy sets are more appropriate for medical images because the images have many uncertainties. In this paper, a new method, based on Sugeno's intuitionistic fuzzy set (SIFS), is proposed. First, medical images are converted into Sugeno's intuitionistic fuzzy image (SIFI). An exponential intuitionistic fuzzy entropy calculates the optimum values of membership, non-membership, and hesitation degree functions. Then, the two SIFIs are disintegrated into image blocks for calculating the count of blackness and whiteness of the blocks. Finally, the fused image is rebuilt from the recombination of SIFI image blocks. The efficiency of the use of SIFS in multimodal medical image fusion is demonstrated on several pairs of images and the results are compared with existing studies in recent literature.

• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2021년도 추계학술발표대회
• /
• pp.655-658
• /
• 2021

The purpose of multimodal medical image fusion (MMIF) is to integrate images of different modes with different details into a result image with rich information, which is convenient for doctors to accurately diagnose and treat the diseased tissues of patients. Encouraged by this purpose, this paper proposes a novel method based on a two-scale decomposer and detail preservation model. The first step is to use the two-scale decomposer to decompose the source image into the energy layers and structure layers, which have the characteristic of detail preservation. And then, structure tensor operator and max-abs are combined to fuse the structure layers. The detail preservation model is proposed for the fusion of the energy layers, which greatly improves the image performance. The fused image is achieved by summing up the two fused sub-images obtained by the above fusion rules. Experiments demonstrate that the proposed method has superior performance compared with the state-of-the-art fusion methods.

• 대한임베디드공학회논문지
• /
• 제18권6호
• /
• pp.285-292
• /
• 2023

With the wide spread of smart farms and the advancements in IoT technology, it is easy to obtain additional data in addition to crop images. Consequently, deep learning-based crop disease diagnosis research utilizing multimodal data has become important. This study proposes a crop disease diagnosis method using multimodal supervised contrastive learning by expanding upon the multimodal self-supervised learning. RandAugment method was used to augment crop image and time series of environment data. These augmented data passed through encoder and projection head for each modality, yielding low-dimensional features. Subsequently, the proposed multimodal supervised contrastive loss helped features from the same class get closer while pushing apart those from different classes. Following this, the pretrained model was fine-tuned for crop disease diagnosis. The visualization of t-SNE result and comparative assessments of crop disease diagnosis performance substantiate that the proposed method has superior performance than multimodal self-supervised learning.

• 정보처리학회논문지:컴퓨터 및 통신 시스템
• /
• 제11권6호
• /
• pp.185-192
• /
• 2022

다중모드 의료영상 융합(MMIF)은 각기 다른 특징들을 나타내는 여러 종류의 모드의 이미지를 풍부한 정보가 포함된 하나의 결과 이미지로 통합하는 것이다. 이러한 의료영상 융합은 의사가 환자의 병변을 정확하게 관찰하고 치료하는 것을 도와줄 수 있다. 이러한 목적에 영향을 받아 본 논문에서는 복층 분해기 및 미세구조 보존 모델에 기반한 새로운 방법을 제안한다. 첫째, 복층 분해기를 사용하여 소스 이미지를 미세정보 보존의 특성을 갖는 에너지 층과 구조적 층으로 분해하였다. 둘째, 구조 텐서 연산자와 max-abs를 결합하여 구조적 층을 융합한다. 에너지 층의 융합을 위해 미세구조 보존 모델을 제안하였으며 이미지 융합성능을 크게 향상시킬 수 있었다. 마지막으로, 융합규칙을 통해 형성된 두 개의 융합된 하위 이미지를 합산하여 구축하였다. 실험을 통하여 제안된 방법이 현재까지 최첨단 융합 방법들과 비교하여 우수한 성능을 나타내는 것을 검증하였다.

• 한국지능시스템학회:학술대회논문집
• /
• 한국퍼지및지능시스템학회 1998년도 The Third Asian Fuzzy Systems Symposium
• /
• pp.713-718
• /
• 1998

The paper is part of an investigation by the authors on development of a knowledge based frame work for multimodal medical image in collaboration with the All India Institute of Medical Science, new Delhi. After presenting the key aspects of the Dempster-Shafer Evidence theory we have presented implementation of registration and fusion of T₁and T₂ weighted MR images and CT images of the brain of an Alzheimer's patient for minimising the uncertainty and increasing the reliability for dianostics and therapeutic planning.

• 반도체디스플레이기술학회지
• /
• 제18권2호
• /
• pp.63-66
• /
• 2019

Due to the development of medical imaging technology, different imaging technologies provide a large amount of effective information. However, different imaging method caused the limitations of information integrity by using single type of image. Combining different image together so that doctor can obtain the information from medical image comprehensively. Image registration algorithm based on mutual information has become one of the hotspots in the field of image registration with its high registration accuracy and wide applicability. Because the information theory-based registration technology is not dependent on the gray value difference of the image, and it is very suitable for multimodal medical image registration. However, the method based on mutual information has a robustness problem. The essential reason is that the mutual information itself is not have enough information between the pixel pairs, so that the mutual information is unstable during the registration process. A large number of local extreme values are generated, which finally cause mismatch. In order to overcome the shortages of mutual information registration method, this paper proposes a registration method combined with image spatial structure information and mutual information.

• Investigative Magnetic Resonance Imaging
• /
• 제25권4호
• /
• pp.218-228
• /
• 2021

Due to their high degree of freedom to transfer and acquire light, fiber optics can be used in the presence of strong magnetic fields. Hence, optical sensing and imaging based on fiber optics can be integrated with magnetic resonance imaging (MRI) diagnostic systems to acquire valuable information on biological tissues and organs based on a magnetic field. In this article, we explored the combination of MRI and optical sensing/imaging techniques by classifying them into the following topics: 1) functional near-infrared spectroscopy with functional MRI for brain studies and brain disease diagnoses, 2) integration of fiber-optic molecular imaging and optogenetic stimulation with MRI, and 3) optical therapeutic applications with an MRI guidance system. Through these investigations, we believe that a combination of MRI and optical sensing/imaging techniques can be employed as both research methods for multidisciplinary studies and clinical diagnostic/therapeutic devices.

• Journal of Applied and Pure Mathematics
• /
• 제5권5_6호
• /
• pp.389-406
• /
• 2023

In the pursuit of enhancing image fusion techniques, this research presents a novel approach for fusing multimodal images, specifically infrared (IR) and visible (VIS) images, utilizing a combination of partial differential equations (PDE) and discrete cosine transformation (DCT). The proposed method seeks to leverage the thermal and structural information provided by IR imaging and the fine-grained details offered by VIS imaging create composite images that are superior in quality and informativeness. Through a meticulous fusion process, which involves PDE-guided fusion, DCT component selection, and weighted combination, the methodology aims to strike a balance that optimally preserves essential features and minimizes artifacts. Rigorous evaluations, both objective and subjective, are conducted to validate the effectiveness of the approach. This research contributes to the ongoing advancement of multimodal image fusion, addressing applications in fields like medical imaging, surveillance, and remote sensing, where the marriage of IR and VIS data is of paramount importance.

• 한국의학물리학회:학술대회논문집
• /
• 한국의학물리학회 2002년도 Proceedings
• /
• pp.450-453
• /
• 2002

In radiotherapy treatment planning, it is critical to deliver the radiation dose to tumor and protect surrounding normal tissue. Recent developments in functional imaging and radiotherapy treatment technology have been raising chances to control tumor saving normal tissues. A brain phantom which could be used for image registration technique of CT-MR and CT-SPECT images using surface matching was developed. The brain phantom was specially designed to obtain imaging dataset of CT, MR, and SPECT. The phantom had an external frame with 4 N-shaped pipes filled with acryl rods, Pb rods for CT, MR, and SPECT imaging, respectively. 8 acrylic pipes were inserted into the empty space of the brain phantom to be imaged for geometric evaluation of the matching. For an optimization algorithm of image registration, we used Downhill simplex algorithm suggested as a fast surface matching algorithm. Accuracy of image fusion was assessed by the comparison between the center points of the section of N-shaped bars in the external frame and the inserted pipes of the phantom and minimized cost functions of the optimization algorithm. Technique with partially transparent, mixed images using color on gray was used for visual assessment of the image registration process. The errors of image registration of CT-MR and CT-SPECT were within 2mm and 4mm, respectively. Since these errors were considered within a reasonable margin from the phantom study, the phantom is expected to be used for conventional image registration between multimodal image datasets..