• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.3
• /
• pp.1167-1187
• /
• 2020

With the development of medical imaging technology, image registration has been widely used in the field of disease diagnosis. The registration between different modal images of brain magnetic resonance (MR) is particularly important for the diagnosis of brain diseases. However, previous registration methods don't take advantage of the prior knowledge of bilateral brain symmetry. Moreover, the difference in gray scale information of different modal images increases the difficulty of registration. In this paper, a multimodal medical image registration method based on image segmentation and symmetric self-similarity is proposed. This method uses modal independent self-similar information and modal consistency information to register images. More particularly, we propose two novel symmetric self-similarity constraint operators to constrain the segmented medical images and convert each modal medical image into a unified modal for multimodal image registration. The experimental results show that the proposed method can effectively reduce the error rate of brain MR multimodal medical image registration with rotation and translation transformations (average 0.43mm and 0.60mm) respectively, whose accuracy is better compared to state-of-the-art image registration methods.

• ETRI Journal
• /
• v.39 no.2
• /
• pp.173-180
• /
• 2017

Multimodal medical image fusion is the process of retrieving valuable information from medical images. The primary goal of medical image fusion is to combine several images obtained from various sources into a distinct image suitable for improved diagnosis. Complexity in medical images is higher, and many soft computing methods are applied by researchers to process them. Intuitionistic fuzzy sets are more appropriate for medical images because the images have many uncertainties. In this paper, a new method, based on Sugeno's intuitionistic fuzzy set (SIFS), is proposed. First, medical images are converted into Sugeno's intuitionistic fuzzy image (SIFI). An exponential intuitionistic fuzzy entropy calculates the optimum values of membership, non-membership, and hesitation degree functions. Then, the two SIFIs are disintegrated into image blocks for calculating the count of blackness and whiteness of the blocks. Finally, the fused image is rebuilt from the recombination of SIFI image blocks. The efficiency of the use of SIFS in multimodal medical image fusion is demonstrated on several pairs of images and the results are compared with existing studies in recent literature.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2021.11a
• /
• pp.655-658
• /
• 2021

The purpose of multimodal medical image fusion (MMIF) is to integrate images of different modes with different details into a result image with rich information, which is convenient for doctors to accurately diagnose and treat the diseased tissues of patients. Encouraged by this purpose, this paper proposes a novel method based on a two-scale decomposer and detail preservation model. The first step is to use the two-scale decomposer to decompose the source image into the energy layers and structure layers, which have the characteristic of detail preservation. And then, structure tensor operator and max-abs are combined to fuse the structure layers. The detail preservation model is proposed for the fusion of the energy layers, which greatly improves the image performance. The fused image is achieved by summing up the two fused sub-images obtained by the above fusion rules. Experiments demonstrate that the proposed method has superior performance compared with the state-of-the-art fusion methods.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.18 no.6
• /
• pp.285-292
• /
• 2023

With the wide spread of smart farms and the advancements in IoT technology, it is easy to obtain additional data in addition to crop images. Consequently, deep learning-based crop disease diagnosis research utilizing multimodal data has become important. This study proposes a crop disease diagnosis method using multimodal supervised contrastive learning by expanding upon the multimodal self-supervised learning. RandAugment method was used to augment crop image and time series of environment data. These augmented data passed through encoder and projection head for each modality, yielding low-dimensional features. Subsequently, the proposed multimodal supervised contrastive loss helped features from the same class get closer while pushing apart those from different classes. Following this, the pretrained model was fine-tuned for crop disease diagnosis. The visualization of t-SNE result and comparative assessments of crop disease diagnosis performance substantiate that the proposed method has superior performance than multimodal self-supervised learning.

• KIPS Transactions on Computer and Communication Systems
• /
• v.11 no.6
• /
• pp.185-192
• /
• 2022

Multimodal medical image fusion (MMIF) fuses two images containing different structural details generated in two different modes into a comprehensive image with saturated information, which can help doctors improve the accuracy of observation and treatment of patients' diseases. Therefore, a method based on double-layer decomposer and fine structure preservation model is proposed. Firstly, a double-layer decomposer is applied to decompose the source images into the energy layers and structure layers, which can preserve details well. Secondly, The structure layer is processed by combining the structure tensor operator (STO) and max-abs. As for the energy layers, a fine structure preservation model is proposed to guide the fusion, further improving the image quality. Finally, the fused image can be achieved by performing an addition operation between the two sub-fused images formed through the fusion rules. Experiments manifest that our method has excellent performance compared with several typical fusion methods.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 1998.06a
• /
• pp.713-718
• /
• 1998

The paper is part of an investigation by the authors on development of a knowledge based frame work for multimodal medical image in collaboration with the All India Institute of Medical Science, new Delhi. After presenting the key aspects of the Dempster-Shafer Evidence theory we have presented implementation of registration and fusion of T₁and T₂ weighted MR images and CT images of the brain of an Alzheimer's patient for minimising the uncertainty and increasing the reliability for dianostics and therapeutic planning.

• Journal of the Semiconductor & Display Technology
• /
• v.18 no.2
• /
• pp.63-66
• /
• 2019

Due to the development of medical imaging technology, different imaging technologies provide a large amount of effective information. However, different imaging method caused the limitations of information integrity by using single type of image. Combining different image together so that doctor can obtain the information from medical image comprehensively. Image registration algorithm based on mutual information has become one of the hotspots in the field of image registration with its high registration accuracy and wide applicability. Because the information theory-based registration technology is not dependent on the gray value difference of the image, and it is very suitable for multimodal medical image registration. However, the method based on mutual information has a robustness problem. The essential reason is that the mutual information itself is not have enough information between the pixel pairs, so that the mutual information is unstable during the registration process. A large number of local extreme values are generated, which finally cause mismatch. In order to overcome the shortages of mutual information registration method, this paper proposes a registration method combined with image spatial structure information and mutual information.

• Investigative Magnetic Resonance Imaging
• /
• v.25 no.4
• /
• pp.218-228
• /
• 2021

Due to their high degree of freedom to transfer and acquire light, fiber optics can be used in the presence of strong magnetic fields. Hence, optical sensing and imaging based on fiber optics can be integrated with magnetic resonance imaging (MRI) diagnostic systems to acquire valuable information on biological tissues and organs based on a magnetic field. In this article, we explored the combination of MRI and optical sensing/imaging techniques by classifying them into the following topics: 1) functional near-infrared spectroscopy with functional MRI for brain studies and brain disease diagnoses, 2) integration of fiber-optic molecular imaging and optogenetic stimulation with MRI, and 3) optical therapeutic applications with an MRI guidance system. Through these investigations, we believe that a combination of MRI and optical sensing/imaging techniques can be employed as both research methods for multidisciplinary studies and clinical diagnostic/therapeutic devices.

• Journal of Applied and Pure Mathematics
• /
• v.5 no.5_6
• /
• pp.389-406
• /
• 2023

In the pursuit of enhancing image fusion techniques, this research presents a novel approach for fusing multimodal images, specifically infrared (IR) and visible (VIS) images, utilizing a combination of partial differential equations (PDE) and discrete cosine transformation (DCT). The proposed method seeks to leverage the thermal and structural information provided by IR imaging and the fine-grained details offered by VIS imaging create composite images that are superior in quality and informativeness. Through a meticulous fusion process, which involves PDE-guided fusion, DCT component selection, and weighted combination, the methodology aims to strike a balance that optimally preserves essential features and minimizes artifacts. Rigorous evaluations, both objective and subjective, are conducted to validate the effectiveness of the approach. This research contributes to the ongoing advancement of multimodal image fusion, addressing applications in fields like medical imaging, surveillance, and remote sensing, where the marriage of IR and VIS data is of paramount importance.

• Proceedings of the Korean Society of Medical Physics Conference
• /
• 2002.09a
• /
• pp.450-453
• /
• 2002

In radiotherapy treatment planning, it is critical to deliver the radiation dose to tumor and protect surrounding normal tissue. Recent developments in functional imaging and radiotherapy treatment technology have been raising chances to control tumor saving normal tissues. A brain phantom which could be used for image registration technique of CT-MR and CT-SPECT images using surface matching was developed. The brain phantom was specially designed to obtain imaging dataset of CT, MR, and SPECT. The phantom had an external frame with 4 N-shaped pipes filled with acryl rods, Pb rods for CT, MR, and SPECT imaging, respectively. 8 acrylic pipes were inserted into the empty space of the brain phantom to be imaged for geometric evaluation of the matching. For an optimization algorithm of image registration, we used Downhill simplex algorithm suggested as a fast surface matching algorithm. Accuracy of image fusion was assessed by the comparison between the center points of the section of N-shaped bars in the external frame and the inserted pipes of the phantom and minimized cost functions of the optimization algorithm. Technique with partially transparent, mixed images using color on gray was used for visual assessment of the image registration process. The errors of image registration of CT-MR and CT-SPECT were within 2mm and 4mm, respectively. Since these errors were considered within a reasonable margin from the phantom study, the phantom is expected to be used for conventional image registration between multimodal image datasets..