• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2009.05a
• /
• pp.599-602
• /
• 2009

This paper presents a comparative analysis of five different fusion methods. The five different methods to merge multispectral images and panchromatic image are IHS, FIHS, PCA, BT and WT methods. The comparative analysis based on visual analysis and quantitative analysis are performed using the merged results. From the results the FIHS method provide good result, BT, PCA, IHS and WT method show the next order.

• Journal of Multimedia Information System
• /
• v.9 no.1
• /
• pp.1-10
• /
• 2022

China is a big country in animal fur industry. The total production and consumption of fur are increasing year by year. However, the recognition of fur in the fur production process still mainly relies on the visual identification of skilled workers, and the stability and consistency of products cannot be guaranteed. In response to this problem, this paper proposes a feature fusion-based animal fur recognition network on the basis of typical convolutional neural network structure, relying on rapidly developing deep learning techniques. This network superimposes texture feature - the most prominent feature of fur image - into the channel dimension of input image. The output feature map of the first layer convolution is inverted to obtain the inverted feature map and concat it into the original output feature map, then Leaky ReLU is used for activation, which makes full use of the texture information of fur image and the inverted feature information. Experimental results show that the algorithm improves the recognition accuracy by 9.08% on Fur_Recognition dataset and 6.41% on CIFAR-10 dataset. The algorithm in this paper can change the current situation that fur recognition relies on manual visual method to classify, and can lay foundation for improving the efficiency of fur production technology.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.3
• /
• pp.877-893
• /
• 2022

With the development of deep learning, face inpainting has been significantly enhanced in the past few years. Although image inpainting framework integrated with generative adversarial network or attention mechanism enhanced the semantic understanding among facial components, the issues of reconstruction on corrupted regions are still worthy to explore, such as blurred edge structure, excessive smoothness, unreasonable semantic understanding and visual artifacts, etc. To address these issues, we propose a Learnable Structure Knowledge of Fusion Network (LSK-FNet), which learns a prior knowledge by edge generation network for image inpainting. The architecture involves two steps: Firstly, structure information obtained by edge generation network is used as the prior knowledge for face inpainting network. Secondly, both the generated prior knowledge and the incomplete image are fed into the face inpainting network together to get the fusion information. To improve the accuracy of inpainting, both of gated convolution and region normalization are applied in our proposed model. We evaluate our LSK-FNet qualitatively and quantitatively on the CelebA-HQ dataset. The experimental results demonstrate that the edge structure and details of facial images can be improved by using LSK-FNet. Our model surpasses the compared models on L1, PSNR and SSIM metrics. When the masked region is less than 20%, L1 loss reduce by more than 4.3%.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.10
• /
• pp.3390-3405
• /
• 2022

Long Short-Term Memory (LSTM) combined with attention mechanism is extensively used to generate semantic sentences of images in image captioning models. However, features of salient regions and spatial information are not utilized sufficiently in most related works. Meanwhile, the LSTM also suffers from the problem of underutilized information in a single time step. In the paper, two innovative approaches are proposed to solve these problems. First, the Synergy-Gated Attention (SGA) method is proposed, which can process the spatial features and the salient region features of given images simultaneously. SGA establishes a gated mechanism through the global features to guide the interaction of information between these two features. Then, the Recurrent Fusion LSTM (RF-LSTM) mechanism is proposed, which can predict the next hidden vectors in one time step and improve linguistic coherence by fusing future information. Experimental results on the benchmark dataset of MSCOCO show that compared with the state-of-the-art methods, the proposed method can improve the performance of image captioning model, and achieve competitive performance on multiple evaluation indicators.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.11a
• /
• pp.637-640
• /
• 2023

This paper presents multimodal image fusion with human pose for detecting abnormal human behaviors in low illumination conditions. Detecting human behaviors in low illumination conditions is challenging due to its limited visibility of the objects of interest in the scene. Multimodal image fusion simultaneously combines visual information in the visible spectrum and thermal radiation information in the long-wave infrared spectrum. We propose an abnormal event detection scheme based on the multimodal fused image and the human poses using the keypoints to characterize the action of the human body. Our method assumes that human behaviors are well correlated to body keypoints such as shoulders, elbows, wrists, hips. In detail, we extracted the human keypoint coordinates from human targets in multimodal fused videos. The coordinate values are used as inputs to train a multilayer perceptron network to classify human behaviors as normal or abnormal. Our experiment demonstrates a significant result on multimodal imaging dataset. The proposed model can capture the complex distribution pattern for both normal and abnormal behaviors.

• Journal of Institute of Control, Robotics and Systems
• /
• v.18 no.1
• /
• pp.54-61
• /
• 2012

In this paper, the pose estimation method for the satellite GTB (Ground Test Bed) using vision/MEMS IMU (Inertial Measurement Unit) integrated system is presented. The GTB for verifying a satellite system on the ground is similar to the mobile robot having thrusters and a reaction wheel as actuators and floating on the floor by compressed air. The EKF (Extended Kalman Filter) is also used for fusion of MEMS IMU and vision system that consists of a single camera and infrared LEDs that is ceiling landmarks. The fusion filter generally utilizes the position of feature points from the image as measurement. However, this method can cause position error due to the bias of MEMS IMU when the camera image is not obtained if the bias is not properly estimated through the filter. Therefore, it is proposed that the fusion method which uses the position of feature points and the velocity of the camera determined from optical flow of feature points. It is verified by experiments that the performance of the proposed method is robust to the bias of IMU compared to the method that uses only the position of feature points.

• International Journal of Contents
• /
• v.18 no.3
• /
• pp.11-20
• /
• 2022

Human Emotion Recognition is an exciting topic that has been attracting many researchers for a lengthy time. In recent years, there has been an increasing interest in exploiting contextual information on emotion recognition. Some previous explorations in psychology show that emotional perception is impacted by facial expressions, as well as contextual information from the scene, such as human activities, interactions, and body poses. Those explorations initialize a trend in computer vision in exploring the critical role of contexts, by considering them as modalities to infer predicted emotion along with facial expressions. However, the contextual information has not been fully exploited. The scene emotion created by the surrounding environment, can shape how people perceive emotion. Besides, additive fusion in multimodal training fashion is not practical, because the contributions of each modality are not equal to the final prediction. The purpose of this paper was to contribute to this growing area of research, by exploring the effectiveness of the emotional scene gist in the input image, to infer the emotional state of the primary target. The emotional scene gist includes emotion, emotional feelings, and actions or events that directly trigger emotional reactions in the input image. We also present an attention-based fusion network, to combine multimodal features based on their impacts on the target emotional state. We demonstrate the effectiveness of the method, through a significant improvement on the EMOTIC dataset.

• Journal of Korea Multimedia Society
• /
• v.15 no.3
• /
• pp.291-298
• /
• 2012

In automatic target recognition(ATR) systems, target extraction techniques are very important because ATR performance depends on segmentation result. So, this paper proposes a multi-sensor image fusion method based on adaptive weights. To incorporate the FLIR image and CCD image, we used information such as the bi-modality, distance and texture. A weight of the FLIR image is derived from the bi-modality and distance measure. For the weight of CCD image, the information that the target's texture is more uniform than the background region is used. The proposed algorithm is applied to many images and its performance is compared with the segmentation result using the single image. Experimental results show that the proposed method has the accurate extraction performance.

• Journal of Korea Multimedia Society
• /
• v.21 no.7
• /
• pp.764-771
• /
• 2018

In this paper, we propose a high-quality image acquisition algorithm using only a single image, which the high-quality image is normally referred as HDR ones. In order to acquire the HDR image, conventional methods need many images having different exposure values at the same scene and should delicately adjust the color values for a bit-expansion or an exposure fusion. Thus, they require considerable calculations and complex structures. Therefore, the proposed algorithm suggests a completely new approach using one image for the high-quality image acquisition by applying statistical difference and histogram manipulation, or histogram specification, techniques. The techniques could control the pixel's statistical distribution of the input image into the desired one through the local and the global modifications, respectively. As the result, the quality of the proposed algorithm is better than those of conventional methods implemented in commercial image editing softwares.

• Journal of Korean Society for Geospatial Information Science
• /
• v.21 no.2
• /
• pp.63-69
• /
• 2013

Change detection technique is essential to various applications of Very High-Resolution(VHR) satellite imagery and land monitoring. However, change detection accuracy of VHR satellite imagery can be decreased due to various geometrical dissimilarity. In this paper, the existing fusion evaluation indexes were revised and applied to improve VHR imagery based change detection accuracy between multi-temporal images. In addition, appropriate change detection methodology of VHR images are proposed through comparison of general change detection algorithm with cross-sharpened image based change detection algorithm. For these purpose, ERGAS, UIQI and SAM, which were representative fusion evaluation index, were applied to unsupervised change detection, and then, these were compared with CVA based change detection result. Methodologies for minimizing the geometrical error of change detection algorithm are analyzed through evaluation of change detection accuracy corresponding to image fusion method, also. The experimental results are shown that change detection accuracy based on ERGAS index by using cross-sharpened images is higher than these based on other estimation index by using general fused image.