• The KIPS Transactions:PartB
• /
• v.12B no.3 s.99
• /
• pp.247-256
• /
• 2005

One of the classic research problems in computer vision is that of stereo, i.e., the reconstruction of three dimensional shape from two or more images. This paper deals with the problem of extracting depth information of non-rigid dynamic 3D scenes from general 2D video sequences taken by monocular camera, such as movies, documentaries, and dramas. Depth of the blocks are extracted from the resultant block motions throughout following two steps: (i) calculation of global parameters concerned with camera translations and focal length using the locations of blocks and their motions, (ii) calculation of each block depth relative to average image depth using the global parameters and the location of the block and its motion, Both singular and non-singular cases are experimented with various video sequences. The resultant relative depths and ego-motion object shapes are virtually identical to human vision.

• Journal of Information Processing Systems
• /
• v.14 no.5
• /
• pp.1176-1194
• /
• 2018

The object classification using the images' contents is a big challenge in computer vision. The superpixels' information can be used to detect and classify objects in an image based on locations. In this paper, we proposed a methodology to detect and classify the image's pixels' locations using enhanced bag of words (BOW). It calculates the initial positions of each segment of an image using superpixels and then ranks it according to the region score. Further, this information is used to extract local and global features using a hybrid approach of Scale Invariant Feature Transform (SIFT) and GIST, respectively. To enhance the classification accuracy, the feature fusion technique is applied to combine local and global features vectors through weight parameter. The support vector machine classifier is a supervised algorithm is used for classification in order to analyze the proposed methodology. The Pascal Visual Object Classes Challenge 2007 (VOC2007) dataset is used in the experiment to test the results. The proposed approach gave the results in high-quality class for independent objects' locations with a mean average best overlap (MABO) of 0.833 at 1,500 locations resulting in a better detection rate. The results are compared with previous approaches and it is proved that it gave the better classification results for the non-rigid classes.

• Journal of Information Processing Systems
• /
• v.8 no.1
• /
• pp.55-66
• /
• 2012

This paper presents a dual modeling method that simulates the graphic and haptic behavior of a volumetric deformable object and conveys the behavior to a human operator. Although conventional modeling methods (a mass-spring model and a finite element method) are suitable for the real-time computation of an object's deformation, it is not easy to compute the haptic behavior of a volumetric deformable object with the conventional modeling method in real-time (within a 1kHz) due to a computational burden. Previously, we proposed a fast volume haptic rendering method based on the S-chain model that can compute the deformation of a volumetric non-rigid object and its haptic feedback in real-time. When the S-chain model represents the object, the haptic feeling is realistic, whereas the graphical results of the deformed shape look linear. In order to improve the graphic and haptic behavior at the same time, we propose a dual modeling framework in which a volumetric haptic model and a surface graphical model coexist. In order to inspect the graphic and haptic behavior of objects represented by the proposed dual model, experiments are conducted with volumetric objects consisting of about 20,000 nodes at a haptic update rate of 1000Hz and a graphic update rate of 30Hz. We also conduct human factor studies to show that the haptic and graphic behavior from our model is realistic. Our experiments verify that our model provides a realistic haptic and graphic feeling to users in real-time.

• Journal of Broadcast Engineering
• /
• v.27 no.2
• /
• pp.207-215
• /
• 2022

The latest volumetric technology's high geometrical accuracy and realism ensure a high degree of correspondence between the real object and the captured 3D model. Nevertheless, since the 3D model obtained in this way constitutes a sequence as a completely independent 3D model between frames, the consistency of the model surface structure (geometry) is not guaranteed for every frame, and the density of vertices is very high. It can be seen that the interconnection node (Edge) becomes very complicated. 3D models created using this technology are inherently different from models created in movie or video game production pipelines and are not suitable for direct use in applications such as real-time rendering, animation and simulation, and compression. In contrast, our method achieves consistency in the quality of the volumetric 3D model sequence by linking re-meshing, which ensures high consistency of the 3D model surface structure between frames and the gradual deformation and texture transfer through correspondence and matching of non-rigid surfaces. And It maintains the consistency of volumetric 3D model sequence quality and provides post-processing automation.

• Proceedings of the Korean Information Science Society Conference
• /
• 2007.10c
• /
• pp.409-414
• /
• 2007

Active Appearance Model is a well-known model that can represent a non-rigid object effectively. However, since it uses the fixed appearance model, the fitting results are often unsatisfactory when the imaging condition of the target image is different from that of training images. To alleviate this problem, incremental AAM was proposed which updates its appearance bases in an on-line manner. However, it cannot deal with the sudden changes of illumination. To overcome this, we propose a novel scheme to update the appearance bases. When a new person appears in the input image, we synthesize illuminated images of that person and update the appearance bases of AAM using it. Since we update the appearance bases using synthesized illuminated images in advance, the AAM can fit their model to a target image well when the illumination changes drastically. The experimental results show that our proposed algorithm improves the fitting performance over both the incremental AAM and the original AAM.

• The KIPS Transactions:PartB
• /
• v.17B no.5
• /
• pp.355-362
• /
• 2010

Face tracking is to estimate the motion of a non-rigid face together with a rigid head in 3D, and plays important roles in higher levels such as face/facial expression/emotion recognition. In this paper, we propose an AAM-based face tracking algorithm. AAM has been widely used to segment and track deformable objects, but there are still many difficulties. Particularly, it often tends to diverge or converge into local minima when a target object is self-occluded, partially or completely occluded. To address this problem, we utilize the scale invariant feature transform (SIFT). SIFT is an effective method for self and partial occlusion because it is able to find correspondence between feature points under partial loss. And it enables an AAM to continue to track without re-initialization in complete occlusions thanks to the good performance of global matching. We also register and use the SIFT features extracted from multi-view face images during tracking to effectively track a face across large pose changes. Our proposed algorithm is validated by comparing other algorithms under the above 3 kinds of occlusions.

• Journal of Institute of Control, Robotics and Systems
• /
• v.16 no.10
• /
• pp.957-962
• /
• 2010

If we want to recognize the human's emotion via the facial image, first of all, we need to extract the emotional features from the facial image by using a feature extraction algorithm. And we need to classify the emotional status by using pattern classification method. The AAM (Active Appearance Model) is a well-known method that can represent a non-rigid object, such as face, facial expression. The Bayesian Network is a probability based classifier that can represent the probabilistic relationships between a set of facial features. In this paper, our approach to facial feature extraction lies in the proposed feature extraction method based on combining AAM with FACS (Facial Action Coding System) for automatically modeling and extracting the facial emotional features. To recognize the facial emotion, we use the DBNs (Dynamic Bayesian Networks) for modeling and understanding the temporal phases of facial expressions in image sequences. The result of emotion recognition can be used to rehabilitate based on biofeedback for emotional disabled.

• Journal of Internet Computing and Services
• /
• v.10 no.4
• /
• pp.25-33
• /
• 2009

In the paper, we proposed a new system to retrieve 3D wrinkle data based on stereo images. Usually, 3D reconstruction based on stereo images or video is very popular and it is the research focus, which has been applied for culture heritage, building and other scene. The target is object measurement, the scene depth calculation and 3D data obtained. There are several challenges in our research. First, it is hard to take the full information wrinkle images by cameras because of light influence, skin with non-rigid object and camera performance. We design a particular computer vision system to take winkle images with a long length camera lens. Second, it is difficult to get the dense stereo data because of the hard skin texture image segmentation and corner detection. We focus on semi-dense stereo matching algorithm for the wrinkle depth. Compared with the 3D scanner, our system is much cheaper and compared with the physical modeling based method, our system is more flexible with high performance.

• Journal of KIISE:Software and Applications
• /
• v.29 no.5
• /
• pp.319-325
• /
• 2002

Tracking moving objects is one of the most important techniques in motion analysis and understanding, and it has many difficult problems to solve. Especially, estimating and identifying moving objects, when the background and moving objects vary dynamically, are very difficult. It is possible under such a complex environment that targets may disappear totally or partially due to occlusion by other objects. The Kalman filter has been used to estimate motion information and use the information in predicting the appearance of targets in succeeding frames. In this paper, we propose another version of the Kalman filter, to be called structured Kalman filter, which can successfully work its role of estimating motion information under a deteriorating condition such as occlusion. Experimental results show that the suggested approach is very effective in estimating and tracking non-rigid moving objects reliably.

• The KIPS Transactions:PartB
• /
• v.16B no.5
• /
• pp.385-394
• /
• 2009

Face tracking and recognition are difficult problems because the face is a non-rigid object. The main reasons for the failure to track and recognize the faces are the changes of a face pose and environmental illumination. To solve these problems, we propose a nonlinear manifold framework for the face pose and the face illumination normalization processing. Specifically, to track and recognize a face on the video that has various pose variations, we approximate a face pose density to single Gaussian density by PCA(Principle Component Analysis) using images sampled from training video sequences and then construct the GMM(Gaussian Mixture Model) for each person. To solve the illumination problem for the face tracking and recognition, we decompose the face images into the reflectance and the illuminance using the SSR(Single Scale Retinex) model. To obtain the normalized reflectance, the reflectance is rescaled by histogram equalization on the defined range. We newly approximate the illuminance by the trained manifold since the illuminance has almost variations by illumination. By combining these two features into our manifold framework, we derived the efficient face tracking and recognition results on indoor and outdoor video. To improve the video based tracking results, we update the weights of each face pose density at each frame by the tracking result at the previous frame using EM algorithm. Our experimental results show that our method is more efficient than other methods.