• Journal of Broadcast Engineering
• /
• v.27 no.5
• /
• pp.619-628
• /
• 2022

Although face recognition and face generation have been growing in popularity, the privacy issues of using facial images in the wild have been a concurrent topic. In this paper, we propose a face editing network that can reduce privacy issues by generating face images with various properties from a small number of real face images and facial mask information. Unlike the existing methods of learning face attributes using a lot of real face images, the proposed method generates new facial images using a facial segmentation mask and texture images from five parts as styles. The images are then trained with our network to learn the styles and locations of each reference image. Once the proposed framework is trained, we can generate various face images using only a small number of real face images and segmentation information. In our extensive experiments, we show that the proposed method can not only generate new faces, but also localize facial attribute editing, despite using very few real face images.

• Journal of Broadcast Engineering
• /
• v.27 no.4
• /
• pp.519-526
• /
• 2022

Identifying and searching for characters appearing in scenes during multimedia video editing is an arduous and time-consuming process. Applying artificial intelligence to labor-intensive media editing tasks can greatly reduce media production time, improving the creative process efficiency. In this paper, a method is proposed which combines existing artificial intelligence based techniques to automate character recognition and search tasks for video editing. Object detection, face detection, and pose estimation are used for character localization and face recognition and color space analysis are used to extract unique representation information.

• Journal of Intelligence and Information Systems
• /
• v.29 no.4
• /
• pp.15-30
• /
• 2023

Recently, various services using artificial intelligence(AI) are emerging in the media field as well However, most of the video editing, which involves finding an editing point and attaching the video, is carried out in a passive manner, requiring a lot of time and human resources. Therefore, this study proposes a methodology that can detect the edit points of video according to whether person in video are spoken by using Video Swin Transformer. First, facial keypoints are detected through face alignment. To this end, the proposed structure first detects facial keypoints through face alignment. Through this process, the temporal and spatial changes of the face are reflected from the input video data. And, through the Video Swin Transformer-based model proposed in this study, the behavior of the person in the video is classified. Specifically, after combining the feature map generated through Video Swin Transformer from video data and the facial keypoints detected through Face Alignment, utterance is classified through convolution layers. In conclusion, the performance of the image editing point detection model using facial keypoints proposed in this paper improved from 87.46% to 89.17% compared to the model without facial keypoints.

• Journal of the Korea Society of Computer and Information
• /
• v.25 no.5
• /
• pp.1-9
• /
• 2020

In this paper, we propose a model that performs more natural facial attribute editing by utilizing mask information in the hair and hat region. STGAN, one of state-of-the-art research of facial attribute editing, has shown results of naturally editing multiple facial attributes. However, editing hair-related attributes can produce unnatural results. The key idea of the proposed method is to additionally utilize information on the face regions that was lacking in the existing model. To do this, we apply three ideas. First, hair information is supplemented by adding hair ratio attributes through masks. Second, unnecessary changes in the image are suppressed by adding cycle consistency loss. Third, a hat segmentation network is added to prevent hat region distortion. Through qualitative evaluation, the effectiveness of the proposed method is evaluated and analyzed. The method proposed in the experimental results generated hair and face regions more naturally and successfully prevented the distortion of the hat region.

• Journal of Broadcast Engineering
• /
• v.24 no.3
• /
• pp.440-451
• /
• 2019

We propose a system that allows three dimensional manipulation of face in video. The 3D face manipulation of the proposed system overlays the 3D face model with the user 's manipulation on the face region of the video frame, and it allows 3D manipulation of the video in real time unlike existing applications or methods. To achieve this feature, first, the 3D morphable face model is registered with the image. At the same time, user's manipulation is applied to the registered model. Finally, the frame image mapped to the model as texture, and the texture-mapped and deformed model is rendered. Since this process requires lots of operations, parallel processing is adopted for real-time processing; the system is divided into modules according to functionalities, and each module runs in parallel on each thread. Experimental results show that specific parts of the face in video can be manipulated in real time.

• Journal of the Korea Society of Computer and Information
• /
• v.25 no.5
• /
• pp.37-46
• /
• 2020

Recently, studies of facial attribute editing have obtained realistic results using generative adversarial net (GAN) and encoder-decoder structure. Spatial attention GAN (SaGAN), one of the latest researches, is the method that can change only desired attribute in a face image by spatial attention mechanism. However, sometimes unnatural results are obtained due to insufficient information on face areas. In this paper, we propose an improved SaGAN (MSaGAN) using a guide mask for learning and applying multitask learning approach to improve the limitations of the existing methods. Through extensive experiments, we evaluated the results of the facial attribute editing in therms of the mask loss function and the neural network structure. It has been shown that the proposed method can efficiently produce more natural results compared to the previous methods.

• Journal of Broadcast Engineering
• /
• v.27 no.4
• /
• pp.499-510
• /
• 2022

This paper introduces a video editing system for senior creator who is not familiar to video editing. Based on video analysis techniques, it provide various information and delete unwanted shot. The system detects shot boundaries based on RNN(Recurrent Neural Network), and it determines the deletion of video shots. The shots can be deleted using shot-level significance, which is computed by detecting focused area. It is possible to delete unfocused shots or motion-blurred shots using the significance. The system detects object and face, and extract the information of emotion, age, and gender from face image. Users can create video contents using the information. Decorating tools are also prepared, and in the tools, the preferred design, which is determined from user history, places in the front of the design element list. With the video editing system, senior creators can make their own video contents easily and quickly.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.6
• /
• pp.2115-2127
• /
• 2021

Although significant progress has been made in synthesizing visually realistic face images by Generative Adversarial Networks (GANs), there still lacks effective approaches to provide fine-grained control over the generation process for semantic facial attribute editing. In this work, we propose a novel cross channel self-attention based generative adversarial network (CCA-GAN), which weights the importance of multiple channels of features and archives pixel-level feature alignment and conversion, to reduce the impact on irrelevant attributes while editing the target attributes. Evaluation results show that CCA-GAN outperforms state-of-the-art models on the CelebA dataset, reducing Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) by 15~28% and 25~100%, respectively. Furthermore, visualization of generated samples confirms the effect of disentanglement of the proposed model.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.10
• /
• pp.3685-3707
• /
• 2021

In recent years, facial attribute editing has been successfully used to effectively change face images of various attributes based on generative adversarial networks and encoder-decoder models. However, existing models have a limitation in that they may change an unintended part in the process of changing an attribute or may generate an unnatural result. In this paper, we propose a model that improves the learning of the attention mask by adding a spatial attention mechanism based on the unified selective transfer network (referred to as STGAN) using semi-supervised learning. The proposed model can edit multiple attributes while preserving details independent of the attributes being edited. This study makes two main contributions to the literature. First, we propose an encoder-decoder model structure that learns and edits multiple facial attributes and suppresses distortion using an attention mask. Second, we define guide masks and propose a method and an objective function that use the guide masks for multiple facial attribute editing through semi-supervised learning. Through qualitative and quantitative evaluations of the experimental results, the proposed method was proven to yield improved results that preserve the image details by suppressing unintended changes than existing methods.

• International Journal of Advanced Culture Technology
• /
• v.7 no.4
• /
• pp.289-294
• /
• 2019

As social network service platforms grow and one-person media market expands, people upload their own photos and/or videos through multiple open platforms. However, it can be illegal to upload the digital contents containing the faces of others on the public sites without their permission. Therefore, many people are spending much time and effort in editing such digital contents so that the faces of others should not be exposed to the public. In this paper, we propose an automatic face hiding system called 'autoblur', which detects all the unregistered faces and mosaic them automatically. The system has been implemented using the GitHub MIT open-source 'Face Recognition' which is based on deep learning technology. In this system, two dozens of face images of the user are taken from different angles to register his/her own face. Once the face of the user is learned and registered, the system detects all the other faces for the given photo or video and then blurs them out. Our experiments show that it produces quick and correct results for the sample photos.