• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.11
• /
• pp.1482-1484
• /
• 2019

We present a method for transmitting 3D object information in real time in a telepresence system. Three-dimensional object information consists of a large amount of point cloud data, which requires high performance computing power and ultra-wideband network transmission environment to process and transmit such a large amount of data in real time. In this paper, multiple users can transmit object motion and facial expression information in real time even in small network bands by using GANs (Generative Adversarial Networks), a non-supervised learning machine learning algorithm, for real-time transmission of 3D point cloud data. In particular, we propose the creation of an object similar to the original using only the feature information of 3D objects using conditional GANs.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2021.11a
• /
• pp.764-767
• /
• 2021

While Generative Adversarial Networks (GANs) have seen huge success in image synthesis tasks, synthesizing high-quality images from text descriptions is a challenging problem in computer vision. This paper proposes a method named Text-to-Face Generation Using Multi-Scale Gradients for Conditional Generative Adversarial Networks (T2F-MSGGANs) that combines GANs and a natural language processing model to create human faces has features found in the input text. The proposed method addresses two problems of GANs: model collapse and training instability by investigating how gradients at multiple scales can be used to generate high-resolution images. We show that T2F-MSGGANs converge stably and generate good-quality images.

• Korean Journal of Remote Sensing
• /
• v.38 no.6_1
• /
• pp.1357-1369
• /
• 2022

Cloud removal is an essential image processing step for any task requiring time-series optical images, such as vegetation monitoring and change detection. This paper presents a two-stage cloud removal method that combines conditional generative adversarial networks (cGANs) with regression-based calibration to construct a cloud-free time-series optical image set. In the first stage, the cGANs generate initial prediction results using quantitative relationships between optical and synthetic aperture radar images. In the second stage, the relationships between the predicted results and the actual values in non-cloud areas are first quantified via random forest-based regression modeling and then used to calibrate the cGAN-based prediction results. The potential of the proposed method was evaluated from a cloud removal experiment using Sentinel-2 and COSMO-SkyMed images in the rice field cultivation area of Gimje. The cGAN model could effectively predict the reflectance values in the cloud-contaminated rice fields where severe changes in physical surface conditions happened. Moreover, the regression-based calibration in the second stage could improve the prediction accuracy, compared with a regression-based cloud removal method using a supplementary image that is temporally distant from the target image. These experimental results indicate that the proposed method can be effectively applied to restore cloud-contaminated areas when cloud-free optical images are unavailable for environmental monitoring.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2019.05a
• /
• pp.157-159
• /
• 2019

Hologram technology has been actively developed in terms of generation, transmission, and reproduction of 3D objects, but it is currently in a state of rest because of various limitations. Beyond VR and AR, the pseudo-hologram market is growing at an intermediate stage to meet the needs of new technologies. The key to the technology of hologram is to generate vast 3 dimensional data in the form of a point cloud, transmit the vast amount of data through the communication network in real time, and reproduce it like the original at the destination. In this paper, we propose a method to transmit massive 3 - D data in real - time and transmit the minutiae points of 3 - dimensional object information to reproduce the object as similar to original.

• The Journal of the Acoustical Society of Korea
• /
• v.41 no.3
• /
• pp.351-358
• /
• 2022

Recently, Generative Adversarial Networks (GAN) and Variational AutoEncoders (VAE) have been applied to voice conversion that can make use of non-parallel training data. Especially, Conditional Cycle-Consistent Generative Adversarial Networks (CC-GAN) and Cycle-Consistent Variational AutoEncoders (CycleVAE) show promising results in many-to-many voice conversion among multiple speakers. However, the number of speakers has been relatively small in the conventional voice conversion studies using the CC-GANs and the CycleVAEs. In this paper, we extend the number of speakers to 100, and analyze the performances of the many-to-many voice conversion methods experimentally. It has been found through the experiments that the CC-GAN shows 4.5 % less Mel-Cepstral Distortion (MCD) for a small number of speakers, whereas the CycleVAE shows 12.7 % less MCD in a limited training time for a large number of speakers.

• The Bulletin of The Korean Astronomical Society
• /
• v.44 no.1
• /
• pp.51.3-51.3
• /
• 2019

Solar magnetograms are important for studying solar activity and predicting space weather disturbances1. Farside magnetograms can be constructed from local helioseismology without any farside data2-4, but their quality is lower than that of typical frontside magnetograms. Here we generate farside solar magnetograms from STEREO/Extreme UltraViolet Imager (EUVI) $304-{\AA}$ images using a deep learning model based on conditional generative adversarial networks (cGANs). We train the model using pairs of Solar Dynamics Observatory (SDO)/Atmospheric Imaging Assembly (AIA) $304-{\AA}$ images and SDO/Helioseismic and Magnetic Imager (HMI) magnetograms taken from 2011 to 2017 except for September and October each year. We evaluate the model by comparing pairs of SDO/HMI magnetograms and cGAN-generated magnetograms in September and October. Our method successfully generates frontside solar magnetograms from SDO/AIA $304-{\AA}$ images and these are similar to those of the SDO/HMI, with Hale-patterned active regions being well replicated. Thus we can monitor the temporal evolution of magnetic fields from the farside to the frontside of the Sun using SDO/HMI and farside magnetograms generated by our model when farside extreme-ultraviolet data are available. This study presents an application of image-to-image translation based on cGANs to scientific data.

• The Bulletin of The Korean Astronomical Society
• /
• v.44 no.1
• /
• pp.81.2-81.2
• /
• 2019

We develop an image-to-image translation model, which is a popular deep learning method based on conditional Generative Adversarial Networks (cGANs), to generate solar magnetograms and EUV images from sunspot drawings. For this, we train the model using pairs of sunspot drawings from Mount Wilson Observatory (MWO) and their corresponding SDO/HMI magnetograms and SDO/AIA EUV images (512 by 512) from January 2012 to September 2014. We test the model by comparing pairs of actual SDO images (magnetogram and EUV images) and the corresponding AI-generated ones from October to December in 2014. Our results show that bipolar structures and coronal loop structures of AI-generated images are consistent with those of the original ones. We find that their unsigned magnetic fluxes well correlate with those of the original ones with a good correlation coefficient of 0.86. We also obtain pixel-to-pixel correlations EUV images and AI-generated ones. The average correlations of 92 test samples for several SDO lines are very good: 0.88 for AIA 211, 0.87 for AIA 1600 and 0.93 for AIA 1700. These facts imply that AI-generated EUV images quite similar to AIA ones. Applying this model to the Galileo sunspot drawings in 1612, we generate HMI-like magnetograms and AIA-like EUV images of the sunspots. This application will be used to generate solar images using historical sunspot drawings.

• The Bulletin of The Korean Astronomical Society
• /
• v.44 no.1
• /
• pp.51.2-51.2
• /
• 2019

Multi-wavelength observations become very popular in astronomy. Even though there are some correlations among different sensor images, it is not easy to translate from one to the other one. In this study, we apply a deep learning method for image-to-image translation, based on conditional generative adversarial networks (cGANs), to solar images. To examine the validity of the method for scientific data, we consider several different types of pairs: (1) Generation of SDO/EUV images from SDO/HMI magnetograms, (2) Generation of backside magnetograms from STEREO/EUVI images, (3) Generation of EUV & X-ray images from Carrington sunspot drawing, and (4) Generation of solar magnetograms from Ca II images. It is very impressive that AI-generated ones are quite consistent with actual ones. In addition, we apply the convolution neural network to the forecast of solar flares and find that our method is better than the conventional method. Our study also shows that the forecast of solar proton flux profiles using Long and Short Term Memory method is better than the autoregressive method. We will discuss several applications of these methodologies for scientific research.

• The Bulletin of The Korean Astronomical Society
• /
• v.44 no.2
• /
• pp.42.3-42.3
• /
• 2019

We translate Solar Dynamics Observatory/Atmospheric Imaging Assembly (AIA) ultraviolet (UV) multi-channel images into another UV single-channel image using a deep learning algorithm based on conditional generative adversarial networks (cGANs). The base input channel, which has the highest correlation coefficient (CC) between UV channels of AIA, is 193 Å. To complement this channel, we choose two channels, 1600 and 304 Å, which represent upper photosphere and chromosphere, respectively. Input channels for three models are single (193 Å), dual (193+1600 Å), and triple (193+1600+304 Å), respectively. Quantitative comparisons are made for test data sets. Main results from this study are as follows. First, the single model successfully produce other coronal channel images but less successful for chromospheric channel (304 Å) and much less successful for two photospheric channels (1600 and 1700 Å). Second, the dual model shows a noticeable improvement of the CC between the model outputs and Ground truths for 1700 Å. Third, the triple model can generate all other channel images with relatively high CCs larger than 0.89. Our results show a possibility that if three channels from photosphere, chromosphere, and corona are selected, other multi-channel images could be generated by deep learning. We expect that this investigation will be a complementary tool to choose a few UV channels for future solar small and/or deep space missions.