• Title/Summary/Keyword: Synthetic data generation

Search Result 116, Processing Time 0.027 seconds

Enhanced Regular Expression as a DGL for Generation of Synthetic Big Data

  • Kai, Cheng;Keisuke, Abe
    • Journal of Information Processing Systems
    • /
    • v.19 no.1
    • /
    • pp.1-16
    • /
    • 2023
  • Synthetic data generation is generally used in performance evaluation and function tests in data-intensive applications, as well as in various areas of data analytics, such as privacy-preserving data publishing (PPDP) and statistical disclosure limit/control. A significant amount of research has been conducted on tools and languages for data generation. However, existing tools and languages have been developed for specific purposes and are unsuitable for other domains. In this article, we propose a regular expression-based data generation language (DGL) for flexible big data generation. To achieve a general-purpose and powerful DGL, we enhanced the standard regular expressions to support the data domain, type/format inference, sequence and random generation, probability distributions, and resource reference. To efficiently implement the proposed language, we propose caching techniques for both the intermediate and database queries. We evaluated the proposed improvement experimentally.

Synthetic Image Generation for Military Vehicle Detection (군용물체탐지 연구를 위한 가상 이미지 데이터 생성)

  • Se-Yoon Oh;Hunmin Yang
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.26 no.5
    • /
    • pp.392-399
    • /
    • 2023
  • This research paper investigates the effectiveness of using computer graphics(CG) based synthetic data for deep learning in military vehicle detection. In particular, we explore the use of synthetic image generation techniques to train deep neural networks for object detection tasks. Our approach involves the generation of a large dataset of synthetic images of military vehicles, which is then used to train a deep learning model. The resulting model is then evaluated on real-world images to measure its effectiveness. Our experimental results show that synthetic training data alone can achieve effective results in object detection. Our findings demonstrate the potential of CG-based synthetic data for deep learning and suggest its value as a tool for training models in a variety of applications, including military vehicle detection.

Generation of Synthetic Particle Images for Particle Image Velocimetry using Physics-Informed Neural Network (물리 기반 인공신경망을 이용한 PIV용 합성 입자이미지 생성)

  • Hyeon Jo Choi;Myeong Hyeon, Shin;Jong Ho, Park;Jinsoo Park
    • Journal of the Korean Society of Visualization
    • /
    • v.21 no.1
    • /
    • pp.119-126
    • /
    • 2023
  • Acquiring experimental data for PIV verification or machine learning training data is resource-demanding, leading to an increasing interest in synthetic particle images as simulation data. Conventional synthetic particle image generation algorithms do not follow physical laws, and the use of CFD is time-consuming and requires computing resources. In this study, we propose a new method for synthetic particle image generation, based on a Physics-Informed Neural Networks(PINN). The PINN is utilized to infer the flow fields, enabling the generation of synthetic particle images that follow physical laws with reduced computation time and have no constraints on spatial resolution compared to CFD. The proposed method is expected to contribute to the verification of PIV algorithms.

Game Engine Driven Synthetic Data Generation for Computer Vision-Based Construction Safety Monitoring

  • Lee, Heejae;Jeon, Jongmoo;Yang, Jaehun;Park, Chansik;Lee, Dongmin
    • International conference on construction engineering and project management
    • /
    • 2022.06a
    • /
    • pp.893-903
    • /
    • 2022
  • Recently, computer vision (CV)-based safety monitoring (i.e., object detection) system has been widely researched in the construction industry. Sufficient and high-quality data collection is required to detect objects accurately. Such data collection is significant for detecting small objects or images from different camera angles. Although several previous studies proposed novel data augmentation and synthetic data generation approaches, it is still not thoroughly addressed (i.e., limited accuracy) in the dynamic construction work environment. In this study, we proposed a game engine-driven synthetic data generation model to enhance the accuracy of the CV-based object detection model, mainly targeting small objects. In the virtual 3D environment, we generated synthetic data to complement training images by altering the virtual camera angles. The main contribution of this paper is to confirm whether synthetic data generated in the game engine can improve the accuracy of the CV-based object detection model.

  • PDF

A Study on Synthetic Flight Vehicle Trajectory Data Generation Using Time-series Generative Adversarial Network and Its Application to Trajectory Prediction of Flight Vehicles (시계열 생성적 적대 신경망을 이용한 비행체 궤적 합성 데이터 생성 및 비행체 궤적 예측에서의 활용에 관한 연구)

  • Park, In Hee;Lee, Chang Jin;Jung, Chanho
    • Journal of IKEEE
    • /
    • v.25 no.4
    • /
    • pp.766-769
    • /
    • 2021
  • In order to perform tasks such as design, control, optimization, and prediction of flight vehicle trajectories based on machine learning techniques including deep learning, a certain amount of flight vehicle trajectory data is required. However, there are cases in which it is difficult to secure more than a certain amount of flight vehicle trajectory data for various reasons. In such cases, synthetic data generation could be one way to make machine learning possible. In this paper, to explore this possibility, we generated and evaluated synthetic flight vehicle trajectory data using time-series generative adversarial neural network. In addition, various ablation studies (comparative experiments) were performed to explore the possibility of using synthetic data in the aircraft trajectory prediction task. The experimental results presented in this paper are expected to be of practical help to researchers who want to conduct research on the possibility of using synthetic data in the generation of synthetic flight vehicle trajectory data and the work related to flight vehicle trajectories.

Synthetic Infra-Red Image Dataset Generation by CycleGAN based on SSIM Loss Function (SSIM 목적 함수와 CycleGAN을 이용한 적외선 이미지 데이터셋 생성 기법 연구)

  • Lee, Sky;Leeghim, Henzeh
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.25 no.5
    • /
    • pp.476-486
    • /
    • 2022
  • Synthetic dynamic infrared image generation from the given virtual environment is being the primary goal to simulate the output of the infra-red(IR) camera installed on a vehicle to evaluate the control algorithm for various search & reconnaissance missions. Due to the difficulty to obtain actual IR data in complex environments, Artificial intelligence(AI) has been used recently in the field of image data generation. In this paper, CycleGAN technique is applied to obtain a more realistic synthetic IR image. We added the Structural Similarity Index Measure(SSIM) loss function to the L1 loss function to generate a more realistic synthetic IR image when the CycleGAN image is generated. From the simulation, it is applicable to the guided-missile flight simulation tests by using the synthetic infrared image generated by the proposed technique.

A comparison of synthetic data approaches using utility and disclosure risk measures (유용성과 노출 위험성 지표를 이용한 재현자료 기법 비교 연구)

  • Seongbin An;Trang Doan;Juhee Lee;Jiwoo Kim;Yong Jae Kim;Yunji Kim;Changwon Yoon;Sungkyu Jung;Dongha Kim;Sunghoon Kwon;Hang J Kim;Jeongyoun Ahn;Cheolwoo Park
    • The Korean Journal of Applied Statistics
    • /
    • v.36 no.2
    • /
    • pp.141-166
    • /
    • 2023
  • This paper investigates synthetic data generation methods and their evaluation measures. There have been increasing demands for releasing various types of data to the public for different purposes. At the same time, there are also unavoidable concerns about leaking critical or sensitive information. Many synthetic data generation methods have been proposed over the years in order to address these concerns and implemented in some countries, including Korea. The current study aims to introduce and compare three representative synthetic data generation approaches: Sequential regression, nonparametric Bayesian multiple imputations, and deep generative models. Several evaluation metrics that measure the utility and disclosure risk of synthetic data are also reviewed. We provide empirical comparisons of the three synthetic data generation approaches with respect to various evaluation measures. The findings of this work will help practitioners to have a better understanding of the advantages and disadvantages of those synthetic data methods.

Studies on the Stochastic Generation of Long Term Runoff (2) (장기유출량의 추계학적 모의 발생에 관한 연구 (II))

  • 이순혁;맹승진;박종국
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.35 no.3
    • /
    • pp.117-129
    • /
    • 1993
  • This study was conducted to get reasonable and abundant hydrological time series of monthly flows simulated by a best fitting stochastic simulation model for the establishment of rational design and the rationalization of management for agricultural hydraulic structures including reservoirs. Comparative analysis carried out for both statistical characteristics and synthetic monthly flows simulated by the multi-season first order Markov model based on Gamma distribution which is confirmed as good one in the first report of this study and by Harmonic synthetic model analyzed in this report for the six watersheds of Yeong San and Seom Jin river systems. 1.Arithmetic mean values of synthetic monthly flows simulated by Gamma distribution are much closer to the results of the observed data than those of Harmonic synthetic model in the applied watersheds. 2.In comparison with the coefficients of variation, index of fluctuation for monthly flows simulated by two kinds of synthetic models, those based on Gamma distribution are appeared closer to the observed data than those of Harmonic synthetic model both in Yeong San and Seom Jin river systems. 3.It was found that synthetic monthly flows based on Gamma distribution are considered to give better results than those of Harmonic synthetic model in the applied watersheds. 4.Continuation studies by comparison with other simulation techniques are to be desired for getting reasonable generation technique of synthetic monthly flows for the various river systems in Korea.

  • PDF

Generation of Synthetic Time Series Wind Speed Data using Second-Order Markov Chain Model (2차 마르코프 사슬 모델을 이용한 시계열 인공 풍속 자료의 생성)

  • Ki-Wahn Ryu
    • Journal of Wind Energy
    • /
    • v.14 no.1
    • /
    • pp.37-43
    • /
    • 2023
  • In this study, synthetic time series wind data was generated numerically using a second-order Markov chain. One year of wind data in 2020 measured by the AWS on Wido Island was used to investigate the statistics for measured wind data. Both the transition probability matrix and the cumulative transition probability matrix for annual hourly mean wind speed were obtained through statistical analysis. Probability density distribution along the wind speed and autocorrelation according to time were compared with the first- and the second-order Markov chains with various lengths of time series wind data. Probability density distributions for measured wind data and synthetic wind data using the first- and the second-order Markov chains were also compared to each other. For the case of the second-order Markov chain, some improvement of the autocorrelation was verified. It turns out that the autocorrelation converges to zero according to increasing the wind speed when the data size is sufficiently large. The generation of artificial wind data is expected to be useful as input data for virtual digital twin wind turbines.

A Study on Synthetic Data Generation Based Safe Differentially Private GAN (차분 프라이버시를 만족하는 안전한 GAN 기반 재현 데이터 생성 기술 연구)

  • Kang, Junyoung;Jeong, Sooyong;Hong, Dowon;Seo, Changho
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.5
    • /
    • pp.945-956
    • /
    • 2020
  • The publication of data is essential in order to receive high quality services from many applications. However, if the original data is published as it is, there is a risk that sensitive information (political tendency, disease, ets.) may reveal. Therefore, many research have been proposed, not the original data but the synthetic data generating and publishing to privacy preserve. but, there is a risk of privacy leakage still even if simply generate and publish the synthetic data by various attacks (linkage attack, inference attack, etc.). In this paper, we propose a synthetic data generation algorithm in which privacy preserved by applying differential privacy the latest privacy protection technique to GAN, which is drawing attention as a synthetic data generative model in order to prevent the leakage of such sensitive information. The generative model used CGAN for efficient learning of labeled data, and applied Rényi differential privacy, which is relaxation of differential privacy, considering the utility aspects of the data. And validation of the utility of the generated data is conducted and compared through various classifiers.