• Title/Summary/Keyword: Dataset Generation

Search Result 195, Processing Time 0.025 seconds

A Brief Survey into the Field of Automatic Image Dataset Generation through Web Scraping and Query Expansion

  • Bart Dikmans;Dongwann Kang
    • Journal of Information Processing Systems
    • /
    • v.19 no.5
    • /
    • pp.602-613
    • /
    • 2023
  • High-quality image datasets are in high demand for various applications. With many online sources providing manually collected datasets, a persisting challenge is to fully automate the dataset collection process. In this study, we surveyed an automatic image dataset generation field through analyzing a collection of existing studies. Moreover, we examined fields that are closely related to automated dataset generation, such as query expansion, web scraping, and dataset quality. We assess how both noise and regional search engine differences can be addressed using an automated search query expansion focused on hypernyms, allowing for user-specific manual query expansion. Combining these aspects provides an outline of how a modern web scraping application can produce large-scale image datasets.

Default Prediction for Real Estate Companies with Imbalanced Dataset

  • Dong, Yuan-Xiang;Xiao, Zhi;Xiao, Xue
    • Journal of Information Processing Systems
    • /
    • v.10 no.2
    • /
    • pp.314-333
    • /
    • 2014
  • When analyzing default predictions in real estate companies, the number of non-defaulted cases always greatly exceeds the defaulted ones, which creates the two-class imbalance problem. This lowers the ability of prediction models to distinguish the default sample. In order to avoid this sample selection bias and to improve the prediction model, this paper applies a minority sample generation approach to create new minority samples. The logistic regression, support vector machine (SVM) classification, and neural network (NN) classification use an imbalanced dataset. They were used as benchmarks with a single prediction model that used a balanced dataset corrected by the minority samples generation approach. Instead of using prediction-oriented tests and the overall accuracy, the true positive rate (TPR), the true negative rate (TNR), G-mean, and F-score are used to measure the performance of default prediction models for imbalanced dataset. In this paper, we describe an empirical experiment that used a sampling of 14 default and 315 non-default listed real estate companies in China and report that most results using single prediction models with a balanced dataset generated better results than an imbalanced dataset.

Hangul Font Dataset for Korean Font Research Based on Deep Learning (딥러닝 기반의 한글 폰트 연구를 위한 한글 폰트 데이터셋)

  • Ko, Debbie Honghee;Lee, Hyunsoo;Suk, Jungjae;Hassan, Ammar Ul;Choi, Jaeyoung
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.10 no.2
    • /
    • pp.73-78
    • /
    • 2021
  • Recently, as interest in deep learning has increased, many researches in various fields using deep learning techniques have been conducted. Studies on automatic generation of fonts using deep learning-based generation models are limited to several languages such as Roman or Chinese characters. Generating Korean font is a very time-consuming and expensive task, and can be easily created using deep learning. For research on generating Korean fonts, it is important to prepare a Korean font dataset from the viewpoint of process automation in order to keep pace with deep learning-based generation models. In this paper, we propose a Korean font dataset for deep learning-based Korean font research and describe a method of constructing the dataset. Based on the Korean font data set proposed in this paper, we show the usefulness of the proposed dataset configuration through the process of applying it to a deep learning Korean font generation application.

Construction of a Video Dataset for Face Tracking Benchmarking Using a Ground Truth Generation Tool

  • Do, Luu Ngoc;Yang, Hyung Jeong;Kim, Soo Hyung;Lee, Guee Sang;Na, In Seop;Kim, Sun Hee
    • International Journal of Contents
    • /
    • v.10 no.1
    • /
    • pp.1-11
    • /
    • 2014
  • In the current generation of smart mobile devices, object tracking is one of the most important research topics for computer vision. Because human face tracking can be widely used for many applications, collecting a dataset of face videos is necessary for evaluating the performance of a tracker and for comparing different approaches. Unfortunately, the well-known benchmark datasets of face videos are not sufficiently diverse. As a result, it is difficult to compare the accuracy between different tracking algorithms in various conditions, namely illumination, background complexity, and subject movement. In this paper, we propose a new dataset that includes 91 face video clips that were recorded in different conditions. We also provide a semi-automatic ground-truth generation tool that can easily be used to evaluate the performance of face tracking systems. This tool helps to maintain the consistency of the definitions for the ground-truth in each frame. The resulting video data set is used to evaluate well-known approaches and test their efficiency.

Automatic Dataset Generation of Object Detection and Instance Segmentation using Mask R-CNN (Mask R-CNN을 이용한 물체인식 및 개체분할의 학습 데이터셋 자동 생성)

  • Jo, HyunJun;Kim, Dawit;Song, Jae-Bok
    • The Journal of Korea Robotics Society
    • /
    • v.14 no.1
    • /
    • pp.31-39
    • /
    • 2019
  • A robot usually adopts ANN (artificial neural network)-based object detection and instance segmentation algorithms to recognize objects but creating datasets for these algorithms requires high labeling costs because the dataset should be manually labeled. In order to lower the labeling cost, a new scheme is proposed that can automatically generate a training images and label them for specific objects. This scheme uses an instance segmentation algorithm trained to give the masks of unknown objects, so that they can be obtained in a simple environment. The RGB images of objects can be obtained by using these masks, and it is necessary to label the classes of objects through a human supervision. After obtaining object images, they are synthesized with various background images to create new images. Labeling the synthesized images is performed automatically using the masks and previously input object classes. In addition, human intervention is further reduced by using the robot arm to collect object images. The experiments show that the performance of instance segmentation trained through the proposed method is equivalent to that of the real dataset and that the time required to generate the dataset can be significantly reduced.

Geometric and Semantic Improvement for Unbiased Scene Graph Generation

  • Ruhui Zhang;Pengcheng Xu;Kang Kang;You Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.10
    • /
    • pp.2643-2657
    • /
    • 2023
  • Scene graphs are structured representations that can clearly convey objects and the relationships between them, but are often heavily biased due to the highly skewed, long-tailed relational labeling in the dataset. Indeed, the visual world itself and its descriptions are biased. Therefore, Unbiased Scene Graph Generation (USGG) prefers to train models to eliminate long-tail effects as much as possible, rather than altering the dataset directly. To this end, we propose Geometric and Semantic Improvement (GSI) for USGG to mitigate this issue. First, to fully exploit the feature information in the images, geometric dimension and semantic dimension enhancement modules are designed. The geometric module is designed from the perspective that the position information between neighboring object pairs will affect each other, which can improve the recall rate of the overall relationship in the dataset. The semantic module further processes the embedded word vector, which can enhance the acquisition of semantic information. Then, to improve the recall rate of the tail data, the Class Balanced Seesaw Loss (CBSLoss) is designed for the tail data. The recall rate of the prediction is improved by penalizing the body or tail relations that are judged incorrectly in the dataset. The experimental findings demonstrate that the GSI method performs better than mainstream models in terms of the mean Recall@K (mR@K) metric in three tasks. The long-tailed imbalance in the Visual Genome 150 (VG150) dataset is addressed better using the GSI method than by most of the existing methods.

A Virtual Battlefield Situation Dataset Generation for Battlefield Analysis based on Artificial Intelligence

  • Cho, Eunji;Jin, Soyeon;Shin, Yukyung;Lee, Woosin
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.6
    • /
    • pp.33-42
    • /
    • 2022
  • In the existing intelligent command control system study, the analysis results of the commander's battlefield situation questions are provided from knowledge-based situation data. Analysis reporters write these results in various expressions of natural language. However, it is important to analyze situations about information and intelligence according to context. Analyzing the battlefield situation using artificial intelligence is necessary. We propose a virtual dataset generation method based on battlefield simulation scenarios in order to provide a dataset necessary for the battlefield situation analysis based on artificial intelligence. Dataset is generated after identifying battlefield knowledge elements in scenarios. When a candidate hypothesis is created, a unit hypothesis is automatically created. By combining unit hypotheses, similar identification hypothesis combinations are generated. An aggregation hypothesis is generated by grouping candidate hypotheses. Dataset generator SW implementation demonstrates that the proposed method can be generated the virtual battlefield situation dataset.

A Study on the Generation of Datasets for Applied AI to OLED Life Prediction

  • CHUNG, Myung-Ae;HAN, Dong Hun;AHN, Seongdeok;KANG, Min Soo
    • Korean Journal of Artificial Intelligence
    • /
    • v.10 no.2
    • /
    • pp.7-11
    • /
    • 2022
  • OLED displays cannot be used permanently due to burn-in or generation of dark spots due to degradation. Therefore, the time when the display can operate normally is very important. It is close to impossible to physically measure the time when the display operates normally. Therefore, the time that works normally should be predicted in a way other than a physical way. Therefore, if you do computer simulations based on artificial intelligence, you can increase the accuracy of prediction by saving time and continuous learning. Therefore, if we do computer simulations based on artificial intelligence, we can increase the accuracy of prediction by saving time and continuous learning. In this paper, a dataset in the form of development from generation to diffusion of dark spots, which is one of the causes related to the life of OLED, was generated by applying the finite element method. The dark spots were generated in nine conditions, such as 0.1 to 2.0 ㎛ with the size of pinholes, the number was 10 to 100, and 50% with water content. The learning data created in this way may be a criterion for generating an artificial intelligence-based dataset.

Method for Importance based Streamline Generation on the Massive Fluid Dynamics Dataset (대용량 유동해석 데이터에서의 중요도 기반 스트림라인 생성 방법)

  • Lee, Joong-Youn;Kim, Min Ah;Lee, Sehoon
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.6
    • /
    • pp.27-37
    • /
    • 2018
  • Streamline generation is one of the most representative visualization methods to analyze the flow stream of fluid dynamics dataset. It is a challenging problem, however, to determine the seed locations for effective streamline visualization. Meanwhile, it needs much time to compute effective seed locations and streamlines on the massive flow dataset. In this paper, we propose not only an importance based method to determine seed locations for the effective streamline placements but also a parallel streamline visualization method on the distributed visualization system. Moreover, we introduce case studies on the real fluid dynamics dataset using GLOVE visualization system to evaluate the proposed method.

Synthetic Image Dataset Generation for Defense using Generative Adversarial Networks (국방용 합성이미지 데이터셋 생성을 위한 대립훈련신경망 기술 적용 연구)

  • Yang, Hunmin
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.22 no.1
    • /
    • pp.49-59
    • /
    • 2019
  • Generative adversarial networks(GANs) have received great attention in the machine learning field for their capacity to model high-dimensional and complex data distribution implicitly and generate new data samples from the model distribution. This paper investigates the model training methodology, architecture, and various applications of generative adversarial networks. Experimental evaluation is also conducted for generating synthetic image dataset for defense using two types of GANs. The first one is for military image generation utilizing the deep convolutional generative adversarial networks(DCGAN). The other is for visible-to-infrared image translation utilizing the cycle-consistent generative adversarial networks(CycleGAN). Each model can yield a great diversity of high-fidelity synthetic images compared to training ones. This result opens up the possibility of using inexpensive synthetic images for training neural networks while avoiding the enormous expense of collecting large amounts of hand-annotated real dataset.