• Journal of the Korean Society of Industry Convergence
• /
• v.25 no.4_2
• /
• pp.687-697
• /
• 2022

An automated system is needed for the effectiveness of non-destructive testing. In order to utilize the radiographic testing data accumulated in the film, the types of welding defects were classified into 9 and the shape of defects were analyzed. Data was preprocessed to use deep learning with high performance in image classification, and a combination of one-stage/two-stage method and convolutional neural networks/Transformer backbone was compared to confirm a model suitable for welding defect detection. The combination of two-stage, which can learn step-by-step, and deep-layered CNN backbone, showed the best performance with mean average precision 0.868.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2022.06a
• /
• pp.214-216
• /
• 2022

비디오 프레임 보간 기술은 시간 해상도를 증가시키는 기술로 최근 Convolutional Neural Network(이하 CNN) 기반의 다양한 연구가 진행되고 있다. 하지만 일부 시각에서는 CNN 기반의 연구가 동일한 커널을 모든 화소에 적용하는 것과 객체의 움직임을 예측하기 위해 장기간의 데이터를 활용하는 것에 한계점이 있다고 주장한다. 이에 따라 장기간의 데이터 활용에 특화된 트랜스포머 기반의 비디오 프레임 보간 기술이 제안되었다. 본 논문에서는 트랜스포머 기반의 기존 연구에서 합성 네트워크의 성능을 향상시키기 위해 광학 흐름 안내 기반의 새로운 학습 방법을 제안한다 실험 결과를 통해 평균 PSNR 0.09dB와 SSIM 0.0031 성능 향상을 확인한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.520-522
• /
• 2022

의료 인공지능은 특정 진단에서 높은 정확도를 보이지만 모델의 신뢰성 문제로 인해 활발하게 쓰이지 못하고 있다. 이에 따라 인공지능 모델의 진단에 대한 원인 설명의 필요성이 대두되었고 설명가능한 의료 인공지능에 관한 연구가 활발히 진행되고 있다. 하지만 MRI 등 의료 영상 인공지능 분야에서 주로 진행되고 있으며, 이미지 형태가 아닌 전자의무기록 데이터 (Electronic Health Record, EHR) 를 기반으로 한 모델의 설명가능성 연구는 EHR 데이터 자체의 복잡성 때문에 활발하게 진행 되지 않고 있다. 본 논문에서는 전자의무기록 데이터인 MIMIC-III (Medical Information Mart for Intensive Care) 를 전처리 및 그래프로 표현하고, GCT (Graph Convolutional Transformer) 모델을 학습시켰다. 학습 후, 어텐션 흐름 그래프를 시각화해서 모델의 예측에 대한 직관적인 설명을 제공한다.

• Smart Media Journal
• /
• v.12 no.9
• /
• pp.45-59
• /
• 2023

Weakly supervised object localization (WSOL) is a task of localizing an object in an image using only image-level labels. Previous studies have followed the conventional class activation mapping (CAM) pipeline. However, we reveal the current CAM approach suffers from problems which cause original CAM could not capture the complete defects features. This work utilizes a convolutional neural network (CNN) pretrained on image-level labels to generate class activation maps in a multi-scale manner to highlight discriminative regions. Additionally, a vision transformer (ViT) pretrained was treated to produce multi-head attention maps as an auxiliary detector. By integrating the CNN-based CAMs and attention maps, our approach localizes defective regions without requiring bounding box or pixel-level supervision during training. We evaluate our approach on a dataset of apple images with only image-level labels of defect categories. Experiments demonstrate our proposed method aligns with several Object Detection models performance, hold a promise for improving localization.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.4
• /
• pp.19-26
• /
• 2022

With the recent development of intelligent transportation systems, car license plate recognition systems are being used in various fields. Such systems need to guarantee real-time performance to recognize the license plate of a driving car. Also, they should keep a high recognition rate even in problematic situations such as small license plates in low-resolution and unclear image due to distortion. In this paper, we propose a real-time car license plate recognition system that improved processing speed using object detection algorithm based on anchor-free method and text recognition algorithm based on Convolutional Neural Network(CNN). In addition, we used Spatial Transformer Network to increase the recognition rate on the low resolution or distorted images. We confirm that the proposed system is faster than previously existing car license plate recognition systems and maintains a high recognition rate in a variety of environment and quality images because the proposed system's recognition rate is 93.769% and the processing speed per image is about 0.006 seconds.

• Journal of Korea Society of Industrial Information Systems
• /
• v.29 no.4
• /
• pp.13-24
• /
• 2024

In this paper, we analyzed the performance of normalization according to various neural network models and input-output structures. For the analysis, a simulation-based dataset for noise environments with homogeneous and up to three interfering signals was used. As a result, the end-to-end structure that directly outputs noise variance showed superior performance when using a 1-D convolutional neural network and BiLSTM model, and was analyzed to be particularly robust against interference signals. This is because the 1-D convolutional neural network and bidirectional long short-term memory models have stronger inductive bias than the multilayer perceptron and transformer models. The analysis of this paper are expected to be used as a useful reference for future research on deep learning-based normalization.

• Smart Structures and Systems
• /
• v.30 no.6
• /
• pp.613-626
• /
• 2022

Guaranteeing the quality and integrity of structural health monitoring (SHM) data is very important for an effective assessment of structural condition. However, sensory system may malfunction due to sensor fault or harsh operational environment, resulting in multiple types of data anomaly existing in the measured data. Efficiently and automatically identifying anomalies from the vast amounts of measured data is significant for assessing the structural conditions and early warning for structural failure in SHM. The major challenges of current automated data anomaly detection methods are the imbalance of dataset categories. In terms of the feature of actual anomalous data, this paper proposes a data anomaly detection method based on data-level and deep learning technique for SHM of civil engineering structures. The proposed method consists of a data balancing phase to prepare a comprehensive training dataset based on data-level technique, and an anomaly detection phase based on a sophisticatedly designed network. The advanced densely connected convolutional network (DenseNet) and Transformer encoder are embedded in the specific network to facilitate extraction of both detail and global features of response data, and to establish the mapping between the highest level of abstractive features and data anomaly class. Numerical studies on a steel frame model are conducted to evaluate the performance and noise immunity of using the proposed network for data anomaly detection. The applicability of the proposed method for data anomaly classification is validated with the measured data of a practical supertall structure. The proposed method presents a remarkable performance on data anomaly detection, which reaches a 95.7% overall accuracy with practical engineering structural monitoring data, which demonstrates the effectiveness of data balancing and the robust classification capability of the proposed network.

• Journal of Broadcast Engineering
• /
• v.28 no.2
• /
• pp.178-184
• /
• 2023

Finding texts in general scene images and recognizing their contents is a very important task that can be used as a basis for robot vision, visual assistance, and so on. However, for the low-resolution text images, the degradations, such as noise or blur included in text images, are more noticeable, which leads to severe performance degradation of text recognition accuracy. In this paper, we propose a new Korean text image super-resolution based on a Transformer-based model, which generally shows higher performance than convolutional neural networks. In the experiments, we show that text recognition accuracy for Korean text images can be improved when our proposed text image super-resolution method is used. We also propose a new Korean text image dataset for training our model, which contains massive HR-LR Korean text image pairs.

• Smart Structures and Systems
• /
• v.31 no.4
• /
• pp.351-363
• /
• 2023

Visual structural inspections are an inseparable part of post-earthquake damage assessments. With unmanned aerial vehicles (UAVs) establishing a new frontier in visual inspections, there are major computational challenges in processing the collected massive amounts of high-resolution visual data. We propose twin deep learning models that can provide accurate high-resolution structural components and damage segmentation masks efficiently. The traditional approach to cope with high memory computational demands is to either uniformly downsample the raw images at the price of losing fine local details or cropping smaller parts of the images leading to a loss of global contextual information. Therefore, our twin models comprising Trainable Resizing for high-resolution Segmentation Network (TRS-Net) and DmgFormer approaches the global and local semantics from different perspectives. TRS-Net is a compound, high-resolution segmentation architecture equipped with learnable downsampler and upsampler modules to minimize information loss for optimal performance and efficiency. DmgFormer utilizes a transformer backbone and a convolutional decoder head with skip connections on a grid of crops aiming for high precision learning without downsizing. An augmented inference technique is used to boost performance further and reduce the possible loss of context due to grid cropping. Comprehensive experiments have been performed on the 3D physics-based graphics models (PBGMs) synthetic environments in the QuakeCity dataset. The proposed framework is evaluated using several metrics on three segmentation tasks: component type, component damage state, and global damage (crack, rebar, spalling). The models were developed as part of the 2^nd International Competition for Structural Health Monitoring.

• Journal of IKEEE
• /
• v.28 no.3
• /
• pp.352-359
• /
• 2024

This paper proposes a CNN-ViT hybrid model that automatically evaluates the aesthetic quality of images by combining local and global features. In this approach, CNN is used to extract local features such as color and object placement, while ViT is employed to analyze the aesthetic value of the image by reflecting global features. Color composition is derived by extracting the primary colors from the input image, creating a color palette, and then passing it through the CNN. The Rule of Thirds is quantified by calculating how closely objects in the image are positioned near the thirds intersection points. These values provide the model with critical information about the color balance and spatial harmony of the image. The model then analyzes the relationship between these factors to predict scores that align closely with human judgment. Experimental results on the AADB image database show that the proposed model achieved a Spearman's Rank Correlation Coefficient (SRCC) of 0.716, indicating more consistent rank predictions, and a Pearson Correlation Coefficient (LCC) of 0.72, which is 2~4% higher than existing models.