• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.46 no.3
• /
• pp.123-138
• /
• 2023

The Fourth Industrial Revolution and sensor technology have led to increased utilization of sensor data. In our modern society, data complexity is rising, and the extraction of valuable information has become crucial with the rapid changes in information technology (IT). Recurrent neural networks (RNN) and long short-term memory (LSTM) models have shown remarkable performance in natural language processing (NLP) and time series prediction. Consequently, there is a strong expectation that models excelling in NLP will also excel in time series prediction. However, current research on Transformer models for time series prediction remains limited. Traditional RNN and LSTM models have demonstrated superior performance compared to Transformers in big data analysis. Nevertheless, with continuous advancements in Transformer models, such as GPT-2 (Generative Pre-trained Transformer 2) and ProphetNet, they have gained attention in the field of time series prediction. This study aims to evaluate the classification performance and interval prediction of remaining useful life (RUL) using an advanced Transformer model. The performance of each model will be utilized to establish a health index (HI) for cutting blades, enabling real-time monitoring of machine health. The results are expected to provide valuable insights for machine monitoring, evaluation, and management, confirming the effectiveness of advanced Transformer models in time series analysis when applied in industrial settings.

• ETRI Journal
• /
• v.44 no.5
• /
• pp.794-804
• /
• 2022

Transformer models have had a great impact on natural language processing (NLP) in recent years by realizing outstanding and efficient contextualized language models. Recent studies have used transformer-based language models for various NLP tasks, including Persian named entity recognition (NER). However, in complex tasks, for example, NER, it is difficult to determine which contextualized embedding will produce the best representation for the tasks. Considering the lack of comparative studies to investigate the use of different contextualized pretrained models with sequence modeling classifiers, we conducted a comparative study about using different classifiers and embedding models. In this paper, we use different transformer-based language models tuned with different classifiers, and we evaluate these models on the Persian NER task. We perform a comparative analysis to assess the impact of text representation and text classification methods on Persian NER performance. We train and evaluate the models on three different Persian NER datasets, that is, MoNa, Peyma, and Arman. Experimental results demonstrate that XLM-R with a linear layer and conditional random field (CRF) layer exhibited the best performance. This model achieved phrase-based F-measures of 70.04, 86.37, and 79.25 and word-based F scores of 78, 84.02, and 89.73 on the MoNa, Peyma, and Arman datasets, respectively. These results represent state-of-the-art performance on the Persian NER task.

• ETRI Journal
• /
• v.46 no.1
• /
• pp.137-153
• /
• 2024

This study introduces a new approach in Korean morphological analysis combining dictionary-based techniques with Transformer-based deep learning models. The key innovation is the use of a BERT-based reranking system, significantly enhancing the accuracy of traditional morphological analysis. The method generates multiple suboptimal paths, then employs BERT models for reranking, leveraging their advanced language comprehension. Results show remarkable performance improvements, with the first-stage reranking achieving over 20% improvement in error reduction rate compared with existing models. The second stage, using another BERT variant, further increases this improvement to over 30%. This indicates a significant leap in accuracy, validating the effectiveness of merging dictionary-based analysis with contemporary deep learning. The study suggests future exploration in refined integrations of dictionary and deep learning methods as well as using probabilistic models for enhanced morphological analysis. This hybrid approach sets a new benchmark in the field and offers insights for similar challenges in language processing applications.

• Journal of international Conference on Electrical Machines and Systems
• /
• v.3 no.1
• /
• pp.78-82
• /
• 2014

Power Transformers with more than one secondary winding are not uncommon in industrial applications. But new classes of applications where very large number of independent secondaries are used are becoming popular in controlled converters for medium and high voltage applications. Cascade H-bridge medium voltage drives and Pulse Step Modulation (PSM) based high voltage power supplies are such applications. Regulated high voltage power supplies (Fig. 1) with 35-100 kV, 5-10 MW output range with very fast dynamics (${\mu}S$ order) uses such transformers. Such power supplies are widely used in fusion research. Here series connection of isolated voltage sources with conventional switching semiconductor devices is achieved by large number of separate transformers or by single unit of multi-secondary transformer. Naturally, a transformer having numbers of secondary windings (~40) on single core is the preferred solution due to space and cost considerations. For design and simulation analysis of such a power supply, the model of a multi-secondary transformer poses special problem to any circuit analysis software as many simulation softwares provide transformer models with limited number (3-6) of secondary windings. Multi-Secondary transformer models with 3 different schemes are available. A comparison of test results from a practical Multi-secondary transformer with a simulation model using magnetic component is found to describe the behavior closer to observed test results. Earlier models assumed magnetising inductance in a linear loss less core model although in actual it is saturable core made-up of CRGO steel laminations. This article discusses a more detailed representation of flux coupled magnetic model with saturable core properties to simulate actual transformers very close to its observed parameters in test and actual usage.

• Journal of Electrical Engineering and Technology
• /
• v.1 no.3
• /
• pp.302-307
• /
• 2006

The purpose of this paper is to develop an improved three-phase transformer model for ATP and parameter estimation methods that can efficiently utilize the limited available information such as factory test reports. In this paper, improved topologically-correct duality-based models are developed for three-phase autotransformers having shell-form cores. The problem in the implementation of detailed models is the lack of complete and reliable data. Therefore, parameter estimation methods are developed to determine the parameters of a given model in cases where available information is incomplete. The transformer nameplate data is required and relative physical dimensions of the core are estimated. The models include a separate representation of each segment of the core, including hysteresis of the core, ${\lambda}-i$ saturation characteristic and core loss.

• Journal of Broadcast Engineering
• /
• v.27 no.7
• /
• pp.1011-1020
• /
• 2022

Recent high-performance panoptic segmentation models are based on transformer architectures. However, transformer-based panoptic segmentation methods are basically slower than convolution-based methods, since the attention mechanism in the transformer requires quadratic complexity w.r.t. image resolution. Also, sine and cosine computation for positional embedding in the transformer also yields a bottleneck for computation time. To address these problems, we adopt three modules to speed up the inference runtime of the transformer-based panoptic segmentation. First, we perform channel-level reduction using depth-wise separable convolution for inputs of the transformer decoder. Second, we replace sine and cosine-based positional encoding with convolution operations, called conv-embedding. We also apply a separable self-attention to the transformer encoder to lower quadratic complexity to linear one for numbers of image pixels. As result, the proposed model achieves 44% faster frame per second than baseline on ADE20K panoptic validation dataset, when we use all three modules.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.11
• /
• pp.81-87
• /
• 2023

Intrusion detection systems that learn metadata of network packets have been proposed recently. However these approaches require time to analyze packets to generate metadata for model learning, and time to pre-process metadata before learning. In addition, models that have learned specific metadata cannot detect intrusion by using original packets flowing into the network as they are. To address the problem, this paper propose a natural language processing-based intrusion detection system that detects intrusions by learning the packet payload as a single sentence without an additional conversion process. To verify the performance of our approach, we utilized the UNSW-NB15 and Transformer models. First, the PCAP files of the dataset were labeled, and then two Transformer (BERT, DistilBERT) models were trained directly in the form of sentences to analyze the detection performance. The experimental results showed that the binary classification accuracy was 99.03% and 99.05%, respectively, which is similar or superior to the detection performance of the techniques proposed in previous studies. Multi-class classification showed better performance with 86.63% and 86.36%, respectively.

• Proceedings of the KIEE Conference
• /
• 2006.07a
• /
• pp.103-104
• /
• 2006

Since the first proposition of IEC61850 object model at 1993, many questions about making a seamless model have been issued. the reason which they have worry about is that the functions of the equipment are supposed to be changed properly and new equipment and scheme are need to be introduced according to user's application. To handle those issues, TC57 which is a IEC committee for power control and communication has continuously updated the object model. Nowadays along with the new object model involving power quality, distribution resource and wind power, the committee has a plan to announce the revision of IEC61850-7-4. In the study, authors will present the prediction and diagnosis object models for transformer. Transformer models for protection and control have already been dealt with in the international standard but the models for prediction and diagnosis have never mentioned until now. Designing the prediction and diagnosis functions with the existing IEC61850-7-4, it'll be shown what is a proper object model for prediction and diagnosis.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.18 no.4
• /
• pp.693-700
• /
• 2023

Sentiment analysis, a branch of natural language processing that classifies and identifies subjective opinions and emotions in text documents as positive or negative, can be used for various promotions and services through customer preference analysis. To this end, recent research has been conducted utilizing various techniques in machine learning and deep learning. In this study, we propose an optimal language model by comparing the accuracy of sentiment analysis for movie, product, and game reviews using existing RNN-based models and recent Transformer-based language models. In our experiments, LMKorBERT and GPT3 showed relatively good accuracy among the models pre-trained on the Korean corpus.

• The Journal of Information Systems
• /
• v.32 no.4
• /
• pp.149-162
• /
• 2023

Purpose In the textile industry, fabric defects significantly impact product quality and consumer satisfaction. This research seeks to enhance defect detection by developing a transformer-based deep learning image segmentation model for learning high-dimensional image features, overcoming the limitations of traditional image classification methods. Design/methodology/approach This study utilizes the ZJU-Leaper dataset to develop a model for detecting defects in fabrics. The ZJU-Leaper dataset includes defects such as presses, stains, warps, and scratches across various fabric patterns. The dataset was built using the defect labeling and image files from ZJU-Leaper, and experiments were conducted with deep learning image segmentation models including Deeplabv3, SegformerB0, SegformerB1, and Dinov2. Findings The experimental results of this study indicate that the SegformerB1 model achieved the highest performance with an mIOU of 83.61% and a Pixel F1 Score of 81.84%. The SegformerB1 model excelled in sensitivity for detecting fabric defect areas compared to other models. Detailed analysis of its inferences showed accurate predictions of diverse defects, such as stains and fine scratches, within intricated fabric designs.