• Title/Summary/Keyword: Transformer Models

Search Result 157, Processing Time 0.021 seconds

Cloud Detection from Sentinel-2 Images Using DeepLabV3+ and Swin Transformer Models (DeepLabV3+와 Swin Transformer 모델을 이용한 Sentinel-2 영상의 구름탐지)

  • Kang, Jonggu;Park, Ganghyun;Kim, Geunah;Youn, Youjeong;Choi, Soyeon;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.6_2
    • /
    • pp.1743-1747
    • /
    • 2022
  • Sentinel-2 can be used as proxy data for the Korean Compact Advanced Satellite 500-4 (CAS500-4), also known as Agriculture and Forestry Satellite, in terms of spectral wavelengths and spatial resolution. This letter examined cloud detection for later use in the CAS500-4 based on deep learning technologies. DeepLabV3+, a traditional Convolutional Neural Network (CNN) model, and Shifted Windows (Swin) Transformer, a state-of-the-art (SOTA) Transformer model, were compared using 22,728 images provided by Radiant Earth Foundation (REF). Swin Transformer showed a better performance with a precision of 0.886 and a recall of 0.875, which is a balanced result, unbiased between over- and under-estimation. Deep learning-based cloud detection is expected to be a future operational module for CAS500-4 through optimization for the Korean Peninsula.

Iron Core Design of 3-Phase 40MVA HTS Power Transformer Considering Voltages per Turn

  • Lee, Chan-joo;Seok, Bok-yeol
    • KIEE International Transaction on Electrical Machinery and Energy Conversion Systems
    • /
    • v.4B no.2
    • /
    • pp.54-58
    • /
    • 2004
  • This paper presents the iron core design method of a high temperature superconducting (HTS) transformer considering voltages per turn (V/T). In this research, solenoid type HTS coils were selected for low voltage (LV) winding and double pancake coils for high voltage (HV) winding, just as in conventional large power transformers. V/T is one of the most fundamental elements used in designing transformers, as it decides the core cross sectional area and the number of primary and secondary winding turns. By controlling the V/T, the core dimension and core loss can be changed diversely. The leakage flux is another serious consideration in core design. The magnetic field perpendicular to the HTS wire causes its critical current to fall rapidly as the magnitude of the field increases slowly. Therefore in the design of iron core as well as superconducting windings, contemplation of leakage flux should be preceded. In this paper, the relationship between the V/T and core loss was observed and also, through computational calculations, the leakage magnetic fields perpendicular to the windings were found and their critical current decrement effects were considered in relation to the core design. The % impedance was calculated by way of the numerical method. Finally, various models were suggested.

Time-Series Forecasting Based on Multi-Layer Attention Architecture

  • Na Wang;Xianglian Zhao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.1
    • /
    • pp.1-14
    • /
    • 2024
  • Time-series forecasting is extensively used in the actual world. Recent research has shown that Transformers with a self-attention mechanism at their core exhibit better performance when dealing with such problems. However, most of the existing Transformer models used for time series prediction use the traditional encoder-decoder architecture, which is complex and leads to low model processing efficiency, thus limiting the ability to mine deep time dependencies by increasing model depth. Secondly, the secondary computational complexity of the self-attention mechanism also increases computational overhead and reduces processing efficiency. To address these issues, the paper designs an efficient multi-layer attention-based time-series forecasting model. This model has the following characteristics: (i) It abandons the traditional encoder-decoder based Transformer architecture and constructs a time series prediction model based on multi-layer attention mechanism, improving the model's ability to mine deep time dependencies. (ii) A cross attention module based on cross attention mechanism was designed to enhance information exchange between historical and predictive sequences. (iii) Applying a recently proposed sparse attention mechanism to our model reduces computational overhead and improves processing efficiency. Experiments on multiple datasets have shown that our model can significantly increase the performance of current advanced Transformer methods in time series forecasting, including LogTrans, Reformer, and Informer.

Robust transformer-based anomaly detection for nuclear power data using maximum correntropy criterion

  • Shuang Yi;Sheng Zheng;Senquan Yang;Guangrong Zhou;Junjie He
    • Nuclear Engineering and Technology
    • /
    • v.56 no.4
    • /
    • pp.1284-1295
    • /
    • 2024
  • Due to increasing operational security demands, digital and intelligent condition monitoring of nuclear power plants is becoming more significant. However, establishing an accurate and effective anomaly detection model is still challenging. This is mainly because of data characteristics of nuclear power data, including the lack of clear class labels combined with frequent interference from outliers and anomalies. In this paper, we introduce a Transformer-based unsupervised model for anomaly detection of nuclear power data, a modified loss function based on the maximum correntropy criterion (MCC) is applied in the model training to improve the robustness. Experimental results on simulation datasets demonstrate that the proposed Trans-MCC model achieves equivalent or superior detection performance to the baseline models, and the use of the MCC loss function is proven can obviously alleviate the negative effect of outliers and anomalies in the training procedure, the F1 score is improved by up to 0.31 compared to Trans-MSE on a specific dataset. Further studies on genuine nuclear power data have verified the model's capability to detect anomalies at an earlier stage, which is significant to condition monitoring.

Korean Lip-Reading: Data Construction and Sentence-Level Lip-Reading (한국어 립리딩: 데이터 구축 및 문장수준 립리딩)

  • Sunyoung Cho;Soosung Yoon
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.27 no.2
    • /
    • pp.167-176
    • /
    • 2024
  • Lip-reading is the task of inferring the speaker's utterance from silent video based on learning of lip movements. It is very challenging due to the inherent ambiguities present in the lip movement such as different characters that produce the same lip appearances. Recent advances in deep learning models such as Transformer and Temporal Convolutional Network have led to improve the performance of lip-reading. However, most previous works deal with English lip-reading which has limitations in directly applying to Korean lip-reading, and moreover, there is no a large scale Korean lip-reading dataset. In this paper, we introduce the first large-scale Korean lip-reading dataset with more than 120 k utterances collected from TV broadcasts containing news, documentary and drama. We also present a preprocessing method which uniformly extracts a facial region of interest and propose a transformer-based model based on grapheme unit for sentence-level Korean lip-reading. We demonstrate that our dataset and model are appropriate for Korean lip-reading through statistics of the dataset and experimental results.

Semantic Segmentation of the Habitats of Ecklonia Cava and Sargassum in Undersea Images Using HRNet-OCR and Swin-L Models (HRNet-OCR과 Swin-L 모델을 이용한 조식동물 서식지 수중영상의 의미론적 분할)

  • Kim, Hyungwoo;Jang, Seonwoong;Bak, Suho;Gong, Shinwoo;Kwak, Jiwoo;Kim, Jinsoo;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.5_3
    • /
    • pp.913-924
    • /
    • 2022
  • In this paper, we presented a database construction of undersea images for the Habitats of Ecklonia cava and Sargassum and conducted an experiment for semantic segmentation using state-of-the-art (SOTA) models such as High Resolution Network-Object Contextual Representation (HRNet-OCR) and Shifted Windows-L (Swin-L). The result showed that our segmentation models were superior to the existing experiments in terms of the 29% increased mean intersection over union (mIOU). Swin-L model produced better performance for every class. In particular, the information of the Ecklonia cava class that had small data were also appropriately extracted by Swin-L model. Target objects and the backgrounds were well distinguished owing to the Transformer backbone better than the legacy models. A bigger database under construction will ensure more accuracy improvement and can be utilized as deep learning database for undersea images.

Harmonic Analysis Model based on PSCAD/EMTDC

  • Lee, Han-Min;Lee, Chang-Mu;Jang, Gil-Soo;Kwon, Sae-Hyuk
    • KIEE International Transactions on Power Engineering
    • /
    • v.4A no.3
    • /
    • pp.115-121
    • /
    • 2004
  • This paper presents the model for an AC electric railway system using the PSCAD/EMTDC program. It is composed of a scott-transformer, an auto-transformer, catenary and electric trains, etc. After obtaining the models of the fundamental elements describing the AC electric railway system and its behavior, we have analyzed and tested an actual AC electric railway system focused on the amplification of harmonic current to verify the proposed model. The simulation results from the proposed approach and the measurement data from the test are described.

Interlaminar Flux Density Distribution at Joints of Overlapping Stacked Electrical Steel and Amorphous Ribbons

  • Erdem, Sezer;Derebasi, Naim
    • Journal of Magnetics
    • /
    • v.15 no.4
    • /
    • pp.190-193
    • /
    • 2010
  • The design of joints in a transformer core significantly affects the transformer's efficiency. Air gaps cause variations in the flux distribution at the joints of the laminations, which depend on the geometry. Two similar samples consisting of electrical steel strips and amorphous ribbons were made. The spatial flux distributions were determined using an array of search coils for each sample. 2D models of these samples were created and examined by finite element analysis. The magnetic flux distribution for each lamination in the samples was computed. The results show that the flux density in amorphous ribbons above and below the air gap starts to approach saturation at lower flux density levels than for electrical steel. The flux density measured using the search coil under the air gap is increased in amorphous ribbons and decreased in the electrical steel with increasing frequency.

Analysis of the amplification of Harmonic Current Using EMTDC in Kyongbu High-Speed Railway (EMTDC를 이용한 경부고속철도 고조파 전류 확대율 해석)

  • Lee H.M;Lee C.M
    • Proceedings of the KIEE Conference
    • /
    • summer
    • /
    • pp.1379-1381
    • /
    • 2004
  • This paper presents the AC electric railway system modeling using PSCAD/EMTDC program. This system model is composed of the scott-transformer the auto-transformer. the running rails. the protection wires, the feeders. the catenary and contact wires, etc. After obtaining the models of the fundamental elements describing the AC electric railway system and its behavior, we have analyzed and tested real traction power feeding system focused on the amplification of harmonic current to verify the proposed model. The simulation results from the proposed approach and the measurement data from the test are described in the paper.

  • PDF

The Analysis for the Simulational Characteristics of the Electronic CT & PT using EMTP (EMTP를 이용한 전자식 CT, PT의 시뮬레이션 특성 분석)

  • Shin Jung Eun;Kim Jae Chul;Song Seung Youp;Kwon Young Mok;Kim On Suk;Choo Dong Wook
    • Proceedings of the KIEE Conference
    • /
    • summer
    • /
    • pp.310-312
    • /
    • 2004
  • Generally, instrument transformer of the core type have been used for measuring current. Because instrument transformer has defects, electronic instrument was appeasred. This paper is introduced a basic concept and thoery of electronic instrument and described the the simulation of core type CT and no saturation CT. The evaluation of CT models implemented using EMTP(Electro Magnetic Transients Program) was carried out. The result of comparison both core type CT and no saturation CT are presented.

  • PDF