• 제목/요약/키워드: Attention module

검색결과 240건 처리시간 0.03초

Time-Series Forecasting Based on Multi-Layer Attention Architecture

  • Na Wang;Xianglian Zhao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권1호
    • /
    • pp.1-14
    • /
    • 2024
  • Time-series forecasting is extensively used in the actual world. Recent research has shown that Transformers with a self-attention mechanism at their core exhibit better performance when dealing with such problems. However, most of the existing Transformer models used for time series prediction use the traditional encoder-decoder architecture, which is complex and leads to low model processing efficiency, thus limiting the ability to mine deep time dependencies by increasing model depth. Secondly, the secondary computational complexity of the self-attention mechanism also increases computational overhead and reduces processing efficiency. To address these issues, the paper designs an efficient multi-layer attention-based time-series forecasting model. This model has the following characteristics: (i) It abandons the traditional encoder-decoder based Transformer architecture and constructs a time series prediction model based on multi-layer attention mechanism, improving the model's ability to mine deep time dependencies. (ii) A cross attention module based on cross attention mechanism was designed to enhance information exchange between historical and predictive sequences. (iii) Applying a recently proposed sparse attention mechanism to our model reduces computational overhead and improves processing efficiency. Experiments on multiple datasets have shown that our model can significantly increase the performance of current advanced Transformer methods in time series forecasting, including LogTrans, Reformer, and Informer.

Improving Transformer with Dynamic Convolution and Shortcut for Video-Text Retrieval

  • Liu, Zhi;Cai, Jincen;Zhang, Mengmeng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제16권7호
    • /
    • pp.2407-2424
    • /
    • 2022
  • Recently, Transformer has made great progress in video retrieval tasks due to its high representation capability. For the structure of a Transformer, the cascaded self-attention modules are capable of capturing long-distance feature dependencies. However, the local feature details are likely to have deteriorated. In addition, increasing the depth of the structure is likely to produce learning bias in the learned features. In this paper, an improved Transformer structure named TransDCS (Transformer with Dynamic Convolution and Shortcut) is proposed. A Multi-head Conv-Self-Attention module is introduced to model the local dependencies and improve the efficiency of local features extraction. Meanwhile, the augmented shortcuts module based on a dual identity matrix is applied to enhance the conduction of input features, and mitigate the learning bias. The proposed model is tested on MSRVTT, LSMDC and Activity-Net benchmarks, and it surpasses all previous solutions for the video-text retrieval task. For example, on the LSMDC benchmark, a gain of about 2.3% MdR and 6.1% MnR is obtained over recently proposed multimodal-based methods.

다결정 실리콘 PV모듈의 하절기 표면온도 예측을 위한 알고리즘 검토 및 외부인자별 영향 평가 (Evaluation on Calculation Algorithms for Polycrystalline Silicon PV Module Surface Temperatures by Varying External Factors during the Summer Period)

  • 정동은;염규환;이찬욱;도성록
    • 대한건축학회논문집:구조계
    • /
    • 제35권8호
    • /
    • pp.177-184
    • /
    • 2019
  • Recently, electric power usages and peak loads from buildings are increasing due to higher outdoor air temperatures and/or abnormal climate during the summer period. As one of the eco-friendly measures, a renewable energy system has been received much attention. Particularly, interest on a photovoltaic (PV) system using solar energy has been rapidly increasing in a building sector due to its broad applicability. In using the PV system, one of important factors is the PV efficiency. The normal PV efficiency is determined based on the STC(Standard Test Condition) and the NOCT(Nominal Operating Cell Temperature) performance test. However, the actual PV efficiency is affected by the temperature change at the module surface. Especially, higher module temperatures generally reduce the PV efficiency, and it leads to less power generation from the PV system. Therefore, the analysis of the relation between the module temperature and PV efficiency is required to evaluate the PV performance during the summer period. This study investigates existing algorithms for calculating module surface temperatures and analyzes resultant errors with the algorithms by comparing the measured module temperatures.

10kW급 지붕재용 태양전지모듈 실증연구 (Demonstration Study of 10kW Poly Metal Panel integrated PV Module)

  • 이소미;노지희;주만식
    • 한국신재생에너지학회:학술대회논문집
    • /
    • 한국신재생에너지학회 2007년도 추계학술대회 논문집
    • /
    • pp.246-249
    • /
    • 2007
  • The application of photovoltaics into building as integrated building components has been paid more attention worldwide. Photovoltaics or solar electric modules are sol id state devices, directly converting solar radiation into electricity; the process does not require fuel and any moving parts, and produce no pollutants. And the prefab building method is very effective because the pre- manufactured building components is simply assembled to making up buildings in the construction fields especially the sandwich panel. So, this paper describes a design and performance test of the 10kW poly metal pv module(pmpp) system. It is concluded that the prediction of BIPV system's performance should be based on the more accurate PV module installation.

  • PDF

전기적인 특성을 고려한 태양전지모듈의 노화 분석 (Degradation Analysis of PV Module Considering Electrical Characteristics)

  • 김승태;강기환;박지홍;안형근;유권종;한득영
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2008년도 제39회 하계학술대회
    • /
    • pp.1110-1111
    • /
    • 2008
  • The life time of PV module is semi-permanent. But, because of installation and module fabrication process, its important part can not be finished. In this paper, we analyze 15 years old modules made from different company. Among the PV modules, the maximum power drop ratio was 12.23% minimum and 80.63% maximum. Also the effect of solar cell's short circuit current difference was analyzed. The PV module exposed about 65days, its the maximum power drop ratio was 1.29% minimum and 23.43% maximum. It is for reduction of current value. And the reason for current reduction was due to reduction of parallel resistance of solar cell. To prevent early degradation, it is need to have attention to fabrication, installation and maintenance.

  • PDF

지붕재 일체형 태양전지 모듈의 개발에 따른 내구성 평가 (조립식 건축시스템을 중심으로) (A Study on the Development of Roof Integrated PV Module (Focused on the Prefab Building System))

  • 이소미;노지희;이응직
    • KIEAE Journal
    • /
    • 제6권4호
    • /
    • pp.17-24
    • /
    • 2006
  • The application of photovoltaics into building as integrated building components has been paid more attention worldwide. Photovoltaics or solar electric modules are solid state devices, directly converting solar radiation into electricity; the process does not require fuel and any moving parts, and produce no pollutants. And the prefab building method is very effective because the pre- manufactured building components is simply assembled to making up buildings in the construction fields especially the sandwich panel. Architecture considerations for the integration of PV module to building envelope such as building structure, construction type, safety, regulation, maintenance etc. have been carefully refelected from the early stage of BIPV module design. Trial product of BIPV module are manufactured and sample construction details for demonstration building are purposed. Therefore, this paper intends to advanced its practical use by proposing how to get integrated PV system which can be applied to prefab building material, and how to apply it.

파워모듈의 TLP 접합 및 와이어 본딩 (TLP and Wire Bonding for Power Module)

  • 강혜준;정재필
    • 마이크로전자및패키징학회지
    • /
    • 제26권4호
    • /
    • pp.7-13
    • /
    • 2019
  • Power module is getting attention from electronic industries such as solar cell, battery and electric vehicles. Transient liquid phase (TLP) boding, sintering with Ag and Cu powders and wire bonding are applied to power module packaging. Sintering is a popular process but it has some disadvantages such as high cost, complex procedures and long bonding time. Meanwhile, TLP bonding has lower bonding temperature, cost effectiveness and less porosity. However, it also needs to improve ductility of the intermetallic compounds (IMCs) at the joint. Wire boding is also an important interconnection process between semiconductor chip and metal lead for direct bonded copper (DBC). In this study, TLP bonding using Sn-based solders and wire bonding process for power electronics packaging are described.

전압원 멀티레벨 컨버터 밸브 시험회로 연구 (Voltage source multilevel module converter valve test circuit research)

  • 원진;이진희;정택선;백승택
    • 전력전자학회:학술대회논문집
    • /
    • 전력전자학회 2014년도 전력전자학술대회 논문집
    • /
    • pp.79-80
    • /
    • 2014
  • Voltage source multilevel module converter attracts more and more attention recently. The core component of the voltage source multilevel module converter is the valve based on IGBT. So the test circuit for the valve is very important, reliable test method can guarantee the converter valve design meet the operation requirement. This paper analyzes the valve voltage and current stress during the operation, and according to IEC standard test requirement, object, condition, introduces a kind of test circuit. Finally, through the simulation model, to verify the test circuit can provide the proper test condition for the voltage source multilevel module converter valve.

  • PDF

합성곱 신경망 기반 밝기-색상 정보를 이용한 얼굴 위변조 검출 방법 (Face Anti-Spoofing Based on Combination of Luminance and Chrominance with Convolutional Neural Networks)

  • 김은석;김원준
    • 방송공학회논문지
    • /
    • 제24권6호
    • /
    • pp.1113-1121
    • /
    • 2019
  • 본 논문에서는 얼굴의 밝기와 색상 정보를 함께 이용한 합성곱 신경망 기반의 얼굴 위변조 검출 방법을 제안한다. 제안하는 방법은 적층된 합성곱 신경망과 보조 신경망을 이용하여 실제 얼굴과 위변조된 얼굴의 밝기 특징과 색상 특징을 독립적으로 추출한다. 기존의 방법과는 달리, 본 논문에서는 추출된 특징을 단순 결합(Concatenation)하는 것이 아니라 주의 모듈(Attention Module)을 이용하여 적응적(Adaptively)으로 조합할 수 있도록 하였다. 또한, 효과적인 분류기 학습을 위하여 대비 손실함수(Contrast Loss Function)를 새롭게 제안하였는데, 대비 손실함수는 동일 클래스 내의 특징 간의 차이는 최소화 시키고 서로 다른 클래스의 특징 간의 차이는 최대화 시킴으로써 특징의 분별력을 높인다. 다양한 실험을 통해 본 논문에서 제안하는 방법이 기존 얼굴 위변조 검출 방법 대비 개선된 성능을 보임을 확인하고 그 결과를 분석한다.

딥러닝을 이용한 실시간 말벌 분류 시스템 (Real Time Hornet Classification System Based on Deep Learning)

  • 정윤주;이영학;이스라필 안사리;이철희
    • 전기전자학회논문지
    • /
    • 제24권4호
    • /
    • pp.1141-1147
    • /
    • 2020
  • 말벌 종은 모양이 매우 유사하기 때문에 비전문가가 분류하기 어렵고, 객체의 크기가 작고 빠르게 움직이기 때문에 실시간으로 탐지하여 종을 분류하는 것은 더욱 어렵다. 본 논문에서는 바운딩 박스를 이용한 딥러닝 알고리즘을 기반으로 말벌 종을 실시간으로 분류하는 시스템을 개발하였다. 훈련 영상의 레이블링 작업 시 바운딩 박스 안에 포함되는 배경 영역을 최소화하기 위하여 말벌의 머리와 몸통 부분만을 선택하는 방법을 제안한다. 또한 실시간으로 말벌을 탐지하고 그 종을 분류할 수 있는 최선의 알고리즘을 찾기 위하여 기존의 바운딩 박스 기반 객체 인식 알고리즘들을 실험을 통하여 비교한다. 실험 결과 컨볼루션 레이어의 활성함수로 mish 함수를 적용하고, 객체 검출 블록 전에 공간집중모듈(Spatial Attention Module, SAM)을 적용한 YOLOv4 모델을 사용하여 말벌 영상을 테스트한 경우 평균 97.89%의 정밀도(Precision)와 98.69%의 재현율(Recall)을 나타내었다.