• Title/Summary/Keyword: Extract transform and load

Search Result 9, Processing Time 0.021 seconds

Short-term Electric Load Forecasting Based on Wavelet Transform and GMDH

  • Koo, Bon-Gil;Lee, Heung-Seok;Park, Juneho
    • Journal of Electrical Engineering and Technology
    • /
    • v.10 no.3
    • /
    • pp.832-837
    • /
    • 2015
  • The group method of data handling (GMDH) algorithm has proven to be a powerful and effective way to extract rules or polynomials from an electric load pattern. However, because it is nonstationary, the load pattern needs to be decomposed using a discrete wavelet transform. In addition, if a load pattern has a complicated curve pattern, GMDH should use a higher polynomial, which requires complex computing and consumes a lot of time. This paper suggests a method for short-term electric load forecasting that uses a wavelet transform and a GMDH algorithm. Case studies with the proposed algorithm were carried out for one-day-ahead forecasting of hourly electric loads using data during the years 2008-2011. To prove the effectiveness of our proposed approach, the results were evaluated and compared with those obtained by Holt-Winters method and artificial neural network. Our suggested method resulted in better performance than either comparison group.

The comparative study of IWRM between ETL and ELT (통합수자원관리의 ETL, ELT 비교 연구)

  • Soon Yeon Kim;Young Jin Won
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.392-392
    • /
    • 2023
  • 통합수자원관리 IWRM(Integrated Water Resources Management)를 위한 다각도의 연구가 진행되고 있다. 구축된 정보의 활용 증진, 정책적 측면 등이 어우러져 고려되어야 비로소 통합으로서의 가치가 확보될 것이다. 본 고에서는 그중 데이터 측면을 살펴본다. 데이터 관점에서도 정보의 수집, 유통, 품질 등 여러 관점의 주제가 존재한다. 예를 들어 유통을 위한 메타데이터는 ISO 19139의 채택에서 추가 보완할 부분이 무엇인지, 유통을 위한 포맷과 인터페이스는 XML과 OGC WFS를 채택시 충분한 호환성이 확보되는지 등 이 주요한 검토요소가 될 것이다. 본 고에서는 이러한 고민에 앞서, 초기 절차인 시스템 적재 과정의 ETL(Extract, Transform and Load)과 ELT(Extract, Load and Transform) 방법을 비교하고자 한다. 수자원 분야 데이터를 사용한 예시 사례를 통하여, 최근 데이터 산업 분야에서 ETL과 ELT 차이가 이슈화되고 있는 이유를 살펴보고, 온프레미스와 클라우드 상황에 대하여 두 가지 방법의 장단점을 비교함으로서 다양한 데이터를 직면하게 되는 IWRM에서의 데이터 적응에 대한 힌트를 얻고자 한다. 기본적으로는 ETL보다 ELT의 효율성이 높을 것으로 전망된다. 그러나 해당 변환 과정이 일회성인지 반복적인지 고려해야 한다. 또한 변환에 있어 지배적인 요소가 변환 프로세스의 복잡도인지, 레코드 건수인지도 유불리에 영향을 준다. 비용측면에서는 ETL보다 ELT의 저장 비용이 증가할 수 있고, 이는 온프레미스 DBMS에서 PaaS 유형의 DBMS로 전환되어감에 따라 더 큰 부담이 될 수 있다. 무엇보다 ETL, ELT에서는 요구되는 변환 기술의 종류가 다르다. 예를 들어 Desktop GIS 도구와 ANSI 표준 SQL 기술로 대표될 수 있으며 각 수행조직에서는 보유 인적자원의 기술 현황과 학습곡선 등을 고려해야 한다. 결론적으로 ETL, ELT의 비교에 있어서는 처리시간과 동시에, 비용과 인적 자원 그리고 전략적 측면도 함께 검토되어야 한다.

  • PDF

Relational Data Extraction and Transformation: A Study to Enhance Information Systems Performance

  • Forat Falih, Hasan;Muhamad Shahbani Abu, Bakar
    • Journal of information and communication convergence engineering
    • /
    • v.20 no.4
    • /
    • pp.265-272
    • /
    • 2022
  • The most effective method to improve information system capabilities is to enable instant access to several relational database sources and transform data with a logical structure into multiple target relational databases. There are numerous data transformation tools available; however, they typically contain fixed procedures that cannot be changed by the user, making it impossible to fulfill the near-real-time data transformation requirements. Furthermore, some tools cannot build object references or alter attribute constraints. There are various situations in which tool changes in data type cause conflicts and difficulties with data quality while transforming between the two systems. The R-programming language was extensively used throughout this study, and several different relational database structures were utilized to complete the proposed study. Experiments showed that the developed study can improve the performance of information systems by interacting with and exchanging data with various relational databases. The study addresses data quality issues, particularly the completeness and integrity dimensions of the data transformation processes.

A Study on High Fault Detection In Power System (전력계통의 고임피던스 고장 검출 기법에 관한 연구)

  • Yim, Wha-Yeong;Ryu, Chang-Wan;Ko, Jae-Ho
    • The Transactions of the Korean Institute of Electrical Engineers A
    • /
    • v.48 no.1
    • /
    • pp.16-21
    • /
    • 1999
  • The analysis of distribution line faults is essential to the proper protections of the power system. A high impedance fault test, which was carried in Korean electric power systems, it was found that a arcing phenomenon occurred during the high level portion of conductor voltage in each cycle. In this paper, we propose a new method for detection of high impedance faults, which uses the arcing fault current difference during high voltage and low voltage portion of conductor voltage waveform. To extract this difference, we diveded one cycle fault current into equal spanned four data windows according to the magnitude of voltage waveform and applied fast fourier transform(FFT) to each data window. The frequency spectrum of current wavefrom in each portion are used as the inputs of neural network and is trained to detect high impedance faults. The proposed method shows improved accuracy when applied to staged fault data and fault-like load.

  • PDF

Single-Phase Active Power Filter based on Digital Controller (디지털제어기를 기반으로 하는 단상 능동전력필터)

  • Bae, Byung-Yeol;Lee, Ji-Heon;Lee, Hye-Yeon;Ju, Young-Ah;Han, Byung-Moon;Park, Byung-Ju;Yoon, Dong-Chul
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.57 no.5
    • /
    • pp.789-796
    • /
    • 2008
  • This paper describes a single-phase active power filter based on a newly developed digital controller. The developed controller utilizes FFT(Fast Fourier Transform) algorithm to extract the reference signal from the load current, considering the phase-angle delay of each order of harmonics. Optimized technique was applied for whole control algorithm to implement the real-time operation of developed controller. The performance of developed controller for a single-phase active power filter was verified through computer simulations with PSCAD/EMTDC. The feasibility of hardware implementation was confirmed by building and testing a prototype. The developed digital controller for a single-phase active power filter can compensate the harmonic current generated by the power supply for digital equipment.

A Self-Service Business Intelligence System for Recommending New Crops (재배 작물 추천을 위한 셀프서비스 비즈니스 인텔리전스 시스템)

  • Kim, Sam-Keun;Kim, Kwang-Chae;Kim, Hyeon-Woo;Jeong, Woo-Jin;Ahn, Jae-Geun
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.22 no.3
    • /
    • pp.527-535
    • /
    • 2021
  • Traditional business intelligence (BI) systems have been used widely as tools for better decision-making on time. On the other hand, building a data warehouse (DW) for the efficient analysis of rapidly growing data is time-consuming and complex. In particular, the ETL (Extract, Transform, and Load) process required to build a data warehouse has become much more complex as the BI platform moves to a cloud environment. Various BI solutions based on the NoSQL database, such as MongoDB, have been proposed to overcome these ETL issues. Decision-makers want easy access to data without the help of IT departments or BI experts. Recently, self-service BI (SSBI) has emerged as a way to solve these BI issues. This paper proposes a self-service BI system with farming data using the MongoDB cloud as DW to support the selection of new crops by return-farmers. The proposed system includes functions to provide insights to decision-makers, including data visualization using MongoDB charts, reporting for advanced data search, and monitoring for real-time data analysis. Decision makers can access data directly in various ways and can analyze data in a self-service method using the functions of the proposed system.

Agro-Ecosystem Informatics for Rational Crop and Field Management - Remote Sensing, GIS and Modeling -

  • INOUE Yoshio
    • Proceedings of the Korean Society of Crop Science Conference
    • /
    • 2005.08a
    • /
    • pp.22-46
    • /
    • 2005
  • Spatial and timely information on crop and filed conditions is one of the most important basics for rational and efficient planning and management in agriculture. Remote sensing, GIS, and modeling are powerful tools for such applications. This paper presents an overview of the state of the art in remote sensing of crop and field conditions with some case studies. It is also shown that a synergistic linkage between process-based models and remote sensing signatures enables us to estimate the multiple crop/ecosystem variables at a dynamic mode. Remotely sensed information can greatly reduce the uncertainty of simulation models by compensating for insufficient availability of data or parameters. This synergistic approach allows the effective use of infrequent and multi-source remote sensing data for estimating important ecosystem variables such as biomass growth and ecosystem $CO_2$ flux. This paper also shows a geo-spatial information system that enables us to integrate, search, extract, process, transform, and calculate any part of the data based on ID#, attributes, and/or by river-basin boundary, administrative boundary, or boundaries of arbitrary shape/size all over Japan. A case study using the system demonstrates that the nitrogen load from fertilizer was closely related to nitrate concentration of groundwater. The combined use of remote sensing, GIS and modeling would have great potential for various agro-ecosystem applications.

  • PDF

Analysis of Engine Operation Condition by Using Coastdown Test under Gear Engaged Condition (기어 물림 상태의 타행 주행 저항을 이용한 엔진 운전 조건의 분석)

  • Shim, Beom-Joo;Park, Kyoung-Suk;Park, Jun-Su;Min, Byeong-Du
    • Transactions of the Korean Society of Automotive Engineers
    • /
    • v.20 no.6
    • /
    • pp.61-66
    • /
    • 2012
  • Conventional method to transform vehicle driving condition to engine operation mode is to use vehicle road load under neutral gear and mechanical efficiency of drivetrain. But this method requires additional measurement of efficiency of drivetrain on a test rig. And this measurement is normally done at fixed speed and thus estimated accuracy of engine operation mode is not considered to be high enough. This study suggests new method to calculate engine operation mode for prescribed driving mode such as NEDC using vehicle coastdown test under gear engaged condition without measurement of mechanical efficiency of drivetrain. Coastdown test was done under neutral and gear engaged condition for comparison and also trial to extract mechanical loss of drivetrain was carried out. Calculated engine torque by conventional and newly suggested method was compared with actually measured torque of a vehicle on a chassis dynamometer during NEDC. Newly suggested method showed slightly higher accuracy of accumulated brake work during NEDC.

A Study on BIM-based Facility Management System Architecture Development considering the heterogeneous solution data integration (BIM과 이기종 솔류션 데이터 통합을 고려한 시설물관리 시스템 아키텍처 개발에 관한 연구)

  • Kang, Tae Wook;Choi, Hyun Sang
    • Spatial Information Research
    • /
    • v.21 no.4
    • /
    • pp.25-34
    • /
    • 2013
  • The purpose of this study is to develop an efficient BIM-based FM(Facility Management) system architecture to integrate the heterogeneous solution data related to FM. The heterogeneous solution data can be an FM system, asset management system, or facility sensor database systems, as well as files created by a modeler. In this study, we define the consideration for the external heterogeneous data integration related to BIM-based FM and propose the methodology. To implement it, trends in related studies were reviewed and general use cases of BIM-based FM were derived to develop the architecture for the integration between BIM and the heterogeneous solution data. The component architecture was designed to implement the use cases, and the schema was developed for the prototype by using the proposed architecture.