• Title/Summary/Keyword: Big Data Pattern Analysis

Search Result 172, Processing Time 0.027 seconds

Customer Load Pattern Analysis using Clustering Techniques (클러스터링 기법을 이용한 수용가별 전력 데이터 패턴 분석)

  • Ryu, Seunghyoung;Kim, Hongseok;Oh, Doeun;No, Jaekoo
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.2 no.1
    • /
    • pp.61-69
    • /
    • 2016
  • Understanding load patterns and customer classification is a basic step in analyzing the behavior of electricity consumers. To achieve that, there have been many researches about clustering customers' daily load data. Nowadays, the deployment of advanced metering infrastructure (AMI) and big-data technologies make it easier to study customers' load data. In this paper, we study load clustering from the view point of yearly and daily load pattern. We compare four clustering methods; K-means clustering, hierarchical clustering (average & Ward's method) and DBSCAN (Density-Based Spatial Clustering of Applications with Noise). We also discuss the relationship between clustering results and Korean Standard Industrial Classification that is one of possible labels for customers' load data. We find that hierarchical clustering with Ward's method is suitable for clustering load data and KSIC can be well characterized by daily load pattern, but not quite well by yearly load pattern.

Some Considerations on the Problems of PSA(Pulse Sequence Analysis) as a Partial Discharge Analysis Method (부분방전 해석 방법으로 PSA(Pulse Sequence Analysis)의 문제점에 대한 고찰)

  • Kim, Jeong-Tae;Lee, Ho-Keun
    • Proceedings of the Korean Institute of Electrical and Electronic Material Engineers Conference
    • /
    • 2004.11a
    • /
    • pp.327-330
    • /
    • 2004
  • Because of its effectiveness for the PD(partial discharge) pattern recognition, PSA(Pulse Sequence Analysis) has been considered as a new analytic method instead of conventional PRPDA(Phase Resolved Partial Discharge Analysis). However, PSA has a big problem that can misanalyze patterns in case of data missing resulting from poor sensitivity because it analyses the correlation between sequential pulses, which leads to hesitate to apply it to on-site. Therefore, in this paper, the problems of PSA such as data missing and noise adding cases were investigated. For the purpose, PD data obtained from various defects including noise adding data were used and analysed, The result showed that both cases can cause fatal errors in recognizing PD patterns. In case of the data missing, the error depends on the kinds of defect and the degree of degradation. Also, it could be noticed that the error due to adding noises was larger than that due to some data missing.

  • PDF

Structuring of unstructured big data and visual interpretation (부산지역 교통관련 기사를 이용한 비정형 빅데이터의 정형화와 시각적 해석)

  • Lee, Kyeongjun;Noh, Yunhwan;Yoon, Sanggyeong;Cho, Youngseuk
    • Journal of the Korean Data and Information Science Society
    • /
    • v.25 no.6
    • /
    • pp.1431-1438
    • /
    • 2014
  • We analyzed the articles from "Kukje Shinmun" and "Busan Ilbo", which are two local newpapers of Busan Metropolitan City. The articles cover from January 1, 2013 to December 31, 2013. Meaningful pattern inherent in 2889 articles of which the title includes "Busan" and "Traffic" and related data was analyzed. Textmining method, which is a part of datamining, was used for the social network analysis (SNA). HDFS and MapReduce (from Hadoop ecosystem), which is open-source framework based on JAVA, were used with Linux environment (Uubntu-12.04LTS) for the construction of unstructured data and the storage, process and the analysis of big data. We implemented new algorithm that shows better visualization compared with the default one from R package, by providing the color and thickness based on the weight from each node and line connecting the nodes.

A Study on the Production and Consumption Authentication Power Trading System based on Big Data Analysis using Blockchain Network (블록체인 네트워크를 이용한 빅데이터 분석 기반 생산·소비량 인증 전력 거래 시스템에 관한 연구)

  • Kim, Young-Gon;Heo, Keol;Choi, Jung-In
    • Journal of Energy Engineering
    • /
    • v.28 no.4
    • /
    • pp.76-81
    • /
    • 2019
  • This paper is a review of the certification system required for various energy prosumer business models, including P2P energy trading and participation in small demand response programs, which are based on reliable production and consumption certification. One of the most important parameter in energy trading is ensuring the reliability of trading account balancing. Therefore, we studied to use big data pattern analysis based blockchain smart contract between trading partners to make its tradings are more reliable. For this purpose big data analysis system collected from the IoT AMI and a production authentication system using a private blockchain network linked with the AMI is discussed, using the blockchain smart contract are also suggested. Futhermore, energy trading system concept and business models are introduced.

Implementation of User Recommendation System based on Video Contents Story Analysis and Viewing Pattern Analysis (영상 스토리 분석과 시청 패턴 분석 기반의 추천 시스템 구현)

  • Lee, Hyoun-Sup;Kim, Minyoung;Lee, Ji-Hoon;Kim, Jin-Deog
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.12
    • /
    • pp.1567-1573
    • /
    • 2020
  • The development of Internet technology has brought the era of one-man media. An individual produces content on user own and uploads it to related online services, and many users watch the content of online services using devices that allow them to use the Internet. Currently, most users find and watch content they want through search functions provided by existing online services. These features are provided based on information entered by the user who uploaded the content. In an environment where content needs to be retrieved based on these limited word data, user unwanted information is presented to users in the search results. To solve this problem, in this paper, the system actively analyzes the video in the online service, and presents a way to extract and reflect the characteristics held by the video. The research was conducted to extract morphemes based on the story content based on the voice data of a video and analyze them with big data technology.

Medical costs for patients with Facial paralysis : Based on Health Big Data (보건의료 빅데이터를 이용한 얼굴마비환자의 의료비용에 관한 연구)

  • Hong, Min-Jung;Umh, Tae-Woong;Kim, Sina;Kim, Nam-Kwen
    • The Journal of Korean Medicine
    • /
    • v.36 no.3
    • /
    • pp.98-110
    • /
    • 2015
  • Objectives: The purpose of this study was to analyze the medical cost of facial paralysis in payer perspective and to estimate the practice pattern of patient using 2011 Health Insurance Review & Assessment Service-National Patients Sample(HIRA-NPS). Methods: Basic statistical system was used for descriptive analysis of NPS dataset. A table for general information (table20) was extracted by disease code, and social demographic characteristics, distribution of the use among inpatients and outpatients, utilization of each kind of medical care institutions, medical cost were analyzed. Subgroup analysis was conducted for assuming the practice pattern of korean medicine and western medicine. Results: A total of 8,219 people and 64,345 claims data were identified as having facial paralysis. Proportion of outpatient was 95.23%, inpatient 0.84% and patient using both services 3.93%. Mean patient charges was 44,229 won per outpatient, 178,886 won per inpatient and 523,542 won per patient using both services. Utilization of korean medical care institutions was 68.81%(claims), 40.46%(patients), utilization of western medical care institutions was 31.19%(claims), 59.54%(patients). The amount charged by korean medical care institutions was 52.61% and western medical care institutions was 47.39%. Cost per claim was higher than those of the korean treatment and cost per patient of western treatment was lower than those of the korean treatment. Conclusions: The research assessed the medical cost and practice pattern associated with facial paralysis. These findings could be used in health care policy and subsequent studies.

Determination of coagulant input rate in water purification plant using K-means algorithm and GBR algorithm (K-means 알고리즘과 GBR 알고리즘을 이용한 정수장 응집제 투입률 결정 기법)

  • Kim, Jinyoung;Kang, Bokseon;Jung, Hoekyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.6
    • /
    • pp.792-798
    • /
    • 2021
  • In this paper, an algorithm for determining the coagulant input rate in the drug-injection tank during the process of the water purification plant was derived through big data analysis and prediction based on artificial intelligence. In addition, analysis of big data technology and AI algorithm application methods and existing academic and technical data were reviewed to analyze and review application cases in similar fields. Through this, the goal was to develop an algorithm for determining the coagulant input rate and to present the optimal input rate through autonomous driving simulator and pilot operation of the coagulant input process. Through this study, the coagulant injection rate, which is an output variable, is determined based on various input variables, and it is developed to simulate the relationship pattern between the input variable and the output variable and apply the learned pattern to the decision-making pattern of water plant operating workers.

Recent Technique Analysis, Infant Commodity Pattern Analysis Scenario and Performance Analysis of Incremental Weighted Maximal Representative Pattern Mining (점진적 가중화 맥시멀 대표 패턴 마이닝의 최신 기법 분석, 유아들의 물품 패턴 분석 시나리오 및 성능 분석)

  • Yun, Unil;Yun, Eunmi
    • Journal of Internet Computing and Services
    • /
    • v.21 no.2
    • /
    • pp.39-48
    • /
    • 2020
  • Data mining techniques have been suggested to find efficiently meaningful and useful information. Especially, in the big data environments, as data becomes accumulated in several applications, related pattern mining methods have been proposed. Recently, instead of analyzing not only static data stored already in files or databases, mining dynamic data incrementally generated in a real time is considered as more interesting research areas because these dynamic data can be only one time read. With this reason, researches of how these dynamic data are mined efficiently have been studied. Moreover, approaches of mining representative patterns such as maximal pattern mining have been proposed since a huge number of result patterns as mining results are generated. As another issue, to discover more meaningful patterns in real world, weights of items in weighted pattern mining have been used, In real situation, profits, costs, and so on of items can be utilized as weights. In this paper, we analyzed weighted maximal pattern mining approaches for data generated incrementally. Maximal representative pattern mining techniques, and incremental pattern mining methods. And then, the application scenarios for analyzing the required commodity patterns in infants are presented by applying weighting representative pattern mining. Furthermore, the performance of state-of-the-art algorithms have been evaluated. As a result, we show that incremental weighted maximal pattern mining technique has better performance than incremental weighted pattern mining and weighted maximal pattern mining.

A Study for Electronic Trading Business System Using Big Data (빅데이터를 활용한 전자무역시스템에 대한 연구)

  • Lee, Cheol-Woong;Cho, Sung-Woo;Cho, Sae-Hong;Hwang, Dae-Hoon
    • Journal of Digital Contents Society
    • /
    • v.14 no.4
    • /
    • pp.573-580
    • /
    • 2013
  • With the growth of the smart-devices and information & communication technology, information society has developed and information can be produced, spread and consumed at much faster pace easily. Hence, individuals can utilize wireless communication and smart-devices to create, share and consume information at anytime and anywhere. The growth of technology has allowed the large-scale transfer and sharing of image, sound and video data; it changed the users' data consumption pattern that was mainly consisted of the text. Therefore, the amount of data that an individual consumes increased significantly. The importance of finding and analyzing practical and necessary data among huge amount of data has arisen. In this study, the current status of Big Data is researched and analyzed and the method to utilize Big Data in the electronic trading field is suggested.

Latent mobility pattern analysis of bus passengers with LDA (LDA 기법을 이용한 버스 승객의 잠재적 이동패턴 분석)

  • Cho, Ah;Lee, Kyung Hee;Cho, Wan Sup
    • Journal of the Korean Data and Information Science Society
    • /
    • v.26 no.5
    • /
    • pp.1061-1069
    • /
    • 2015
  • Recently, transportation big data generated in the transportation sector has been widely used in the transportation policies making and efficient system management. Bus passengers' mobility patterns are useful insight for transportation policy maker to optimize bus lines and time intervals in a city. We propose a new methodology to discover mobility patterns by using transportation card data. We first estimate the bus stations where the passengers get-off because the transportation card data don't have the get-off information in most cities. We then applies LDA (Latent Dirichlet Allocation), the most representative topic modeling technique, to discover mobility patterns of bus passengers in Cheong-Ju city. To understand discovered patterns, we construct a data warehouse and perform multi-dimensional analysis by bus-route, region, time-period, and the mobility patterns (get-on/get-off station). In the case of Cheong Ju, we discovered mobility pattern 1 from suburban area to Cheong-Ju terminal, mobility pattern 2 from residential area to commercial area, mobility pattern 3 from school areas to commercial area.