• 제목/요약/키워드: Distributed Data Analysis

검색결과 2,356건 처리시간 0.036초

A Security Log Analysis System using Logstash based on Apache Elasticsearch (아파치 엘라스틱서치 기반 로그스태시를 이용한 보안로그 분석시스템)

  • Lee, Bong-Hwan;Yang, Dong-Min
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • 제22권2호
    • /
    • pp.382-389
    • /
    • 2018
  • Recently cyber attacks can cause serious damage on various information systems. Log data analysis would be able to resolve this problem. Security log analysis system allows to cope with security risk properly by collecting, storing, and analyzing log data information. In this paper, a security log analysis system is designed and implemented in order to analyze security log data using the Logstash in the Elasticsearch, a distributed search engine which enables to collect and process various types of log data. The Kibana, an open source data visualization plugin for Elasticsearch, is used to generate log statistics and search report, and visualize the results. The performance of Elasticsearch-based security log analysis system is compared to the existing log analysis system which uses the Flume log collector, Flume HDFS sink and HBase. The experimental results show that the proposed system tremendously reduces both database query processing time and log data analysis time compared to the existing Hadoop-based log analysis system.

An Analysis of Effective Throughput in Distributed Wireless Scheduling

  • Radwan, Amr
    • Journal of Korea Multimedia Society
    • /
    • 제19권2호
    • /
    • pp.155-162
    • /
    • 2016
  • Several distributed scheduling policies have been proposed with the objective of attaining the maximum throughput region or a guaranteed fraction throughput region. These policies consider only the theoretical throughput and do not account the lost in throughput due to the time complexity of implementing an algorithm in practice. Therefore, we propose a novel concept called effective throughput to characterize the actual throughput by taking into account the time complexity. Effective throughput can be viewed as the actual transmitted data without including the control message overhead. Numerical results demonstrate that in practical scheduling, time complexity significantly affects throughput. The performance of throughput degrades when the time complexity is high.

Design and Implementation of Uniview Java Debugging System (유니뷰 자바 디버깅 시스템의 설계 및 구현)

  • 옥재호;정연정;이공선;윤기송
    • The Journal of Information Technology and Database
    • /
    • 제7권2호
    • /
    • pp.33-45
    • /
    • 2000
  • Uniview system is a client-server system that runs on heterogeneous distributed systems and supports the debugging of various kinds of programs. Its client system provides a unified debugging concept and interface on various debuggers of heterogeneous hosts. Its server system provides debugging services and features openness and scalability by interoperating with one or more debugger processes existing on the same host. Based on FSF(Free Software Foundation)'s gdb and Sun Microsystems's dbx. Uniview system supports C/C++ programming language in various UNIX environments as well as Windows environments. The proposed system was designed and implemented to support the JAVA language, which is prevalently used in recent heterogeneous distributed systems and was partly extended to make a clear analysis of JAVA class file structure. Sun Microsystems's Jdb supplied as a JAVA debugger has very limited functions compared to other programming language debugger. In this paper, Uniview as a JAVA debugging system was implemented to provide debugging technologies which are necessary to debug Java applications but missing in current JAVA language as well as to provide its users with various information.

  • PDF

Analysis of Storage and Retrieval Results of Audio Sources and Signatures using Blockchain and Distributed Storage System

  • Lee, Kyoung-Sik;Kim, Sang-Kyun
    • Journal of Broadcast Engineering
    • /
    • 제24권7호
    • /
    • pp.1228-1236
    • /
    • 2019
  • Recently, media platforms such as YouTube and Twitch provide services that can generate personal revenue by utilizing media content produced by individuals. In this regard, interest in the copyright of media content is increasing. In particular, in the case of an audio source, competition for securing audio source copyright is fierce because it is an essential element for almost all media content production. In this paper, we propose a method to store the audio source and its signature using a blockchain and distributed storage system to verify the copyright of music content. To identify the possibility of extracting the audio signature of the audio source and to include it as blockchain transaction data, we implement the audio source and its signature file upload system based on the proposed scheme. In addition, we show the effectiveness of the proposed method through experiments on uploading and retrieving audio files and identify future improvements.

Regional Hydrological Analysis using SLURP Model - Soyanggang-dam watershed - (SLURP 모형을 이용한 광역적 수문분석 - 소양강댐 유역을 대상으로 -)

  • Lim, Hyuk-Jin;Kwon, Hyung-Joong;Jang, Cheol-Hee;Kim, Seong-Joon
    • Proceedings of the Korean Society of Agricultural Engineers Conference
    • /
    • 한국농공학회 2003년도 학술발표논문집
    • /
    • pp.523-526
    • /
    • 2003
  • This study is to test the applicability of SLURP (Semi-distributed Land Use-based runoff Process) model that is a semi-distributed, continuous hydrologic model developed by Kite (1997). The Soyanggang-dam watershed ($2,694km^2$) was selected. The DEM, land-cover map, monthly NDVI from NOAA/AVHRR and daily meteorological data of 2001 were prepared. By using the parameter optimization technique, SCE-UA (Shuffled Complex Evolution-University of Arizona), the model was calibrated and the Nash-Sutcliffe efficiency was 0.73.

  • PDF

Analysis of Row and Column Lines in TFT-LCD panels with a Distributed Electrical Model

  • Park, Hyun-Woo;Kim, Soo-Hwan;Kim, Gyoung-Bum;Hwang, Sung-Woo;Kim, Su-Ki;McCartney, Richard I.
    • 한국정보디스플레이학회:학술대회논문집
    • /
    • 한국정보디스플레이학회 2005년도 International Meeting on Information Displayvol.II
    • /
    • pp.882-886
    • /
    • 2005
  • As the TFT-LCD panels become larger and provide higher resolution, the distributed capacitive and resistive lines induce the propagation delay, reduce the TFT-on time and deteriorate the pixel chargingratio. A number of the compensation methods, like the H-LDC (Horizontal Line Delay Compensation), have been proposed to compensate the propagation delay of the large and high resolution panels [1]. These methods, however, require the comparatively accurate gate propagation delay estimates at each column driver. In this paper, by observing the actual gate and data waveform from 15-inch XGA TFT-LCD panels, we could predict the propagation delay along the row and column line.

  • PDF

Call admission control for ATM networks using a sparse distributed memory (ATM 망에서 축약 분산 기억 장치를 사용한 호 수락 제어)

  • 권희용;송승준;최재우;황희영
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • 제35S권3호
    • /
    • pp.1-8
    • /
    • 1998
  • In this paper, we propose a Neural Call Admission Control (CAC) method using a Sparse Distributed Memory(SDM). CAC is a key technology of TM network traffic control. It should be adaptable to the rapid and various changes of the ATM network environment. conventional approach to the ATM CAC requires network analysis in all cases. So, the optimal implementation is said to be very difficult. Therefore, neural approach have recently been employed. However, it does not mett the adaptability requirements. because it requires additional learning data tables and learning phase during CAC operation. We have proposed a neural network CAC method based on SDM that is more actural than conventioal approach to apply it to CAC. We compared it with previous neural network CAC method. It provides CAC with good adaptability to manage changes. Experimenatal results show that it has rapid adaptability and stability without additional learning table or learning phase.

  • PDF

The Network Performance Analysis of Distributed Control System using Software Tool (분산제어시스템 통신망의 소프트웨어 시뮬레이션을 통한 성능 분석)

  • Jo, H.S.;Oh, E.S.;Park, D.Y.;Song, S.I.
    • Proceedings of the KIEE Conference
    • /
    • 대한전기학회 2002년도 하계학술대회 논문집 D
    • /
    • pp.2292-2294
    • /
    • 2002
  • This paper presents the network of Distributed Control System(DCS) considering specification of nuclear power plant. The network is composed of field network, control network and information network. The protocol of control network is ring type and it is compared to ethernet type. This paper proposes the structure of DCS, the protocol of each network and analyzes the network traffic along data capacity of field network, control network, information network and the network performance. Network II.5 is used as traffic simulation tool.

  • PDF

The Study of Distributed Disturbance Recording System (분산형 사고파형저장 시스템에 대한 연구)

  • Park, Jong-Chan;Lee, Se-In
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • 제57권1호
    • /
    • pp.30-35
    • /
    • 2008
  • In this paper, the application of power Quality monitoring based on Sampled Value(SV) which is introduced in the IEC61850 International Standard for substation communication are discussed. Firstly, while Merging Unit(MU) is designed as a process level device transmitting sensor data, the practical problems such as time delay compensation and optical fiber communication are encountered. Secondly, the Sampled Value message which is proper to a power quality monitoring system is presented. Because the power quality monitoring system requests less time-critical service compared to protection and control applications, the Sampled Value service message structure is introduced to improve efficiency. Lastly, the power quality monitoring server having various power quality analysis functions is suggested to verify the performance of Merging Unit With the diverse experiments, it is proved that the process bus distributed solution is flexible and economic for the power quality monitoring.

IEC61850 Process Bus Based Distributed Power Quality Monitoring (IEC61850 프로세서 버스 기반 분산형 전력품질감시)

  • Park, Jong-Chan;Kim, Byung-Jin
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • 제56권1호
    • /
    • pp.13-18
    • /
    • 2007
  • In this paper, authors deal with an application of power quality monitoring using the Sampled Value which is described in the IEC61850 International Standard for substation communication. Firstly, while Merging Unit is designed as a process level device transmitting sensor data, the practical problems such as time delay compensation and optical fiber communication are issued. Secondly, the Sampled Value message which is proper to a power quality monitoring system is presented. Because the power quality monitoring system requests non time critical service comparing to protection and control applications, the Sampled Value service message structure is introduced to improve efficiency. At last, the power quality monitoring server having various power quality analysis functions is suggested to verify the performance of Merging Unit. With the diverse experiments, it is proved that the process bus distributed solution is flexible and economic for the power quality monitoring.