• Title/Summary/Keyword: log analysis system

Search Result 561, Processing Time 0.027 seconds

A Security Log Analysis System using Logstash based on Apache Elasticsearch (아파치 엘라스틱서치 기반 로그스태시를 이용한 보안로그 분석시스템)

  • Lee, Bong-Hwan;Yang, Dong-Min
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.22 no.2
    • /
    • pp.382-389
    • /
    • 2018
  • Recently cyber attacks can cause serious damage on various information systems. Log data analysis would be able to resolve this problem. Security log analysis system allows to cope with security risk properly by collecting, storing, and analyzing log data information. In this paper, a security log analysis system is designed and implemented in order to analyze security log data using the Logstash in the Elasticsearch, a distributed search engine which enables to collect and process various types of log data. The Kibana, an open source data visualization plugin for Elasticsearch, is used to generate log statistics and search report, and visualize the results. The performance of Elasticsearch-based security log analysis system is compared to the existing log analysis system which uses the Flume log collector, Flume HDFS sink and HBase. The experimental results show that the proposed system tremendously reduces both database query processing time and log data analysis time compared to the existing Hadoop-based log analysis system.

A Method for Analyzing Web Log of the Hadoop System for Analyzing a Effective Pattern of Web Users (효과적인 웹 사용자의 패턴 분석을 위한 하둡 시스템의 웹 로그 분석 방안)

  • Lee, Byungju;Kwon, Jungsook;Go, Gicheol;Choi, Yonglak
    • Journal of Information Technology Services
    • /
    • v.13 no.4
    • /
    • pp.231-243
    • /
    • 2014
  • Of the various data that corporations can approach, web log data are important data that correspond to data analysis to implement customer relations management strategies. As the volume of approachable data has increased exponentially due to the Internet and popularization of smart phone, web log data have also increased a lot. As a result, it has become difficult to expand storage to process large amounts of web logs data flexibly and extremely hard to implement a system capable of categorizing, analyzing, and processing web log data accumulated over a long period of time. This study thus set out to apply Hadoop, a distributed processing system that had recently come into the spotlight for its capacity of processing large volumes of data, and propose an efficient analysis plan for large amounts of web log. The study checked the forms of web log by the effective web log collection methods and the web log levels by using Hadoop and proposed analysis techniques and Hadoop organization designs accordingly. The present study resolved the difficulty with processing large amounts of web log data and proposed the activity patterns of users through web log analysis, thus demonstrating its advantages as a new means of marketing.

A Study on the Improvement of Information Service Using Information System Log Analysis (정보 시스템 이용기록 분석을 통한 정보 서비스 개선방안 연구)

  • Jho, Jae-Hyeong
    • Journal of Information Management
    • /
    • v.36 no.4
    • /
    • pp.137-153
    • /
    • 2005
  • For the improvement of information service, users' transaction log can be stored to the system, and the log analysis should be included in the process of service improvement. Also there are differences within kinds of users' log records and methods of analysis according to the institution's strategy. This paper describes the kinds of log records from users' behavior on information system. And its goal is to consider the case of information center which operates log analysis, and to derive a plan for improvement of information services.

A Study on the Analysis of Validity and Importance of Event Log for the Detection of Insider Threats to Control System (제어시스템의 내부자 위협 탐지를 위한 Event Log 타당성 및 중요도 분석에 관한 연구)

  • Kim, Jongmin;Kim, DongMin;Lee, DongHwi
    • Convergence Security Journal
    • /
    • v.18 no.3
    • /
    • pp.77-85
    • /
    • 2018
  • With the convergence of communications network between control system and public network, such threats like information leakage/falsification could be fully shown in control system through diverse routes. Due to the recent diversification of security issues and violation cases of new attack techniques, the security system based on the information database that simply blocks and identifies, is not good enough to cope with the new types of threat. The current control system operates its security system focusing on the outside threats to the inside, and it is insufficient to detect the security threats by insiders with the authority of security access. Thus, this study conducted the importance analysis based on the main event log list of "Spotting the Adversary with Windows Event Log Monitoring" announced by NSA. In the results, the matter of importance of event log for the detection of insider threats to control system was understood, and the results of this study could be contributing to researches in this area.

  • PDF

A Log Analysis System with REST Web Services for Desktop Grids and its Application to Resource Group-based Task Scheduling

  • Gil, Joon-Min;Kim, Mi-Hye
    • Journal of Information Processing Systems
    • /
    • v.7 no.4
    • /
    • pp.707-716
    • /
    • 2011
  • It is important that desktop grids should be able to aggressively deal with the dynamic properties that arise from the volatility and heterogeneity of resources. Therefore, it is required that task scheduling be able to positively consider the execution behavior that is characterized by an individual resource. In this paper, we implement a log analysis system with REST web services, which can analyze the execution behavior by utilizing the actual log data of desktop grid systems. To verify the log analysis system, we conducted simulations and showed that the resource group-based task scheduling, based on the analysis of the execution behavior, offers a faster turnaround time than the existing one even if few resources are used.

A Precursor Phenomena Analysis of APT Hacking Attack and IP Traceback (APT 해킹 공격에 대한 전조현상 분석 및 IP역추적)

  • Noh, Jung Ho;Park, Dea-Woo
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2013.05a
    • /
    • pp.275-278
    • /
    • 2013
  • Log is a file system, a system that uses all remaining data. Want situation now being issued in the IT, media Nate on information disclosure, the press agency server hack by numbness crime occurred. Hacking crisis that's going through this log analysis software professionally for professional analysis is needed. The present study, about APT attacks happening intelligently Log In case of more than traceback in advance to prevent the technology to analyze the pattern for log analysis techniques.

  • PDF

Frequency domain analysis of the urophonography for LUTS diagnosis (하부요로 폐색 진단을 위한 요류음 주파수 분석)

  • Jeong, Do-Un;Jeon, Gye-Rok
    • Journal of Sensor Science and Technology
    • /
    • v.16 no.3
    • /
    • pp.202-210
    • /
    • 2007
  • The aim of this study is to acquire useful information of lower urinary tract symptom (LUTS) diagnosis through urophonography signal as a noninvasive method. The hardware and software which could evaluate the function of compensatory hypertrophy with noninvasive and comfortable method was implemented to measure uroflow and urophonography signal during urination. The PSD (power spectrum density) and the log-log plot gradient analysis were accomplished in frequency domain. For evaluation of the system and analysis method, a model system for the lower urinary system of men was used. From the evaluation of the model system, the PDS and the log-log plot gradient were dependent on the occlusion degree significantly. In a pilot study on normal and abnormal male subjects, the PSD and the log-log plot gradient were highly correlated with the artificial urethral obstruction.

A Study on Event Log Correlation Analysis for Control System Threat Analysis (제어시스템 위협분석을 위한 Event Log 상관분석에 관한 연구)

  • Kim, Jongmin;Kim, Minsu;Lee, DongHwi
    • Convergence Security Journal
    • /
    • v.17 no.5
    • /
    • pp.35-40
    • /
    • 2017
  • The control system can have such threats as information leakage and falsification through various routes due to communications network fusion with public network. As the issues about security and the infringe cases by new attack methods are diversified recently, with the security system that makes information data database by simply blocking and checking it is difficult to cope with new types of threats. It is also difficult to respond security threats by insiders who have security access authority with the existing security equipment. To respond the threats by insiders, it is necessary to collect and analyze Event Log occurring in the internal system realtime. Therefore, this study could find out whether there is correlation of the elements among Event Logs through correlation analysis based on Event Logs that occur real time in the control system, and based on the analysis result, the study is expected to contribute to studies in this field.

Development of the SysLog-based Integrated Log Management system for Firewalls in Distributed Network Environments (분산 환경에서 SysLog기반의 방화벽 통합로그관리시스템 개발)

  • Lee, Dong Young;Seo, Hee Suk;Lee, Eul Suk
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.7 no.4
    • /
    • pp.39-45
    • /
    • 2011
  • Application log files contain error messages; operational data and usage information that can help manage applications and servers. Log analysis system is software that read and parse log files, extract and aggregate information in order to generate reports on the application. In currently, the importance of log files of firewalls is growing bigger and bigger for the forensics of cyber crimes and the establishment of security policy. In this paper, we designed and implemented the SILAS(SysLog-based Integrated Log mAanagement System) in distribute network environments. It help to generate reports on the the log fires of firewalls - IP and users, and statistics of application usage.

Spark-based Network Log Analysis Aystem for Detecting Network Attack Pattern Using Snort (Snort를 이용한 비정형 네트워크 공격패턴 탐지를 수행하는 Spark 기반 네트워크 로그 분석 시스템)

  • Baek, Na-Eun;Shin, Jae-Hwan;Chang, Jin-Su;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.4
    • /
    • pp.48-59
    • /
    • 2018
  • Recently, network technology has been used in various fields due to development of network technology. However, there has been an increase in the number of attacks targeting public institutions and companies by exploiting the evolving network technology. Meanwhile, the existing network intrusion detection system takes much time to process logs as the amount of network log increases. Therefore, in this paper, we propose a Spark-based network log analysis system that detects unstructured network attack pattern. by using Snort. The proposed system extracts and analyzes the elements required for network attack pattern detection from large amount of network log data. For the analysis, we propose a rule to detect network attack patterns for Port Scanning, Host Scanning, DDoS, and worm activity, and can detect real attack pattern well by applying it to real log data. Finally, we show from our performance evaluation that the proposed Spark-based log analysis system is more than two times better on log data processing performance than the Hadoop-based system.