• Proceedings of the Korea Information Processing Society Conference
• /
• 2018.10a
• /
• pp.39-42
• /
• 2018

Log analysis and monitoring have a significant importance in most of the systems. Log management has core importance in applications like distributed applications, cloud based applications, and applications designed for big data. These applications produce a large number of log files which contain essential information. This information can be used for log analytics to understand the relevant patterns from varying log data. However, they need some tools for the purpose of parsing, storing, and visualizing log informations. "Elasticsearch, Logstash, and Kibana"(ELK Stack) is one of the most popular analyzing tools for log management. For the ingestion of log files configuration files have a key importance, as they cover all the services needed to input, process, and output the log files. However, creating configuration files is sometimes very complicated and time consuming in many applications as it requires domain expertise and manual creation. In this paper, an auto configuration module for Logstash is proposed which aims to auto generate the configuration files for Logstash. The primary purpose of this paper is to provide a mechanism, which can be used to auto generate the configuration files for corresponding log files in less time. The proposed module aims to provide an overall efficiency in the log management system.

• Convergence Security Journal
• /
• v.24 no.1
• /
• pp.3-9
• /
• 2024

Currently, cyberattacks and security threats are constantly evolving, and organizations need quick and efficient security response methods. This paper proposes ways to strengthen internal network security by utilizing Unified Threat Management (UTM) equipment to improve network security and effectively manage and analyze the log data of the internal network collected through these equipment using Elastic Stack (Elasticsearch, Logstash, Kibana, hereinafter referred to as ELK Stack).

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.2
• /
• pp.382-389
• /
• 2018

Recently cyber attacks can cause serious damage on various information systems. Log data analysis would be able to resolve this problem. Security log analysis system allows to cope with security risk properly by collecting, storing, and analyzing log data information. In this paper, a security log analysis system is designed and implemented in order to analyze security log data using the Logstash in the Elasticsearch, a distributed search engine which enables to collect and process various types of log data. The Kibana, an open source data visualization plugin for Elasticsearch, is used to generate log statistics and search report, and visualize the results. The performance of Elasticsearch-based security log analysis system is compared to the existing log analysis system which uses the Flume log collector, Flume HDFS sink and HBase. The experimental results show that the proposed system tremendously reduces both database query processing time and log data analysis time compared to the existing Hadoop-based log analysis system.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.2
• /
• pp.185-190
• /
• 2024

In 2022, we can see the real estate market in Korea going down. Corona 19 and the Russian invasion of Ukraine are cited as the biggest causes for this. These two problems ignited the economic recession, causing prices to fall and subsequently raising exchange rates and interest rates. Due to the aforementioned problems in the previously active real estate market, the number of actual transactions has decreased, resulting in a decline in the real estate market due to high interest rates. Data provided by the public data portal, KOSIS, and the Seoul Metropolitan Government were collected through Logstash, transferred to Elasticsearch, and visualized inflation, exchange rates, and loan interest rates using the dashboard function provided by Kibana, to analyze causes and derive results. In addition, three specific apartments in Nowon-gu and Jongno-gu, which have the highest number of actual transactions in Seoul, are selected and the actual transaction prices that change every month are displayed in the Data Table.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2017.11a
• /
• pp.299-301
• /
• 2017

현대사회에서는 사이버 해킹 공격이 많이 일어나고 있다. 공격이 증가함에 따라 이를 다양한 방법으로 방어하고 탐지하는 연구가 많이 이루어지고 있다. 본 논문은 OpenIOC, STIX, MMDEF 등과 같은 공격자의 방법론 또는 증거를 식별하는 기술 특성 설명을 수집해 놓은 표현들을 기반을 머신러닝과 logstash라는 로그 수집기를 결합하는 새로운 시스템을 제안한다. 시스템은 pc에 공격이 가해졌을 때 로그 수집기를 사용하여 로그를 수집한 후에 로그의 속성 값들의 리스트를 가지고 머신러닝 알고리즘을 통해 학습시켜 분석을 진행한다. 향후에는 제안된 시스템을 실시간 처리 머신러닝 알고리즘을 사용하여 필요로그정보의 구성을 해주면 자동으로 로그정보를 수집하고 필터와 출력을 거쳐 학습을 시켜 자동 침입탐지시스템으로 발전할 수 있을 것이라 예상된다.

• Journal of Advanced Navigation Technology
• /
• v.28 no.1
• /
• pp.159-162
• /
• 2024

This paper proposes a novel safety autonomous platform (SAP) architecture that can automatically and precisely manage on-site safety through ensemble artificial intelligence models generated from video information, worker's biometric information, and the safety rule to estimate the risk index. We practically designed the proposed SAP architecture by the Hadoop ecosystem with Kafka/NiFi, Spark/Hive, Hue, ELK (Elasticsearch, Logstash, Kibana), Ansible, etc., and confirmed that it worked well with safety mobility gateways for providing various safety applications.

• Safety and Health at Work
• /
• v.15 no.3
• /
• pp.373-378
• /
• 2024

Introduction: Following the Republic of Korea electric power industry site-specific safety management system, this paper proposes a novel safety autonomous platform (SAP) architecture that can automatically and precisely manage on-site safety through ensemble artificial intelligence (AI) models. The ensemble AI model was generated from video information and worker's biometric information as learning data and the estimation results of this model are based on standard operating procedures of the workplace and safety rules. Methods: The ensemble AI model is designed and implemented by the Hadoop ecosystem with Kafka/NiFi, Spark/Hive, HUE, and ELK (Elasticsearch, Logstash, Kibana). Results: The functional evaluation shows that the main function of this SAP architecture was operated successfully. Discussion: The proposed model is confirmed to work well with safety mobility gateways to provide some safety applications.

• KIISE Transactions on Computing Practices
• /
• v.22 no.7
• /
• pp.332-337
• /
• 2016

System administrator or security managers need to collect logs of computing device (desktop or server), which are used for the purpose of cause-analysis of security incident and discover if damage to system was either caused by hacking or computer virus. Furthermore, appropriate log maintenance helps preventing security breech incidents through identification of vulnerability. In addition, it can be utilized for prevention of data leakage through the insider. In the paper, we present log collection system developed using open source supported by commands and basic methods of Windows. Furthermore, we aim to collect log information to enable search and analysis from diverse perspectives and to propose a way to integrate with open source-based search engine system.