• Title/Summary/Keyword: log Data Analysis

Search Result 975, Processing Time 0.029 seconds

The Method of Analyzing Firewall Log Data using MapReduce based on NoSQL (NoSQL기반의 MapReduce를 이용한 방화벽 로그 분석 기법)

  • Choi, Bomin;Kong, Jong-Hwan;Hong, Sung-Sam;Han, Myung-Mook
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.23 no.4
    • /
    • pp.667-677
    • /
    • 2013
  • As the firewall is a typical network security equipment, it is usually installed at most of internal/external networks and makes many packet data in/out. So analyzing the its logs stored in it can provide important and fundamental data on the network security research. However, along with development of communications technology, the speed of internet network is improved and then the amount of log data is becoming 'Massive Data' or 'BigData'. In this trend, there are limits to analyze log data using the traditional database model RDBMS. In this paper, through our Method of Analyzing Firewall log data using MapReduce based on NoSQL, we have discovered that the introducing NoSQL data base model can more effectively analyze the massive log data than the traditional one. We have demonstrated execellent performance of the NoSQL by comparing the performance of data processing with existing RDBMS. Also the proposed method is evaluated by experiments that detect the three attack patterns and shown that it is highly effective.

A guideline for the statistical analysis of compositional data in immunology

  • Yoo, Jinkyung;Sun, Zequn;Greenacre, Michael;Ma, Qin;Chung, Dongjun;Kim, Young Min
    • Communications for Statistical Applications and Methods
    • /
    • v.29 no.4
    • /
    • pp.453-469
    • /
    • 2022
  • The study of immune cellular composition has been of great scientific interest in immunology because of the generation of multiple large-scale data. From the statistical point of view, such immune cellular data should be treated as compositional. In compositional data, each element is positive, and all the elements sum to a constant, which can be set to one in general. Standard statistical methods are not directly applicable for the analysis of compositional data because they do not appropriately handle correlations between the compositional elements. In this paper, we review statistical methods for compositional data analysis and illustrate them in the context of immunology. Specifically, we focus on regression analyses using log-ratio transformations and the alternative approach using Dirichlet regression analysis, discuss their theoretical foundations, and illustrate their applications with immune cellular fraction data generated from colorectal cancer patients.

MapReduce-Based Partitioner Big Data Analysis Scheme for Processing Rate of Log Analysis (로그 분석 처리율 향상을 위한 맵리듀스 기반 분할 빅데이터 분석 기법)

  • Lee, Hyeopgeon;Kim, Young-Woon;Park, Jiyong;Lee, Jin-Woo
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.5
    • /
    • pp.593-600
    • /
    • 2018
  • Owing to the advancement of Internet and smart devices, access to various media such as social media became easy; thus, a large amount of big data is being produced. Particularly, the companies that provide various Internet services are analyzing the big data by using the MapReduce-based big data analysis techniques to investigate the customer preferences and patterns and strengthen the security. However, with MapReduce, when the big data is analyzed by defining the number of reducer objects generated in the reduce stage as one, the processing rate of big data analysis decreases. Therefore, in this paper, a MapReduce-based split big data analysis method is proposed to improve the log analysis processing rate. The proposed method separates the reducer partitioning stage and the analysis result combining stage and improves the big data processing rate by decreasing the bottleneck phenomenon by generating the number of reducer objects dynamically.

Development of integrated management solution through log analysis based on Big Data (빅데이터기반의 로그분석을 통한 통합 관리 솔루션 개발)

  • Kang, Sun-Kyoung;Lee, Hyun-Chang;Shin, Seong-Yoon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.541-542
    • /
    • 2017
  • In this paper, we intend to develop an integrated management solution that can be easily operated by integrating complex and various cloud environments. This has the advantage that users and administrators can conveniently solve problems by collecting and analyzing fixed log data and unstructured log data based on big data and realizing integrated monitoring in real time. Hypervisor log pattern analysis technology will be able to manage existing complex and various cloud environment more efficiently.

  • PDF

Estimation of Log-Odds Ratios for Incomplete $2{\times}2$ Tables with Covariates using FEFI

  • Kang, Shin-Soo;Bae, Je-Min
    • Journal of the Korean Data and Information Science Society
    • /
    • v.18 no.1
    • /
    • pp.185-194
    • /
    • 2007
  • The information of covariates are available to do fully efficient fractional imputation(FEFI). The new method, FEFI with logistic regression is proposed to construct complete contingency tables. Jackknife method is used to get a standard errors of log-odds ratio from the completed table by the new method. Simulation results, when covariates have more information about categorical variables, reveal that the new method provides more efficient estimates of log-odds ratio than either multiple imputation(MI) based on data augmentation or complete case analysis.

  • PDF

A Study on Event Log Correlation Analysis for Control System Threat Analysis (제어시스템 위협분석을 위한 Event Log 상관분석에 관한 연구)

  • Kim, Jongmin;Kim, Minsu;Lee, DongHwi
    • Convergence Security Journal
    • /
    • v.17 no.5
    • /
    • pp.35-40
    • /
    • 2017
  • The control system can have such threats as information leakage and falsification through various routes due to communications network fusion with public network. As the issues about security and the infringe cases by new attack methods are diversified recently, with the security system that makes information data database by simply blocking and checking it is difficult to cope with new types of threats. It is also difficult to respond security threats by insiders who have security access authority with the existing security equipment. To respond the threats by insiders, it is necessary to collect and analyze Event Log occurring in the internal system realtime. Therefore, this study could find out whether there is correlation of the elements among Event Logs through correlation analysis based on Event Logs that occur real time in the control system, and based on the analysis result, the study is expected to contribute to studies in this field.

Log-based petrophysical analysis of Khatatba Formation in Shoushan Basin, North Western Desert, Egypt

  • Osli, Liyana Nadiah;Yakub, Nur Yusrina;Shalaby, Mohamed Ragab;Islam, Md. Aminul
    • Geosciences Journal
    • /
    • v.22 no.6
    • /
    • pp.1015-1026
    • /
    • 2018
  • This paper aims to investigate the good reservoir quality and hydrocarbon potentiality of the Khatatba Formation, Qasr Field in the Shoushan Basin of the North Western Desert, Egypt by combining results from log-based petrophysical analysis, petrographic description and images from scanning electron microscope (SEM). Promising reservoir units are initially identified and evaluated through well log analysis of three wells in the field of study. Petrophysical results are then compared with petrographic and SEM images from rock samples to identify features that characterize the reservoir quality. Well log results show that Khatatba Formation in the study area has good sandstone reservoir intervals from depths ranging from 12848 ft to 13900 ft, with good effective porosity records of 13-15% and hydrocarbon saturations of greater than 83%. Petrographic analysis of these sandstone reservoir units indicate high concentrations of vacant pore spaces with good permeability that can be easily occupied by hydrocarbon. The availability of these pore spaces are attributed to pore-enhancing diagenetic features, mainly in the form of good primary porosity and dissolution. SEM images and EDX analysis confirmed the presence of hydrocarbon, therefore indicating a good hydrocarbon-storing potential for the Khatatba Formation sandstones.

A Study on Reliability Data Analysis for Components of Machining Center (공작기계 부품의 신뢰성 데이터 해석에 관한 연구)

  • 이수훈;김종수;송준엽;이승우;박화영
    • Proceedings of the Korean Society of Precision Engineering Conference
    • /
    • 2001.04a
    • /
    • pp.88-91
    • /
    • 2001
  • The reliability data analysis for components of CNC machining center is studied in this paper. The failure data of mechanical part is analyzed by Exponetial, Weibull, and Log-normal distributions. And then, the optimum failure distribution model is selected by goodness of fit test. The reliability data analysis program is developed using ASP language. The failure rate, MTBF, life, and failure mode of mechanical parts are estimated and searched by this program. The failure data and analysis results are stored in the database.

  • PDF

A Study on Ground Vehicle Mechanics for Steep Slope Forest Operations - Rubber-Tired Log Skidding Tractor Operations - (급경사지 산림작업을 위한 차량의 역학분석에 관한 연구 -차륜형 집재작업 트랙터를 중심으로-)

  • Chung, Joo Sang;Chung, Woo Dam
    • Journal of Korean Society of Forest Science
    • /
    • v.84 no.2
    • /
    • pp.218-225
    • /
    • 1995
  • In this paper, a mechanical analysis model for steep-slope log-skidding operations of a rubber-tired tractor is discussed and the applicability of the model is investigated. The model largely consists of mathematical analysis models for log drag, dynamic vehicle weight distributions and soil-vehicle traction. For the case study, a theoretical data set for log skidding operations is used in investigating the effect of the factors influencing the results of mechanical analysis or the productivity of skidding operations. The analyses include 1) the effect of log choking methods on tangential log-skidding force, 2) the effects of the change in travel speed and log load on the required input power to the wheels and 3) the log skidding performance of a two-wheel drive compared with that of a four-wheel drive.

  • PDF

The Difference Analyses between Users' Actual Usage and Perceived Preference: The Case of ERP Functions on Legacy Systems (사용자의 실제 이용과 인지된 선호도 차이 분석: 레거시 시스템의 ERP 기능을 중심으로)

  • Cho, Yong-Tak;Kim, Injai
    • The Journal of Information Systems
    • /
    • v.23 no.1
    • /
    • pp.185-202
    • /
    • 2014
  • ERP, a typical enterprise application, helps companies to increase their productivity and to support their decision makings. ERP is composed of diverse functions that are optimized under PC environment, whereas the ERP applications on a mobile platform have many constraints such as a small screen, limited resolution, and computing power. Because all the functions of a ERP legacy system are not required for ERP on a mobile device, the core functions of the ERP system should be selected to increase system efficiency. In this study, two main methods were used; interviews and log analyses. The end users using a ERP system were interviewed for their perceptions, and log data analyses were made for the hitting number of specific ERP functions. The differences between the actual usage based on log data and users' cognitive preferences about ERP functions were analysed. Finally, the functional differences between users' perception and actual usage were suggested for some practical implications.