• Title/Summary/Keyword: Log processing

Search Result 563, Processing Time 0.029 seconds

A Parallel Algorithm for Merging Relaxed Min-Max Heaps (Relaxed min-max 힙을 병합하는 병렬 알고리즘)

  • Min, Yong-Sik
    • The Transactions of the Korea Information Processing Society
    • /
    • v.5 no.5
    • /
    • pp.1162-1171
    • /
    • 1998
  • This paper presents a data structure that implements a mergable double-ended priority queue : namely an improved relaxed min-max-pair heap. By means of this new data structure, we suggest a parallel algorithm to merge priority queues organized in two relaxed heaps of different sizes, n and k, respectively. This new data-structure eliminates the blossomed tree and the lazying method used to merge the relaxed min-max heaps in [9]. As a result, employing max($2^{i-1}$,[(m+1/4)]) processors, this algorithm requires O(log(log(n/k))${\times}$log(n)) time. Also, on the MarPar machine, this method achieves a 35.205-fold speedup with 64 processors to merge 8 million data items which consist of two relaxed min-max heaps of different sizes.

  • PDF

A Study on SysLog-based Integrated Log Management System. (SysLog기반의 통합로그관리시스템에 관한 연구)

  • Lee, Dong-Yooung;Lee, Eul-Suk;Kim, Jin-Chul
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2011.04a
    • /
    • pp.1030-1032
    • /
    • 2011
  • 주요 ISP(Internet Service Provider)와 금융기관 및 공공기관에서는 로그 분석에 대한 관심도가 높아지고 있다. 보안사고 발생시 원인 규명을 위한 근거자료와 재발방지를 위한 정보를 제공하고, 이를 기반으로 정보보호시스템 관리자에게 다양한 보안정책을 수립할 수 있는 기반자료로 활용 로그정보의 수집과 대용량의 로그정보를 백업할 수 있는 통합로그수집/백업시스템의 필요성이 절실히 요구되고 있다. 이에 본 논문에서는 로그메시지를 처리하기 위해서 제공하는 표준 인터페이스 중 하나인 SysLog를 기반으로 이종의 침입차단시스템의 로그를 통합관리하는 시스템을 설계 구현하였다.

On the Significance of Domain-Specific Pretrained Language Models for Log Anomaly Detection (로그 이상 탐지를 위한 도메인별 사전 훈련 언어 모델 중요성 연구)

  • Lelisa Adeba Jilcha;Deuk-Hun Kim;Jin Kwak
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2024.05a
    • /
    • pp.337-340
    • /
    • 2024
  • Pretrained language models (PLMs) are extensively utilized to enhance the performance of log anomaly detection systems. Their effectiveness lies in their capacity to extract valuable semantic information from logs, thereby strengthening the detection performance. Nonetheless, challenges arise due to discrepancies in the distribution of log messages, hindering the development of robust and generalizable detection systems. This study investigates the structural and distributional variation across various log message datasets, underscoring the crucial role of domain-specific PLMs in overcoming the said challenge and devising robust and generalizable solutions.

Log histogram equalization techniques for HDR applicable to ISPs (ISP 에 적용 가능한 HDR 을 위한 Log Histogram Equalization 기법)

  • Ji-hwan Shin;Seok-in Hong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2024.05a
    • /
    • pp.860-861
    • /
    • 2024
  • 본 연구는 ISP(Image Signal Processing) 모듈에 적용 가능한 HDR(High Dynamic Range)을 위한 Log Histogram Equalization 기법을 제안한다. 기존의 HDR 기술은 다양한 노출로 찍은 사진들을 합쳐서 한 장의 사진에서 더 넓은 동적 범위를 담아내는 방식에 집중해 왔다. 이 연구에서는 단일 노출 이미지에서도 향상된 HDR 을 구현하기 위해, 로그 함수를 이용한 히스토그램 평준화 방법을 탐구한다. 이 기법은 로그 함수의 특성을 활용하여 이미지의 대비를 증가시킨다. 또한, 룩업 테이블과 선형 근사를 도입하여 연산량을 줄이고, ISP 모듈 내에서의 실시간 처리 가능성을 높인다.

Log processing using messaging system in SSD Storage Tester (SSD Storage Tester에서 메시징 시스템을 이용한 로그 처리)

  • Nam, Ki-ahn;Kwon, Oh-young
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.8
    • /
    • pp.1531-1539
    • /
    • 2017
  • The existing SSD storage tester processed logs in a 1-N structure between server and client using TCP and network file system. This method causes some problems for example, an increase in CPU usage and difficulty in exception handling, etc. In this paper, we implement a log processing message layer that can deal with asynchronous distributed processing using open source messaging system such as kafka, RabbitMQ and compare this layer with existing log transmission method. A log simulator was implemented to compare the transmission bandwidth and CPU usage. Test results show that the transmission using the message layer has higher performance than the transmission using the message layer, and the CPU usage does not show any significant difference The message layer can be implemented more easily than the conventional method and the efficiency is higher than that of the conventional method.

Microbial Contamination Level and Disinfection Effect of Electrolyzed Water in the Production Process of Dried-Laver Pyropia sp. (마른김(Pyropia sp.) 가공 공정에서의 미생물 오염도 및 전기분해수의 처리 효과)

  • Cho, Jong-Lak;Hong, Do-Hee;Kim, Young-Mog;Kim, Hyun-Joong;Kim, Jeong-Mok
    • Korean Journal of Fisheries and Aquatic Sciences
    • /
    • v.55 no.5
    • /
    • pp.662-669
    • /
    • 2022
  • The purpose of this study is to test the effects of electrolyzed water treatment on dried-laver Pyropia sp. processing facilities to control microbial contamination. Following the progression of the process to the next step, as well as during the lapse between process operating hours, the contamination level of total viable cell counts (TVC) and total coliform (TC) of laver increased. The TVC increased during the aging step, and after the molding-drying steps were completed, it increased by approximately 2.0 log CFU (colony forming unit)/g. Freshwater used for processing in April had a TVC of 4.31 log CFU/mL, which was more polluted than 2.61 log CFU/mL of seawater. Electrolyzed water was used to treat the sponge used in the laver-molding process, which resulted in a 2 log CFU/mL decrease.The TVC of dried-laver decreased by 1 to 2 log CFU/g when electrolyzed water was applied to the process. In conclusion, application of electrolyzed water in dried-laver processing was shown to be effective in reducing the microbiological contamination of the final product.

Algorithm and Design of Double-base Log Encoder for Flash A/D Converters

  • Son, Nguyen-Minh;Kim, In-Soo;Choi, Jae-Ha;Kim, Jong-Soo
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.10 no.4
    • /
    • pp.289-293
    • /
    • 2009
  • This study proposes a novel double-base log encoder (DBLE) for flash Analog-to-Digital converters (ADCs). Analog inputs of flash ADCs are represented in logarithmic number systems with bases of 2 and 3 at the outputs of DBLE. A look up table stores the sets of exponents of base 2 and 3 values. This algorithm improves the performance of a DSP (Digital Signal Processor) system that takes outputs of a flash ADC, since the double-base log number representation does multiplication operation easily within negligible error range in ADC. We have designed and implemented 6 bits DBLE implemented with ROM (Read-Only Memory) architecture in a $0.18\;{\mu}m$ CMOS technology. The power consumption and speed of DBLE are better than the FAT tree and binary ROM encoders at the cost of more chip area. The DBLE can be implemented into SoC architecture with DSP to improve the processing speed.

  • PDF

A Multiclass Classification of the Security Severity Level of Multi-Source Event Log Based on Natural Language Processing (자연어 처리 기반 멀티 소스 이벤트 로그의 보안 심각도 다중 클래스 분류)

  • Seo, Yangjin
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.5
    • /
    • pp.1009-1017
    • /
    • 2022
  • Log data has been used as a basis in understanding and deciding the main functions and state of information systems. It has also been used as an important input for the various applications in cybersecurity. It is an essential part to get necessary information from log data, to make a decision with the information, and to take a suitable countermeasure according to the information for protecting and operating systems in stability and reliability, but due to the explosive increase of various types and amounts of log, it is quite challenging to effectively and efficiently deal with the problem using existing tools. Therefore, this study has suggested a multiclass classification of the security severity level of multi-source event log using machine learning based on natural language processing. The experimental results with the training and test samples of 472,972 show that our approach has archived the accuracy of 99.59%.

An Efficient Log Data Management Architecture for Big Data Processing in Cloud Computing Environments (클라우드 환경에서의 효율적인 빅 데이터 처리를 위한 로그 데이터 수집 아키텍처)

  • Kim, Julie;Bahn, Hyokyung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.13 no.2
    • /
    • pp.1-7
    • /
    • 2013
  • Big data management is becoming increasingly important in both industry and academia of information science community. One of the important categories of big data generated from software systems is log data. Log data is generally used for better services in various service providers and can also be used as information for qualification. This paper presents a big data management architecture specialized for log data. Specifically, it provides the aggregation of log messages sent from multiple clients and provides intelligent functionalities such as analyzing log data. The proposed architecture supports an asynchronous process in client-server architectures to prevent the potential bottleneck of accessing data. Accordingly, it does not affect the client performance although using remote data store. We implement the proposed architecture and show that it works well for processing big log data. All components are implemented based on open source software and the developed prototypes are now publicly available.

LPT Scheduling for Multipurpose Machines (여러 종류의 작업 처리가 가능한 기계 시스템에 대한 LPT 스케줄링)

  • Hwang, Hark-Chin
    • IE interfaces
    • /
    • v.16 no.spc
    • /
    • pp.132-137
    • /
    • 2003
  • We consider scheduling jobs on multipurpose machines where jobs can be processed by a subset of the machines operated in parallel with the objective of minimizing makespan. We apply LPT(Longest Processing Time first) algorithm and prove that its posterior worst-case performance ratio is at most $log_24m/(1+{\lambda})$, where \lambda is the number of machines eligible for processing the job with the latest completion time. In general, LPT is shown to always find a schedule with makespan at most $log_24m/3$ times optimum.