• Title/Summary/Keyword: Distributed memory

Search Result 397, Processing Time 0.024 seconds

SplitScreen: Enabling Efficient, Distributed Malware Detection

  • Cha, Sang-Kil;Moraru, Iulian;Jang, Ji-Yong;Truelove, John;Brumley, David;Andersen, David G.
    • Journal of Communications and Networks
    • /
    • v.13 no.2
    • /
    • pp.187-200
    • /
    • 2011
  • We present the design and implementation of a novel anti-malware system called SplitScreen. SplitScreen performs an additional screening step prior to the signature matching phase found in existing approaches. The screening step filters out most non-infected files (90%) and also identifiesmalware signatures that are not of interest (99%). The screening step significantly improves end-to-end performance because safe files are quickly identified and are not processed further, and malware files can subsequently be scanned using only the signatures that are necessary. Our approach naturally leads to a network-based anti-malware solution in which clients only receive signatures they needed, not every malware signature ever created as with current approaches. We have implemented SplitScreen as an extension to ClamAV, the most popular open source anti-malware software. For the current number of signatures, our implementation is $2{\times}$ faster and requires $2{\times}$ less memory than the original ClamAV. These gaps widen as the number of signatures grows.

Task failure resilience technique for improving the performance of MapReduce in Hadoop

  • Kavitha, C;Anita, X
    • ETRI Journal
    • /
    • v.42 no.5
    • /
    • pp.748-760
    • /
    • 2020
  • MapReduce is a framework that can process huge datasets in parallel and distributed computing environments. However, a single machine failure during the runtime of MapReduce tasks can increase completion time by 50%. MapReduce handles task failures by restarting the failed task and re-computing all input data from scratch, regardless of how much data had already been processed. To solve this issue, we need the computed key-value pairs to persist in a storage system to avoid re-computing them during the restarting process. In this paper, the task failure resilience (TFR) technique is proposed, which allows the execution of a failed task to continue from the point it was interrupted without having to redo all the work. Amazon ElastiCache for Redis is used as a non-volatile cache for the key-value pairs. We measured the performance of TFR by running different Hadoop benchmarking suites. TFR was implemented using the Hadoop software framework, and the experimental results showed significant performance improvements when compared with the performance of the default Hadoop implementation.

Comparing Energy Efficiency of MPI and MapReduce on ARM based Cluster (ARM 클러스터에서 에너지 효율 향상을 위한 MPI와 MapReduce 모델 비교)

  • Maqbool, Jahanzeb;Rizki, Permata Nur;Oh, Sangyoon
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.01a
    • /
    • pp.9-13
    • /
    • 2014
  • The performance of large scale software applications has been automatically increasing for last few decades under the influence of Moore's law - the number of transistors on a microprocessor roughly doubled every eighteen months. However, on-chip transistors limitations and heating issues led to the emergence of multicore processors. The energy efficient ARM based System-on-Chip (SoC) processors are being considered for future high performance computing systems. In this paper, we present a case study of two widely used parallel programming models i.e. MPI and MapReduce on distributed memory cluster of ARM SoC development boards. The case study application, Black-Scholes option pricing equation, was parallelized and evaluated in terms of power consumption and throughput. The results show that the Hadoop implementation has low instantaneous power consumption that of MPI, but MPI outperforms Hadoop implementation by a factor of 1.46 in terms of total power consumption to execution time ratio.

  • PDF

Distributed and Scalable Intrusion Detection System Based on Agents and Intelligent Techniques

  • El-Semary, Aly M.;Mostafa, Mostafa Gadal-Haqq M.
    • Journal of Information Processing Systems
    • /
    • v.6 no.4
    • /
    • pp.481-500
    • /
    • 2010
  • The Internet explosion and the increase in crucial web applications such as ebanking and e-commerce, make essential the need for network security tools. One of such tools is an Intrusion detection system which can be classified based on detection approachs as being signature-based or anomaly-based. Even though intrusion detection systems are well defined, their cooperation with each other to detect attacks needs to be addressed. Consequently, a new architecture that allows them to cooperate in detecting attacks is proposed. The architecture uses Software Agents to provide scalability and distributability. It works in two modes: learning and detection. During learning mode, it generates a profile for each individual system using a fuzzy data mining algorithm. During detection mode, each system uses the FuzzyJess to match network traffic against its profile. The architecture was tested against a standard data set produced by MIT's Lincoln Laboratory and the primary results show its efficiency and capability to detect attacks. Finally, two new methods, the memory-window and memoryless-window, were developed for extracting useful parameters from raw packets. The parameters are used as detection metrics.

Comparison of Genetic Algorithms and Simulated Annealing for Multiprocessor Task Allocation (멀티프로세서 태스크 할당을 위한 GA과 SA의 비교)

  • Park, Gyeong-Mo
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.9
    • /
    • pp.2311-2319
    • /
    • 1999
  • We present two heuristic algorithms for the task allocation problem (NP-complete problem) in parallel computing. The problem is to find an optimal mapping of multiple communicating tasks of a parallel program onto the multiple processing nodes of a distributed-memory multicomputer. The purpose of mapping these tasks into the nodes of the target architecture is the minimization of parallel execution time without sacrificing solution quality. Many heuristic approaches have been employed to obtain satisfactory mapping. Our heuristics are based on genetic algorithms and simulated annealing. We formulate an objective function as a total computational cost for a mapping configuration, and evaluate the performance of our heuristic algorithms. We compare the quality of solutions and times derived by the random, greedy, genetic, and annealing algorithms. Our experimental findings from a simulation study of the allocation algorithms are presented.

  • PDF

WLRU: Remote Cache Management Policy for Distributed Shared Memory Architectures (WLRU: 분산 공유 메모리 구조에 적합한 원격 캐시 관리 정책)

  • Suh Hyo-Joong;Lee Byong-Ho
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2005.07a
    • /
    • pp.61-63
    • /
    • 2005
  • 분산 메모리에 기반한 다중 프로세서 시스템은 기존의 중앙 집중형 메모리 구조의 단점인 메모리 접근의 병목현상을 극복하고 프로세서와 메모리의 부가에 따라 메모리 대역폭을 확장시킬 수 있는 구조로써 최근의 다중 프로세서 시스템 구조의 주류로 대두되고 있다. 다중 프로세서 시스템의 성능은 메모리 접근 지연에 의하여 제한 받고 있는데 이러한 이유는 프로세서의 동작 주파수 속도에 비하여 메모리의 접근 지연이 수십 배 이상이 되기 때문이다. 특히 분산 메모리 다중 프로세서 시스템에 있어서 메모리 접근은 지역 메모리 접근과 원격 메모리 접근의 두 가지 유형으로 나눌 수 있는데 이 중 원격 메모리 접근 지연은 시스템의 상호 접속망 구조에 따라 지역 메모리 접근 지연에 비하여 수 배 내지 수십 배에 이르고 있다. 본 논문에서는 분산 메모리 다중 프로세서 시스템에서 상호 접속 망의 구조에 따라 원격 메모리 접근 간에도 시간 지연의 차이가 있음에 착안하여 원격 메모리 접근 시간 지연에 따른 최적화 된 원격 캐시 관리 정책을 제시하며 각 상호 접속 망의 구조에 따라 이러한 캐시 관리 정책에 의한 성능 향상의 정도를 측정한다.

  • PDF

Implementation of Embedded Micro Web Server for Web based Remote Hardware Control and Monitor (웹 기반 하드웨어 원격감시 및 제어를 위한 초소형 내장형 웹 서버 시스템의 구현)

  • Han, Kyong-Ho
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.20 no.6
    • /
    • pp.104-110
    • /
    • 2006
  • In this paper, we proposed the micro web-server implementation on Strong ARM processor with embedded Linux. The parallel port connecting parallel I/O is controlled via HTTP protocol and web browser program HTTP protocol with Linux, the micro web server program and port control program are installed on-board memory using CGI to be accessed by web browser. The processor parallel input port is monitored and parallel output port is controlled from remote hosts via HTTP protocol. The result of the proposed embedded micro-web server can be used in remote automation systems, distributed control via internet using web browser.

Development of Consistency Algorithm for VOD streaming Data (VOD 스트리밍 데이터를 위한 Consistency 알고리즘 개발)

  • Jang Seung-Ju
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2006.05a
    • /
    • pp.729-732
    • /
    • 2006
  • This paper proposes and implements consistency algorithm that is serviced streaming data efficiently in VOD system. The media data is loaded into each node by Round Robin method. The barrier mechanism is changed into the minimum data factor(SH. GOP) of media data in this paper. In addition in order to fast media data service, the additional features are implemented in the consistency algorithm. Additional feature of the consistency algorithm is added. First, time synchronization algorithm is added the suggested consistency algorithm. Second, the prepaging mechanism supports efficient buffering service. I experimented the suggested consistency algorithm on two DSM systems.

  • PDF

A Connection of Information in the Ubiquitous Space (유비쿼터스 공간에서의 정보 연결)

  • Ko Sung-Bum
    • Journal of Internet Computing and Services
    • /
    • v.5 no.2
    • /
    • pp.1-15
    • /
    • 2004
  • The current Internet space is evolving to the so called Ubiquitous space. Unlike the Internet space, the information in the Ubiquitous space is distributed evenly in the places like computer's memory, human's brain and physical machine. The 'hypertext', the connection model of the information, which is originally designed for the Internet space doesn't suit well to the Ubiquitous space. From this point of view, we proposed the CPM model in this paper. The CPM model is designed for comprising the such three computing mechanism as analog computing, digital computing and human computing. In this paper, we showed that the characteristics of the CPM model might answer the such purpose as the connection of information in the Ubiquitous space.

  • PDF

Long-term Preservation of Digital Heritage: Building a National Strategy (디지털유산의 장기적 보존: 국가정책 수립을 위한 제안)

  • Lee, Soo Yeon
    • The Korean Journal of Archival Studies
    • /
    • no.10
    • /
    • pp.27-62
    • /
    • 2004
  • As the penetration of information technology into everyday life is accelerated day by day, virtually all kinds of human representation of knowledge and arts are produced and distributed in the digital form. It is problematic, however, because digital objects are so volatile that it is not easy to keep them in fixed form. The fatal fragility makes it extremely tricky to preserve the digital heritage of our time for the next generation. The present paper aims to introduce current endeavors made at the international and the national levels and to provide with suggestions for Korean national strategy of digital preservation. It starts with reviewing the global trends of digital archiving and long-term preservation, focusing on standardization, preservation strategies and current experiments and projects being conducted for preserving various digital objects. It then sketches national strategies of several leading countries. Based on the sketch, twofold suggestions for Korean national strategy are proposed: establishing a central coordinating agency and accommodating the digital preservation issue in the legislative and regulatory framework for the information society. The paper concludes with the necessity of cooperation among heritage organizations, including libraries, archives, museums. They should cooperate with each other because they have traditionally been trusted with the custodianship of collective memory of humankind and the digital heritage cannot be passed onto the next generation without their endeavor. They should also work together because any single institution, or any single nation could cover what it takes to complete the task of long-term preservation of our digital heritage.