• 제목/요약/키워드: Distributed memory

검색결과 397건 처리시간 0.028초

빅데이터 처리율 향상을 위한 인-메모리 기반 하이브리드 빅데이터 처리 기법 연구 (Study of In-Memory based Hybrid Big Data Processing Scheme for Improve the Big Data Processing Rate)

  • 이협건;김영운;김기영
    • 한국정보전자통신기술학회논문지
    • /
    • 제12권2호
    • /
    • pp.127-134
    • /
    • 2019
  • IT기술의 발달로 인해 생성되는 데이터의 양은 매년 기하급수적으로 증가하고 있으며, 이에 대한 대안으로 분산시스템과 인-메모리 기반 빅데이터 처리 기법의 연구가 활발히 이루어지고 있다. 기존 빅데이터 처리 기법들의 처리 성능은 노드의 수와 메모리 용량이 증가될수록 보다 빠르게 빅데이터 처리한다. 그러나 노드의 수의 증가는 빅데이터 인프라 환경에서 장애발생 빈도가 높아지며, 인프라 관리 포인트 및 인프라 운영비용도 증가된다. 또한 메모리 용량의 증가는 노드 구성에 대한 인프라 비용이 증가된다. 이에 본 논문에서는 빅데이터 처리율 향상을 위한 인-메모리 기반 하이브리드 빅데이터 처리 기법을 제안한다. 제안하는 기법은 분산시스템 처리기법에 Combiner 단계를 추가하고, 그 단계에서 인-메모리 기반 처리 기술을 적용하여 기존 분산시스템 기반 빅데이터 처리기법에 비해 빅데이터 처리시간을 약 22% 감소시켰다. 향후, 제안하는 기법의 실질적인 검증을 위해 더 많은 노드로 구성된 빅데이터 인프라 환경에서의 현실적 성능평가가 필요하다.

전-후 처리 과정을 포함한 거대 구조물의 유한요소 해석을 위한 효율적 데이터 구조 (Efficient Data Management for Finite Element Analysis with Pre-Post Processing of Large Structures)

  • 박시형;박진우;윤태호;김승조
    • 한국전산구조공학회:학술대회논문집
    • /
    • 한국전산구조공학회 2004년도 봄 학술발표회 논문집
    • /
    • pp.389-395
    • /
    • 2004
  • We consider the interface between the parallel distributed memory multifrontal solver and the finite element method. We give in detail the requirement and the data structure of parallel FEM interface which includes the element data and the node array. The full procedures of solving a large scale structural problem are assumed to have pre-post processors, of which algorithm is not considered in this paper. The main advantage of implementing the parallel FEM interface is shown up in the case that we use a distributed memory system with a large number of processors to solve a very large scale problem. The memory efficiency and the performance effect are examined by analyzing some examples on the Pegasus cluster system.

  • PDF

Concurrency Control Method to Provide Transactional Processing for Cloud Data Management System

  • Choi, Dojin;Song, Seokil
    • International Journal of Contents
    • /
    • 제12권1호
    • /
    • pp.60-64
    • /
    • 2016
  • As new applications of cloud data management system (CDMS) such as online games, cooperation edit, social network, and so on, are increasing, transaction processing capabilities for CDMS are required. Several transaction processing methods for cloud data management system (CDMS) have been proposed. However, existing transaction processing methods have some problems. Some of them provide limited transaction processing capabilities. Some of them are hard to be integrated with existing CDMSs. In this paper, we proposed a new concurrency control method to support transaction processing capability for CDMS to solve these problems. The proposed method was designed and implemented based on Spark, an in-memory distributed processing framework. It uses RDD (Resilient Distributed Dataset) model to provide fault tolerant to data in the main memory. In our proposed method, database stored in CDMS is loaded to main memory managed by Spark. The loaded data set is then transformed to RDD. In addition, we proposed a multi-version concurrency control method through immutable characteristics of RDD. Finally, we performed experiments to show the feasibility of the proposed method.

Thermomechanical interactions in a non local thermoelastic model with two temperature and memory dependent derivatives

  • Lata, Parveen;Singh, Sukhveer
    • Coupled systems mechanics
    • /
    • 제9권5호
    • /
    • pp.397-410
    • /
    • 2020
  • The present investigation is concerned with two-dimensional deformation in a homogeneous isotropic non local thermoelastic solid with two temperatures due to thermomechanical sources. The theory of memory dependent derivatives has been used for the study. The bounding surface is subjected to concentrated and distributed sources (mechanical and thermal sources). The Laplace and Fourier transforms have been used for obtaining the solution to the problem in the transformed domain. The analytical expressions for displacement components, stress components and conductive temperature are obtained in the transformed domain. For obtaining the results in the physical domain, numerical inversion technique has been applied. Numerical simulated results have been depicted graphically for explaining the effects of nonlocal parameter on the components of displacements, stresses and conductive temperature. Some special cases have also been deduced from the present study. The results obtained in the investigation should be useful for new material designers, researchers and physicists working in the field of nonlocal material sciences.

New execution model for CAPE using multiple threads on multicore clusters

  • Do, Xuan Huyen;Ha, Viet Hai;Tran, Van Long;Renault, Eric
    • ETRI Journal
    • /
    • 제43권5호
    • /
    • pp.825-834
    • /
    • 2021
  • Based on its simplicity and user-friendly characteristics, OpenMP has become the standard model for programming on shared-memory architectures. Checkpointing-aided parallel execution (CAPE) is an approach that utilizes the discontinuous incremental checkpointing technique (DICKPT) to translate and execute OpenMP programs on distributed-memory architectures automatically. Currently, CAPE implements the OpenMP execution model by utilizing the DICKPT to distribute parallel jobs and their data to slave machines, and then collects the results after executing these distributed jobs. Although this model has been proven to be effective in terms of performance and compatibility with OpenMP on distributed-memory systems, it cannot fully exploit the capabilities of multicore processors. This paper presents a novel execution model for CAPE that utilizes two levels of parallelism. In the proposed model, we add another level of parallelism in the form of multithreaded processes on slave machines with the goal of better exploiting their multicore CPUs. Initial experimental results presented near the end of this paper demonstrate that this model provides significantly enhanced CAPE performance.

Data Flow 시스템에서 구조체 분산 처리 방식 (A Structure Distributed Processing Method in Data Flow Systems)

  • 맹성열;현운몽;하영호;임인철
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 1987년도 전기.전자공학 학술대회 논문집(II)
    • /
    • pp.1125-1128
    • /
    • 1987
  • This paper proposes a method which distributes the structure data represented by a tree and handles it. To distribute and handle the structure data, this method partitions a structure data and distributes the partitioned structure in multiple processing element and allocates the partitioned structure. Each processing element includes the structure memory to store the partitioned structure and the structure controller to handle efficiently the distributed structure. As the structure is distributed and is stored in the structure memory and is handled by the structure controller, the processing time is reduced.

  • PDF

전력 조류 계산의 분산 병렬처리기법에 관한 연구 (A Development of Distributed Parallel Processing algorithm for Power Flow analysis)

  • 이춘모;이해기
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2001년도 학술대회 논문집 전문대학교육위원
    • /
    • pp.134-140
    • /
    • 2001
  • Parallel processing has the potential to be cost effectively used on computationally intense power system problems. But this technology is not still available is not only parallel computer but also parallel processing scheme. Testing these algorithms to ensure accuracy, and evaluation of their performance is also an issue. Although a significant amount of parallel algorithms of power system problem have been developed in last decade, actual testing on processor architectures lies in the beginning stages. This paper presents the parallel processing algorithm to supply the base being able to treat power flow by newton's method by the distributed memory type parallel computer. This method is to assign and to compute teared blocks of sparse matrix at each parallel processors. The testing to insure accuracy of developed method have been done on serial computer by trying to simulate a parallel environment.

  • PDF

분산 메모리 다중프로세서 환경에서의 병렬 음성인식 모델 (A Parallel Speech Recognition Model on Distributed Memory Multiprocessors)

  • 정상화;김형순;박민욱;황병한
    • 한국음향학회지
    • /
    • 제18권5호
    • /
    • pp.44-51
    • /
    • 1999
  • 본 논문에서는 음성과 자연언어의 통합처리를 위한 효과적인 병렬계산모델을 제안한다. 음소모델은 연속 Hidden Markov Model(HMM)에 기반을 둔 문맥종속형 음소를 사용하며, 언어모델은 지식베이스를 기반으로 한다. 또한 지식베이스를 구성하기 위해 계층구조의 semantic network과 병렬 marker-passing을 추론 메카니즘으로 쓰는 memory-based parsing 기술을 사용한다. 본 연구의 병렬 음성인식 알고리즘은 분산메모리 MIMD(Multiple Instruction Multiple Data) 구조의 다중 Transputer 시스템을 이용하여 구현되었다. 실험결과, 본 연구의 지식베이스 기반 음성인식 시스템의 인식률이 word network 기반 음성인식 시스템보다 높게 나타났으며 code-phoneme 통계정보를 활용하여 인식성능의 향상도 얻을 수 있었다. 또한, 성능향상도(speedup) 관련 실험들을 통하여 병렬 음성인식 시스템의 실시간 구현 가능성을 확인하였다.

  • PDF

Single Junction Charge Pumping 방법을 이용한 전하 트랩형 SONOSFET NVSM 셀의 기억 트랩분포 결정 (Determination of Memory Trap Distribution in Charge Trap Type SONOSFET NVSM Cells Using Single Junction Charge Pumping Method)

  • 양전우;홍순혁;서광열
    • 한국전기전자재료학회논문지
    • /
    • 제13권10호
    • /
    • pp.822-827
    • /
    • 2000
  • The Si-SiO$_2$interface trap and nitride bulk trap distribution of SONOSFET(polysilicon-oxide-nitride-oxide-semiconductor field effect transistor) NVSM (nonvolatile semiconductor memory) cell is investigated by single junction charge pumping method. The device was fabricated by 0.35㎛ standard logic fabrication process including the ONO stack dielectrics. The thickness of ONO dielectricis are 24$\AA$ for tunnel oxide, 74 $\AA$ for nitride and 25 $\AA$ for blocking oxide, respectively. By the use of single junction charge pumping method, the lateral profiles of both interface and memory traps can be calculated directly from experimental charge pumping results without complex numerical simulation. The interface traps were almost uniformly distributed over the whole channel region and its maximum value was 7.97$\times$10$\^$10/㎠. The memory traps were uniformly distributed in the nitride layer and its maximum value was 1.04$\times$10$\^$19/㎤. The degradation characteristics of SONOSFET with write/erase cycling also were investigated.

  • PDF

Bandwidth-aware Memory Placement on Hybrid Memories targeting High Performance Computing Systems

  • Lee, Jongmin
    • 한국컴퓨터정보학회논문지
    • /
    • 제24권8호
    • /
    • pp.1-8
    • /
    • 2019
  • Modern computers provide tremendous computing capability and a large memory system. Hybrid memories consist of next generation memory devices and are adopted in high performance systems. However, the increased complexity of the microprocessor makes it difficult to operate the system effectively. In this paper, we propose a simple data migration method called Bandwidth-aware Data Migration (BDM) to efficiently use memory systems for high performance processors with hybrid memory. BDM monitors the status of applications running on the system using hardware performance monitoring tools and migrates the appropriate pages of selected applications to High Bandwidth Memory (HBM). BDM selects applications whose bandwidth usages are high and also evenly distributed among the threads. Experimental results show that BDM improves execution time by an average of 20% over baseline execution.