• Title/Summary/Keyword: Distributed processing

Search Result 2,309, Processing Time 0.032 seconds

A Quality Evaluation Model for Distributed Processing Systems of Big Data (빅데이터 분산처리시스템의 품질평가모델)

  • Choi, Seung-Jun;Park, Jea-Won;Kim, Jong-Bae;Choi, Jae-Hyun
    • Journal of Digital Contents Society
    • /
    • v.15 no.4
    • /
    • pp.533-545
    • /
    • 2014
  • According to the evolving of IT technologies, the amount of data we are facing increasing exponentially. Thus, the technique for managing and analyzing these vast data that has emerged is a distributed processing system of big data. A quality evaluation for the existing distributed processing systems has been proceeded by the structured data environment. Thus, if we apply this to the evaluation of distributed processing systems of big data which has to focus on the analysis of the unstructured data, a precise quality assessment cannot be made. Therefore, a study of the quality evaluation model for the distributed processing systems is needed, which considers the environment of the analysis of big data. In this paper, we propose a new quality evaluation model by deriving the quality evaluation elements based on the ISO/IEC9126 which is the international standard on software quality, and defining metrics for validating the elements.

Development of the Dynamic Host Management Scheme for Parallel/Distributed Processing on the Web (웹 환경에서의 병렬/분산 처리를 위한 동적 호스트 관리 기법의 개발)

  • Song, Eun-Ha;Jeong, Young-Sik
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.8 no.3
    • /
    • pp.251-260
    • /
    • 2002
  • The parallel/distributed processing with a lot of the idle hosts on the web has the high coot-performance ratio for large-scale applications. It's processing has to show the solutions for unpredictable status such as heterogeneity of hosts, variability of hosts, autonomy of hosts, the supporting performance continuously, and the number of hosts which are participated in computation and so on. In this paper, we propose the strategy of adaptive tack reallocation based on performance the host job processing, spread out geographically Also, It shows the scheme of dynamic host management with dynamic environment, which is changed by lots of hosts on the web during parallel processing for large-scale applications. This paper implements the PDSWeb (Parallel/Distributed Scheme on Web) system, evaluates and applies It to the generation of rendering image with highly intensive computation. The results are showed that the adaptive task reallocation with the variation of hosts has been increased up to maximum 90% and the improvement in performance according to add/delete of hosts.

A Distributed Algorithmfor Weighted Shortest Path Problem (최단경로문제를 해결하는 효율적인 분산 알고리즘)

  • Park, Jeong-Ho;Park, Yun-Yong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.1
    • /
    • pp.42-48
    • /
    • 1999
  • Consider the situation that informations necessary to solve a certain problem are distributed among processors on a network. It is called a distributed algorithm that in this situation each processor exchanges the message with adjacent processors to solve the problems. This paper proposes a distributed algorithm to solve the problem that constructs the weighted shortest path tree in an asynchronous network system. In general, a distributed algorithm is estimated by the number of messages(message complexity of the distributed algorithm proposed in this paper are O(n53) and O(nln) respectively. where n is the number of processors on the network.

  • PDF

Performance of Distributed Database System built on Multicore Systems

  • Kim, Kangseok
    • Journal of Internet Computing and Services
    • /
    • v.18 no.6
    • /
    • pp.47-53
    • /
    • 2017
  • Recently, huge datasets have been generating rapidly in a variety of fields. Then, there is an urgent need for technologies that will allow efficient and effective processing of huge datasets. Therefore the problems of partitioning a huge dataset effectively and alleviating the processing overhead of the partitioned data efficiently have been a critical factor for scalability and performance in distributed database system. In our work we utilized multicore servers to provide scalable service to our distributed system. The partitioning of database over multicore servers have emerged from a need for new architectural design of distributed database system from scalability and performance concerns in today's data deluge. The system allows uniform access through a web service interface to concurrently distributed databases over multicore servers, using SQMD (Single Query Multiple Database) mechanism based on publish/subscribe paradigm. We will present performance results with the distributed database system built on multicore server, which is time intensive with traditional architectures. We will also discuss future works.

DMRUT-MCDS: Discovery Relationships in the Cyber-Physical Integrated Network

  • Lu, Hongliang;Cao, Jiannong;Zhu, Weiping;Jiao, Xianlong;Lv, Shaohe;Wang, Xiaodong
    • Journal of Communications and Networks
    • /
    • v.17 no.6
    • /
    • pp.558-567
    • /
    • 2015
  • In recent years, we have seen a proliferation of mobile-network-enabled smart objects, such as smart-phones and smart-watches, that form a cyber-physical integrated network to connect the cyber and physical worlds through the capabilities of sensing, communicating, and computing. Discovery of the relationship between smart objects is a critical and nontrivial task in cyber-physical integrated network applications. Aiming to find the most stable relationship in the heterogeneous and dynamic cyber-physical network, we propose a distributed and efficient relationship-discovery algorithm, called dynamically maximizing remaining unchanged time with minimum connected dominant set (DMRUT-MCDS) for constructing a backbone with the smallest scale infrastructure. In our proposed algorithm, the impact of the duration of the relationship is considered in order to balance the size and sustain time of the infrastructure. The performance of our algorithm is studied through extensive simulations and the results show that DMRUT-MCDS performs well in different distribution networks.

Experimental deployment and validation of a distributed SHM system using wireless sensor networks

  • Castaneda, Nestor E.;Dyke, Shirley;Lu, Chenyang;Sun, Fei;Hackmann, Greg
    • Structural Engineering and Mechanics
    • /
    • v.32 no.6
    • /
    • pp.787-809
    • /
    • 2009
  • Recent interest in the use of wireless sensor networks for structural health monitoring (SHM) is mainly due to their low implementation costs and potential to measure the responses of a structure at unprecedented spatial resolution. Approaches capable of detecting damage using distributed processing must be developed in parallel with this technology to significantly reduce the power consumption and communication bandwidth requirements of the sensor platforms. In this investigation, a damage detection system based on a distributed processing approach is proposed and experimentally validated using a wireless sensor network deployed on two laboratory structures. In this distributed approach, on-board processing capabilities of the wireless sensor are exploited to significantly reduce the communication load and power consumption. The Damage Location Assurance Criterion (DLAC) is used for localizing damage. Processing of the raw data is conducted at the sensor level, and a reduced data set is transmitted to the base station for decision-making. The results indicate that this distributed implementation can be used to successfully detect and localize regions of damage in a structure. To further support the experimental results obtained, the capabilities of the proposed system were tested through a series of numerical simulations with an expanded set of damage scenarios.

Processing Method of Mass Small File Using Hadoop Platform (하둡 플랫폼을 이용한 대량의 스몰파일 처리방법)

  • Kim, Chang-Bok;Chung, Jae-Pil
    • Journal of Advanced Navigation Technology
    • /
    • v.18 no.4
    • /
    • pp.401-408
    • /
    • 2014
  • Hadoop is composed with MapReduce programming model for distributed processing and HDFS distributed file system. Hadoop is suitable framework for big data processing, but processing of mass small files have many problems. The processing of mass small file in hadoop have problems to created one mapper per one file, and it have problems to needed many memory for store of meta information of file. This paper have comparison evaluation processing method of mass small file with various method in hadoop platform. The processing of general compression format is inadequate because of processing by one mapper regardless of data size. The processing of sequence and hadoop archive file is removed memory problem of namenode by compress and combine of small file. Hadoop archive file is faster then sequence file about combine time of small file. The processing using CombineFileInputFormat class is needed not combine of small file, and it have similar speed big data processing method.

Feasibility Study of a Distributed and Parallel Environment for Implementing the Standard Version of AAM Model

  • Naoui, Moulkheir;Mahmoudi, Said;Belalem, Ghalem
    • Journal of Information Processing Systems
    • /
    • v.12 no.1
    • /
    • pp.149-168
    • /
    • 2016
  • The Active Appearance Model (AAM) is a class of deformable models, which, in the segmentation process, integrates the priori knowledge on the shape and the texture and deformation of the structures studied. This model in its sequential form is computationally intensive and operates on large data sets. This paper presents another framework to implement the standard version of the AAM model. We suggest a distributed and parallel approach justified by the characteristics of the model and their potentialities. We introduce a schema for the representation of the overall model and we study of operations that can be parallelized. This approach is intended to exploit the benefits build in the area of advanced image processing.

Optimal Savepoint in a Loosely-Coupled Resilient Database System (느슨히 결합된 데이타베이스 시스템에서 최적의 저장점 유도)

  • Choe, Jae-Hwa;Kim, Seong-Eon
    • Asia pacific journal of information systems
    • /
    • v.6 no.1
    • /
    • pp.21-38
    • /
    • 1996
  • This paper investigates the performance improvement opportunities through a resiliency mechanism in the distributed primary/backup database system. Recognizing that a distributed transaction executes at several servers during its lifetime, we propose a resiliency mechanism which allows continuous transaction processing in distributed database server systems in the presence of a server failure. In order to perform continuous transaction processing despite failures, every state change of a transaction processing can be saved in the backup server. Obviously, this pessimistic synchronization may give more burdens than benefits to the system. Thus, the tracking needs not be done synchronously with the transaction progress. Instead, the state of all transaction processing in a system is saved periodically. This activity is referred to as a savepoint. Then, the question is how often the savepoint has to be done. We derive the optimal savepoint to identify the optimization parameters for the resilient transaction processing system.

  • PDF

FREE VIEWPOINT IMAGE RECONSTRUCTION FROM 3-D MULTI-FOCUS IMAGING SEQUENCES AND ITS IMPLEMENTATION BY CELL-BASED COMPUTING

  • Yonezawayz, Hiroki;Kodamay, Kazuya;Hamamotoz, Takayuki
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2009.01a
    • /
    • pp.751-754
    • /
    • 2009
  • This paper deals with the Cell-based distributed processing for generating free viewpoint images by merging multiple differently focused images. We previously proposed the method of generating free viewpoint images without any depth estimation. However, it is not so easy to realize real-time image reconstruction based on our previous method. In this paper, we discuss the method to reduce the processing time by dimension reduction for image filtering and Cell-based distributed processing. Especially, the method of high-speed image reconstruction by the Cell processor on SONY PLAYSTATION3(PS3) is described in detail. We show some experimental results by using real images and we discuss the possibility of real-time free viewpoint image reconstruction.

  • PDF