• Title/Summary/Keyword: Distributed memory

Search Result 397, Processing Time 0.022 seconds

A Distributed VOD Server Based on Virtual Interface Architecture and Interval Cache (버추얼 인터페이스 아키텍처 및 인터벌 캐쉬에 기반한 분산 VOD 서버)

  • Oh, Soo-Cheol;Chung, Sang-Hwa
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.33 no.10
    • /
    • pp.734-745
    • /
    • 2006
  • This paper presents a PC cluster-based distributed VOD server that minimizes the load of an interconnection network by adopting the VIA communication protocol and the interval cache algorithm. Video data is distributed to the disks of the distributed VOD server and each server node receives the data through the interconnection network and sends it to clients. The load of the interconnection network increases because of the large amount of video data transferred. This paper developed a distributed VOD file system, which is based on VIA, to minimize cost using interconnection network when accessing remote disks. VIA is a user-level communication protocol removing the overhead of TCP/IP. This papers also improved the performance of the interconnection network by expanding the maximum transfer size of VIA. In addition, the interval cache reduces traffic on the interconnection network by caching, in main memory, the video data transferred from disks of remote server nodes. Experiments using the distributed VOD server of this paper showed a maximum performance improvement of 21.3% compared with a distributed VOD server without VIA and the interval cache, when used with a four-node PC cluster.

Support vector machines for big data analysis (빅 데이터 분석을 위한 지지벡터기계)

  • Choi, Hosik;Park, Hye Won;Park, Changyi
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.5
    • /
    • pp.989-998
    • /
    • 2013
  • We cannot analyze big data, which attracts recent attentions in industry and academy, by batch processing algorithms developed in data mining because big data, by definition, cannot be uploaded and processed in the memory of a single system. So an imminent issue is to develop various leaning algorithms so that they can be applied to big data. In this paper, we review various algorithms for support vector machines in the literature. Particularly, we introduce online type and parallel processing algorithms that are expected to be useful in big data classifications and compare the strengths, the weaknesses and the performances of those algorithms through simulations for linear classification.

Parallel Sorting Algorithm by Median-Median (중위수의 중위수에 의한 병렬 분류 알고리즘)

  • Min, Yong-Sik
    • The Journal of the Acoustical Society of Korea
    • /
    • v.14 no.1E
    • /
    • pp.14-21
    • /
    • 1995
  • This paper presents a parallel sorting algorithm suitable for the SIMD multiprocessor. The algorithm finds pivots for partitioning the data into ordered subsets. The data can be evenly distributed to be sorted since it uses the probability theory. For n data elements to be sorted on p processors, when $n{\geq}p^2$, the algorithm is shown to be asymptotically optimal. In practice, sorting 8 million data items on 64 processors achieved a 48.43-fold speedup, while the PSRS required a 44.4-fold speedup. On a variety of shared and distributed memory machines, the algorithm achieved better than half-linear speedups.

  • PDF

Self-organized Distributed Networks for Precise Modelling of a System (시스템의 정밀 모델링을 위한 자율분산 신경망)

  • Kim, Hyong-Suk;Choi, Jong-Soo;Kim, Sung-Joong
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.31B no.11
    • /
    • pp.151-162
    • /
    • 1994
  • A new neural network structure called Self-organized Distributed Networks (SODN) is proposed for developing the neural network-based multidimensional system models. The learning with the proposed networks is fast and precise. Such properties are caused from the local learning mechanism. The structure of the networks is combination of dual networks such as self-organized networks and multilayered local networks. Each local networks learns only data in a sub-region. Large number of memory requirements and low generalization capability for the untrained region, which are drawbacks of conventional local network learning, are overcomed in the proposed networks. The simulation results of the proposed networks show better performance than the standard multilayer neural networks and the Radial Basis function(RBF) networks.

  • PDF

A Parallel HDFS and MapReduce Functions for Emotion Analysis (감성분석을 위한 병렬적 HDFS와 맵리듀스 함수)

  • Back, BongHyun;Ryoo, Yun-Kyoo
    • Journal of the Korea society of information convergence
    • /
    • v.7 no.2
    • /
    • pp.49-57
    • /
    • 2014
  • Recently, opinion mining is introduced to extract useful information from SNS data and to evaluate the true intention of users. Opinion mining are required several efficient techniques to collect and analyze a large amount of SNS data and extract meaningful data from them. Therefore in this paper, we propose a parallel HDFS(Hadoop Distributed File System) and emotion functions based on Mapreduce to extract some emotional information of users from various unstructured big data on social networks. The experiment results have verified that the proposed system and functions perform faster than O(n) for data gathering time and loading time, and maintain stable load balancing for memory and CPU resources.

  • PDF

The Communication Method at the Auto-Startup System using TCP/IP and VXI and Expert System(G2)

  • Kim, Jung-Soo;Joon Lyon
    • Transactions on Control, Automation and Systems Engineering
    • /
    • v.1 no.2
    • /
    • pp.141-146
    • /
    • 1999
  • This paper describes the communication method of an auto-startup system. The Auto-Startup system is designed to operate a nuclear power plant automatically during the startup operation . In general , the operations during startup in existing plant have only been manually controlled by the operator. The manual operation caused to the operator mistake. The Auto-Startup system consists of the Distributed Control System(DCS) and G2 (Expert System). Also, Functional Test Facility(FTF) provides the plant's real-data for an Auto-Startup system. So, it is necessary to develop the communication method between these systems. We developed two methods ; one is a network and the other is a hardwire line. To communicate between these systems (DCS-G2 and DCS-FTF) , we developed the communication program. In case of DCS-FTF, we used the TCP/IP and VXI. BUt, in case of DCS-G2 , we , what it called , developed the bridge program using the GSI(G2 Standard Interface). We test to check the function of the important parameter, in time, for analysis of the developed communication method. The results are a good performance when we check the communication time of important parameter. We conclude that Auto-startup system could save heat-up time about at least 5 hours and reduced the change of the reactor operation and trip.

  • PDF

Improving Data Accuracy Using Proactive Correlated Fuzzy System in Wireless Sensor Networks

  • Barakkath Nisha, U;Uma Maheswari, N;Venkatesh, R;Yasir Abdullah, R
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.9
    • /
    • pp.3515-3538
    • /
    • 2015
  • Data accuracy can be increased by detecting and removing the incorrect data generated in wireless sensor networks. By increasing the data accuracy, network lifetime can be increased parallel. Network lifetime or operational time is the time during which WSN is able to fulfill its tasks by using microcontroller with on-chip memory radio transceivers, albeit distributed sensor nodes send summary of their data to their cluster heads, which reduce energy consumption gradually. In this paper a powerful algorithm using proactive fuzzy system is proposed and it is a mixture of fuzzy logic with comparative correlation techniques that ensure high data accuracy by detecting incorrect data in distributed wireless sensor networks. This proposed system is implemented in two phases there, the first phase creates input space partitioning by using robust fuzzy c means clustering and the second phase detects incorrect data and removes it completely. Experimental result makes transparent of combined correlated fuzzy system (CCFS) which detects faulty readings with greater accuracy (99.21%) than the existing one (98.33%) along with low false alarm rate.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

Implementing I/O Bandwidth Sharing Scheme between Multiple Linux Containers based on Dm-zoned for Zoned Namespace SSDs

  • Seokjun Lee;Sungyong Ahn
    • International journal of advanced smart convergence
    • /
    • v.12 no.4
    • /
    • pp.237-245
    • /
    • 2023
  • In the cloud service, system resource such as CPU, memory, I/O bandwidth are shared among multiple users. Particularly, in Linux containers environment, I/O bandwidth is distributed in proportion to the weight of each container through the BFQ I/O scheduler. However, since the I/O scheduler can only be applied to conventional block storage devices, it cannot be applied to Zoned Namespace(ZNS) SSD, a new storage interface that has been recently studied. To overcome this limitation, in this paper, we implemented a weighted proportional I/O bandwidth sharing scheme for ZNS SSDs in dm-zoned, which emulates conventional block storage using ZNS SSDs. Each user receives a different amount of budget, which is required to process the user's I/O requests based on the user's weight. If the budget is exhausted I/O requests cannot be processed and requests are queued until the budget replenished. Each budget refill period, the budget is replenished based on the user's weight. In the experiment, as a result, we can confirm that the I/O bandwidth can be distributed on their weight as we expected.

Transactive Memory System of a Virtual Team : Theoretical Exploration and Empirical Examination (가상 팀의 교류활성기억 시스템과 팀 성과의 관계 : 가상 팀 속성을 선행요인으로)

  • Shin, Kyung-Shik;Suh, A-Young
    • The Journal of Society for e-Business Studies
    • /
    • v.15 no.2
    • /
    • pp.137-166
    • /
    • 2010
  • A virtual team is defined a group of people that use electronic communications for some or all of their interactions with other team members. Because team members of a virtual team are physically and temporally distributed, a team's transactive memory system(TMS) is considered to be crucial for the team's effectiveness and performance. TMS refers to a set of individual memory systems which integrate knowledge possessed by particular members through a shared awareness of who knows what. This paper seeks to understand (1) how a virtual team's TMS influences team performance, and (2) what factors contribute to developing the team's TMS. Given these purposes, through the extensive literature review, we first identified components and antecedents to develop a theoretical model that predicts a virtual team's performance. Using the survey data gathered from 172 virtual teams, this study found that expertise location, coordination, and cognition-based trust which were proposed as three components of TMS positively influenced a virtual team's performance. Furthermore, this study uncovered that perceived media richness, network tie strength, and shared norms significantly influenced the components of TMS, while geographical dispersion did not exert any significant influence on TMS.