• 제목/요약/키워드: Distributed Data

검색결과 6,037건 처리시간 0.034초

분산 컴퓨팅 환경하에서의 데이타 자원 관리 (Data Resource Management under Distributed Computing Environment)

  • 조희경;안중호
    • 한국데이타베이스학회:학술대회논문집
    • /
    • 한국데이타베이스학회 1994년도 DB산업기술 활성화를 위한 학술대회 및 기술 심포지움
    • /
    • pp.105-129
    • /
    • 1994
  • The information system of corporations are facing a new environment expressed by miniaturization, decentralization and Open System. It is therefore of utmost importance for corporations to adapt flexibly th such new environment by providing for corresponding changes to their existing information systems. The objectives of this study are to identify this new environment faced by today′s information system and develop effective methods for data resource management under this new environment. In this study, it is assumed that the new environment faced by information systems can be specified as Distributed Computing Environment, and in order to achieve such system, presents Client/server architecture as its representative computing structure, This study defines Client/server architecture as a computing architecture which specialize the fuctionality of the client system and the server system in order to have an application distribute and perform cooperative processing at the best platform. Furthermore, from among the five structures utilized in Client/server architecture for distribution and cooperative processing of application between server and client this study presents two different data management methods under the Client/server environment; one is "Remote Data Management Method" which uses file server or database server and. the other is "Distributed Data Management Method" using distributed database management system. The result of this study leads to the conclusion that in the client/server environment although distributed application is assumed, the data could become centralized (in the case of file server or database server) or decentralized (in the case of distributed database system) and the data management method through a distributed database system where complete responsibility and powers with respect to control of data used by the user are given not only is it more adaptable to modern flexible corporate environment, but in terms of system operation, it presents a more efficient data management alternative compared to existing data management methods in terms of cutting costs.

  • PDF

A Distributed Privacy-Utility Tradeoff Method Using Distributed Lossy Source Coding with Side Information

  • Gu, Yonghao;Wang, Yongfei;Yang, Zhen;Gao, Yimu
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제11권5호
    • /
    • pp.2778-2791
    • /
    • 2017
  • In the age of big data, distributed data providers need to ensure the privacy, while data analysts need to mine the value of data. Therefore, how to find the privacy-utility tradeoff has become a research hotspot. Besides, the adversary may have the background knowledge of the data source. Therefore, it is significant to solve the privacy-utility tradeoff problem in the distributed environment with side information. This paper proposes a distributed privacy-utility tradeoff method using distributed lossy source coding with side information, and quantitatively gives the privacy-utility tradeoff region and Rate-Distortion-Leakage region. Four results are shown in the simulation analysis. The first result is that both the source rate and the privacy leakage decrease with the increase of source distortion. The second result is that the finer relevance between the public data and private data of source, the finer perturbation of source needed to get the same privacy protection. The third result is that the greater the variance of the data source, the slighter distortion is chosen to ensure more data utility. The fourth result is that under the same privacy restriction, the slighter the variance of the side information, the less distortion of data source is chosen to ensure more data utility. Finally, the provided method is compared with current ones from five aspects to show the advantage of our method.

Performance of Distributed Database System built on Multicore Systems

  • Kim, Kangseok
    • 인터넷정보학회논문지
    • /
    • 제18권6호
    • /
    • pp.47-53
    • /
    • 2017
  • Recently, huge datasets have been generating rapidly in a variety of fields. Then, there is an urgent need for technologies that will allow efficient and effective processing of huge datasets. Therefore the problems of partitioning a huge dataset effectively and alleviating the processing overhead of the partitioned data efficiently have been a critical factor for scalability and performance in distributed database system. In our work we utilized multicore servers to provide scalable service to our distributed system. The partitioning of database over multicore servers have emerged from a need for new architectural design of distributed database system from scalability and performance concerns in today's data deluge. The system allows uniform access through a web service interface to concurrently distributed databases over multicore servers, using SQMD (Single Query Multiple Database) mechanism based on publish/subscribe paradigm. We will present performance results with the distributed database system built on multicore server, which is time intensive with traditional architectures. We will also discuss future works.

대용량 로그 데이터 처리를 위한 분산 실시간 자가 진단 시스템 (A Distributed Real-time Self-Diagnosis System for Processing Large Amounts of Log Data)

  • 손시운;김다솔;문양세;최형진
    • 데이타베이스연구회지:데이타베이스연구
    • /
    • 제34권3호
    • /
    • pp.58-68
    • /
    • 2018
  • 분산 컴퓨팅이란 다수의 서버로 구성된 분산 시스템에서 데이터를 효율적으로 저장 및 처리하는 기술이다. 따라서 분산 시스템을 구성하는 서버의 상태에 따라 분산 컴퓨팅의 성능에 큰 영향을 미친다. 본 논문은 분산 시스템에서 실시간으로 발생하는 시스템 자원의 로그 데이터를 수집하고 이상을 탐지하여 결과를 시각화하는 자가 진단 시스템을 제안한다. 먼저, 자가 진단 과정을 수집, 전달, 분석, 저장, 시각화의 다섯 단계로 구분한다. 다음으로, 자가 진단 과정이 실시간성, 확장성, 고가용성의 목표를 만족하도록 실시간 자가 진단 시스템을 설계한다. 본 시스템은 대표적인 실시간 분산 기술인 Apache Flume, Apache Kafka, Apache Storm을 기반으로 구현되어 실시간성, 확장성, 고가용성의 세 가지 목표를 만족할 수 있다. 또한, 자가 진단 과정에서 로그 데이터 처리의 지연을 최소화하도록 간단하지만 효과적인 이동 평균 및 3-시그마 기반 이상 탐지 기법을 사용한다. 본 논문의 결과를 통해, 분산 시스템 내에서 서버 상태를 실시간으로 진단할 수 있는 분산 실시간 자가 진단 시스템을 구축할 수 있다.

Students' Performance Prediction in Higher Education Using Multi-Agent Framework Based Distributed Data Mining Approach: A Review

  • M.Nazir;A.Noraziah;M.Rahmah
    • International Journal of Computer Science & Network Security
    • /
    • 제23권10호
    • /
    • pp.135-146
    • /
    • 2023
  • An effective educational program warrants the inclusion of an innovative construction which enhances the higher education efficacy in such a way that accelerates the achievement of desired results and reduces the risk of failures. Educational Decision Support System (EDSS) has currently been a hot topic in educational systems, facilitating the pupil result monitoring and evaluation to be performed during their development. Insufficient information systems encounter trouble and hurdles in making the sufficient advantage from EDSS owing to the deficit of accuracy, incorrect analysis study of the characteristic, and inadequate database. DMTs (Data Mining Techniques) provide helpful tools in finding the models or forms of data and are extremely useful in the decision-making process. Several researchers have participated in the research involving distributed data mining with multi-agent technology. The rapid growth of network technology and IT use has led to the widespread use of distributed databases. This article explains the available data mining technology and the distributed data mining system framework. Distributed Data Mining approach is utilized for this work so that a classifier capable of predicting the success of students in the economic domain can be constructed. This research also discusses the Intelligent Knowledge Base Distributed Data Mining framework to assess the performance of the students through a mid-term exam and final-term exam employing Multi-agent system-based educational mining techniques. Using single and ensemble-based classifiers, this study intends to investigate the factors that influence student performance in higher education and construct a classification model that can predict academic achievement. We also discussed the importance of multi-agent systems and comparative machine learning approaches in EDSS development.

분산 환경에서 신경망을 응용한 데이터 서버 마이닝 (Data Server Mining applied Neural Networks in Distributed Environment)

  • 박민기;김귀태;이재완
    • 한국정보통신학회:학술대회논문집
    • /
    • 한국해양정보통신학회 2003년도 춘계종합학술대회
    • /
    • pp.473-476
    • /
    • 2003
  • 오늘날 인터넷은 하나의 거대한 분산 정보 서비스센터의 역할을 수행하며 여러 가지 많은 정보들과 이를 관리 운영하는 데이터 베이스 서버들은 분산된 네트워크 환경 속에서 광범위하게 존재하고 있다. 그러나 우리는 데이터 특성에 따라 입력 데이터를 처리할 서버를 결정하는데 여러 가지 어려움을 겪고 있다. 본 논문에서는 분산 환경 속에 존재하는 수많은 데이터들 가운데 신경망을 이용해 입력 데이터 패턴을 가장 효율적으로 처리할 수 있는 목적지 서버를 마이닝하는 기법과 이를 기반으로 한 지능적 데이터 마이닝 시스템 구조를 설계하였다. 그 결과로서 새로운 입력 데이터패턴이 신경망으로 구현된 동적 바인딩 방법에 따라 목적지 서버를 결정한 후 처리됨을 보였다. 이 기법은 데이터 웨어하우스, 통신 및 전력부하패턴 분석, 인구센서스 분석, 의료데이터 분석에 활용될 수 있다.

  • PDF

대규모 분산 컴퓨팅 환경에서 확장성을 고려한 실시간 데이터 공급 기법 (Scalable Data Provisioning Scheme on Large-Scale Distributed Computing Environment)

  • 김병상;윤찬현
    • 정보처리학회논문지A
    • /
    • 제18A권4호
    • /
    • pp.123-128
    • /
    • 2011
  • 본 논문은 원격지간의 연결된 대규모 분산 환경에서 데이터 분석 작업의 실행을 위해 필수적으로 고려되는 데이터 전송 부하를 감소시키는 기법을 제안한다. 계산 노드들이 밀집된 지역 인근에 다수의 데이터 노드를 배치시킴으로서 계산 노드들이 단일 데이터센터가 아닌 자신과 인접한 데이터 노드에 접근하여 작업을 수행함으로서 전송부하를 감소시키고 확장성을 증가시키는 것이 가능하다. 따라서 본 논문은 지역적으로 분산된 데이터 노드들의 데이터 처리율을 기반으로 실시간 데이터 공급을 수행함으로서 전송 지연을 최소화 할 수 있는 이론적인 모델과 시뮬레이션을 통한 성능 평가를 수행한다. 제안된 기법은 PRAGMA 그리드 테스트베드에서 실험을 통하여 성능의 우수성을 검증하였다.

Scheduling algirithm of data sampling times in the real-time distributed control systems

  • Hong, Seung-Ho
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 제어로봇시스템학회 1992년도 한국자동제어학술회의논문집(국제학술편); KOEX, Seoul; 19-21 Oct. 1992
    • /
    • pp.112-117
    • /
    • 1992
  • The Real-time Distributed Control Systems(RDCS) consist of several distributed control processes which share a network medium to exchange their data. Performance of feedback control loops in the RDCS is subject to the network-induced delays from sensor to controller and from controller to actuator. The network-induced delays are directly dependent upon the data sampling times of the control components which share a network medium. In this study, a scheduling algorithm of determining data sampling times is developed using the window concept, where the sampling data from the control components dynamically share a limited number of windows.

  • PDF

Application of Multi-Frontal Method in Collaborative Engineering Environment

  • Cho, Seong-Wook;Choi, Young;Lee, Gyu-Bong;Kwon, Ki-Eak
    • International Journal of CAD/CAM
    • /
    • 제3권1_2호
    • /
    • pp.51-60
    • /
    • 2003
  • The growth of the World Wide Web and the advances in high-speed network access have greatly changed existing CAD/CAE environment. The WWW has enabled us to share various distributed product data and to collaborate in the design process. An international standard for the product model data, STEP, and a standard for the distributed object technology, CORBA, are very important technological components for the interoperability in the advanced design and manufacturing environment. These two technologies provide background for the sharing of product data and the integration of applications on the network. This paper describes a distributed CAD/CAE environment that is integrated on the network by CORBA and product model data standard STEP. Several prototype application modules were implemented to verify the proposed concept and the test result is discussed. Finite element analysis server are further distributed into several frontal servers for the implementation of distributed parallel solution of finite element system equations. Distributed computation of analysis server is also implemented by using CORBA for the generalization of the proposed method.

맵리듀스를 사용한 디컴바인드 분산 VQ 코드북 생성 방법 (Decombined Distributed Parallel VQ Codebook Generation Based on MapReduce)

  • 이현진
    • 디지털콘텐츠학회 논문지
    • /
    • 제15권3호
    • /
    • pp.365-371
    • /
    • 2014
  • 빅 데이터(Big Data)시대로 접어들면서 기존의 IT 환경에서 만들어진 알고리즘들은 하둡과 같은 분산 아키텍처에 그대로 적용할 수 없거나 효율이 떨어진다. 따라서, 맵리듀스와 같은 분산 프레임워크를 적용한 새로운 알고리즘들이 필요하다. 벡터 양자화에 많이 사용되는 Lloyd의 알고리즘도 맵리듀스를 사용하여 개발이 이루어지고 있다. 본 논문에서는 기존의 맵리듀스를 사용한 분산 VQ 코드북 생성 알고리즘을 수정하여 좀 더 빠른 분석 결과를 보일 수 있는 디컴바인드 분산 VQ 코드북 생성 알고리즘을 제안하였다. 제안하는 알고리즘을 빅 데이터에 적용한 결과 기존 방법보다 높은 성능을 보인 것을 확인할 수 있었다.