• Title/Summary/Keyword: Shared Data Storage

Search Result 75, Processing Time 0.023 seconds

Big Data Management System for Biomedical Images to Improve Short-term and Long-term Storage

  • Qamar, Shamweel;Kim, Eun Sung;Park, Peom
    • Journal of the Korean Society of Systems Engineering
    • /
    • v.15 no.2
    • /
    • pp.66-71
    • /
    • 2019
  • In digital pathology, an electronic system in the biomedical domain storage of the files is a big constrain and because all the analysis and annotation takes place at every user-end manually, it becomes even harder to manage the data that is being shared inside an enterprise. Therefore, we need such a storage system which is not only big enough to store all the data but also manage it and making communication of that data much easier without losing its true from. A virtual server setup is one of those techniques which can solve this issue. We set a main server which is the main storage for all the virtual machines(that are being used at user-end) and that main server is controlled through a hypervisor so that if we want to make changes in storage overall or the main server in itself, it could be reached remotely from anywhere by just using the server's IP address. The server in our case includes XML-RPC based API which are transmitted between computers using HTTP protocol. JAVA API connects to HTTP/HTTPS protocol through JAVA Runtime Environment and exists on top of other SDK web services for the productivity boost of the running application. To manage the server easily, we use Tkinter library to develop the GUI and pmw magawidgets library which is also utilized through Tkinter. For managing, monitoring and performing operations on virtual machines, we use Python binding to XML-RPC based API. After all these settings, we approach to make the system user friendly by making GUI of the main server. Using that GUI, user can perform administrative functions like restart, suspend or resume a virtual machine. They can also logon to the slave host of the pool in case of emergency and if needed, they can also filter virtual machine by the host. Network monitoring can be performed on multiple virtual machines at same time in order to detect any loss of network connectivity.

Metadata Structrues of Huge Shared Disk File System for Large Files in GIS (GIS에서 대용량 파일을 위한 대용량 공유 디스크 파일시스템의 메타데이터 구조)

  • 김경배;이용주;박춘서;신범주
    • Spatial Information Research
    • /
    • v.10 no.1
    • /
    • pp.93-106
    • /
    • 2002
  • The traditional file system are designed to store and manage fur small size files. So. we cannot process the huge files related with geographic information data using the traditional file system such as unix file system or linux file system. In this paper, we propose new metadata structures and management mechanisms for the large file system in geographic information system. The proposed mechanisms use dynamic multi-level mode for large files and dynamic bitmap for huge file system. We implement the proposed mechanisms in the metadata structures of SANtopia is shared disk huge file system for storage area networks(SAN).

  • PDF

SANfs: A Scalable Shared File System for Linux Clusters (SANfs: 리눅스 클러스터를 위한 확장성 있는 공유파일시스템)

  • 황주영;안철우;박규호
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2001.10c
    • /
    • pp.91-93
    • /
    • 2001
  • 본 논문에서는 Storage Area Network기반의 대규모용량의 저장장치에 적합한 확장성 있는 공유파일시스템을 제안하고, Linux상에서의 구현을 보인다. 락의 단위는 블록이 아닌 파일단위로 하므로써 락오버헤드를 줄이고, callback방식의 파일 락을 사용한다. 파일데이터일관성 및 디렉토리캐쉬 일관성유지를 위한 Linux상에서의 구현방법을 보인다. 메타서버의 free block관리부담을 줄이기 위해서 분산 free block관리법을 사용한다. 또한 Inode와 data를 분리 저장함으로써 성능을 최적화한다.

  • PDF

Functional Requirements of Data Repository for DMP Support and CoreTrustSeal Authentication

  • Kim, Sun-Tae
    • International Journal of Knowledge Content Development & Technology
    • /
    • v.10 no.1
    • /
    • pp.7-20
    • /
    • 2020
  • For research data to be shared without legal, financial and technical barriers in the Open Science era, data repositories must have the functional requirements asked by DMP and CoreTrustSeal. In order to derive functional requirements for the data repository, this study analyzed the Data Management Plan (DMP) and CoreTrustSeal, the criteria for certification of research data repositories. Deposit, Ethics, License, Discovery, Identification, Reuse, Security, Preservation, Accessibility, Availability, and (Meta) Data Quality, commonly required by DMP and CoreTrustSeal, were derived as functional requirements that should be implemented first in implementing data repositories. Confidentiality, Integrity, Reliability, Archiving, Technical Infrastructure, Documented Storage Procedure, Organizational Infrastructure, (Meta) Data Evaluation, and Policy functions were further derived from CoreTrustSeal. The functional requirements of the data repository derived from this study may be required as a key function when developing the repository. It is also believed that it could be used as a key item to introduce repository functions to researchers for depositing data.

A COMPARATIVE STUDY ON BLOCKCHAIN DATA MANAGEMENT SYSTEMS: BIGCHAINDB VS FALCONDB

  • Abrar Alotaibi;Sarah Alissa;Salahadin Mohammed
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.5
    • /
    • pp.128-134
    • /
    • 2023
  • The widespread usage of blockchain technology in cryptocurrencies has led to the adoption of the blockchain concept in data storage management systems for secure and effective data storage and management. Several innovative studies have proposed solutions that integrate blockchain with distributed databases. In this article, we review current blockchain databases, then focus on two well-known blockchain databases-BigchainDB and FalconDB-to illustrate their architecture and design aspects in more detail. BigchainDB is a distributed database that integrates blockchain properties to enhance immutability and decentralization as well as a high transaction rate, low latency, and accurate queries. Its architecture consists of three layers: the transaction layer, consensus layer, and data model layer. FalconDB, on the other hand, is a shared database that allows multiple clients to collaborate on the database securely and efficiently, even if they have limited resources. It has two layers: the authentication layer and the consensus layer, which are used with client requests and results. Finally, a comparison is made between the two blockchain databases, revealing that they share some characteristics such as immutability, low latency, permission, horizontal scalability, decentralization, and the same consensus protocol. However, they vary in terms of database type, concurrency mechanism, replication model, cost, and the usage of smart contracts.

A Study on the Analysis of Aviation Safety Data Structure and Standard Classification (항공안전데이터 구조 분석 및 표준 분류체계에 관한 연구)

  • Kim, Jun Hwan;Lim, Jae Jin;Lee, Jang Ryong
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.28 no.4
    • /
    • pp.89-101
    • /
    • 2020
  • In order to enhance the safety of the international aviation industry, the International Civil Aviation Organization has recommended establishing an operational foundation for systematic and integrated collection, storage, analysis and sharing of aviation safety data. Accordingly, the Korea aviation industry also needs to comprehensively manage the safety data which generated and collected by various stakeholders related to aviation safety, and through this, it is necessary to previously identify and remove hazards that may cause accident. For more effective data management and utilization, a standard structure should be established to enable integrated management and sharing of safety data. Therefore, this study aims to propose the framework about how to manage and integrate the aviation safety data for big data-based aviation safety management and shared platform.

Adoption of MFER and HL7 Standard for Shared Electronic Medical Record (공유 전자의무기록을 위한 MFER과 HL7 표준 적용)

  • Kim, Hwa-Sun;Park, Chun-Bok;Hong, Hae-Sook;Cho, Hune
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.57 no.3
    • /
    • pp.501-506
    • /
    • 2008
  • Medical environments incorporate complex and integrated data networks to transfer vast amounts of patient information, such as images, waveforms, and other digital data. To assure interoperability of images, waveforms and patient data, health level seven(HL7) was developed as an international standard to facilitate the communication and storage of medical data. We also adopted medical waveform description format encoding rule(MFER) standard for encoding waveform biosignal such as ECG, EEG and so on. And, the study converted a broad domain of clinical data on patients, including MFER, into a HL7 message, and saved them in a clinical database in hospital. According to results obtained in the test environment, it was possible to acquire the same HL7 message and biosignal data as ones acquired during transmission. Through this study, we might conclude that the proposed system can be a promising model for electronic medical record system in u-healthcare environment.

Design of User Data Management System for Grid Service (그리드 서비스를 위한 사용자 데이터 관리 시스템 설계)

  • Oh, Young-Ju;Kim, Beob-Kyun;An, Dong-Un;Chung, Seung-Jong
    • Proceedings of the KIEE Conference
    • /
    • 2005.05a
    • /
    • pp.224-226
    • /
    • 2005
  • Grid computing enables the fundamental computing shift from a localized resource computing model to a fully-distributed virtual organization with shared resources. In the grid computing environment, grid users usually get access rights by mapping their credential to local account. The mapped total account is temporally belongs to grid user. So, data on the secondary storage, which is produced by grid operation, can increase the load of system administration or can issue grid user's privacy. In this paper, we design a data management system for grid user to cover these problems. This system implements template account mechanism and manages local grid data.

  • PDF

Version-Aware Cooperative Caching for Multi-Node Rendering

  • Cho, Kyungwoon;Bahn, Hyokyung
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.2
    • /
    • pp.30-35
    • /
    • 2022
  • Rendering is widely used for visual effects in animations and movies. Although rendering is computing-intensive, we observe that it accompanies heavy I/O because of large input data. This becomes technical hurdles for multi-node rendering performed on public cloud nodes. To reduce the overhead of data transmission in multi-node rendering, this paper analyzes the characteristics of rendering workloads, and presents the cooperative caching scheme for multi-node rendering. Our caching scheme has the function of synchronization between original data in local storage and cached data in rendering nodes, and the cached data are shared between multiple rendering nodes. We perform measurement experiments in real system environments and show that the proposed cooperative caching scheme improves the conventional caching scheme used in the network file system by 27% on average.

Study on Data Processing of the IOT Sensor Network Based on a Hadoop Cloud Platform and a TWLGA Scheduling Algorithm

  • Li, Guoyu;Yang, Kang
    • Journal of Information Processing Systems
    • /
    • v.17 no.6
    • /
    • pp.1035-1043
    • /
    • 2021
  • An Internet of Things (IOT) sensor network is an effective solution for monitoring environmental conditions. However, IOT sensor networks generate massive data such that the abilities of massive data storage, processing, and query become technical challenges. To solve the problem, a Hadoop cloud platform is proposed. Using the time and workload genetic algorithm (TWLGA), the data processing platform enables the work of one node to be shared with other nodes, which not only raises efficiency of one single node but also provides the compatibility support to reduce the possible risk of software and hardware. In this experiment, a Hadoop cluster platform with TWLGA scheduling algorithm is developed, and the performance of the platform is tested. The results show that the Hadoop cloud platform is suitable for big data processing requirements of IOT sensor networks.