• Title/Summary/Keyword: Massive Scientific Data

Search Result 18, Processing Time 0.022 seconds

Design of methodology for management of a large volume of historical archived traffic data (대용량 과거 교통 이력데이터 관리를 위한 방법론 설계)

  • Woo, Chan Il;Jeon, Se Gil
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.6 no.2
    • /
    • pp.19-27
    • /
    • 2010
  • Historical archived traffic data management system enables a long term time-series analysis and provides data necessary to acquire the constantly changing traffic conditions and to evaluate and analyze various traffic related strategies and policies. Such features are provided by maintaining highly reliable traffic data through scientific and systematic management. Now, the management systems for massive traffic data have a several problems such as, the storing and management methods of a large volume of archive data. In this paper, we describe how to storing and management for the massive traffic data and, we propose methodology for logical and physical architecture, collecting and storing, database design and implementation, process design of massive traffic data.

A Case Study on Sharing & Using of National Scientific Data (국가 과학데이터 공유·활용 서비스를 위한 사례 연구)

  • Jin, Young-Goun;Lee, Won-Goo
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.8
    • /
    • pp.9-15
    • /
    • 2018
  • Production and collection of scientific data in all areas is costly and time consuming. The level of sharing and recycling of scientific data is also very low. In order to support interdisciplinary research, massive scientific data should be systematically preserved and shared. In addition, it is essential to build an infrastructure to preserve and utilize the costly experiment and observation data. In this study, we propose a service that can collect, store, manage, share, and utilize national science data. It also suggests interfaces for various scientific data interactions with national R & D related organizations, international scientific and technological organizations and institutions. This will increase the availability of existing scientific databases that are currently being developed.

GLOVE: Distributed Shared Memory Based Parallel Visualization Tool for Massive Scientific Dataset (GLOVE: 대용량 과학 데이터를 위한 분산공유메모리 기반 병렬 가시화 도구)

  • Lee, Joong-Youn;Kim, Min Ah;Lee, Sehoon;Hur, Young Ju
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.6
    • /
    • pp.273-282
    • /
    • 2016
  • Visualization tool can be divided by three components - data I/O, visual transformation and interactive rendering. In this paper, we present requirements of three major components on visualization tools for massive scientific dataset and propose strategies to develop the tool which satisfies those requirements. In particular, we present how to utilize open source softwares to efficiently realize our goal. Furthermore, we also study the way to combine several open source softwares which are separately made to produce a single visualization software and optimize it for realtime visualization of massiv espatio-temporal scientific dataset. Finally, we propose a distributed shared memory based scientific visualization tool which is called "GLOVE". We present a performance comparison among GLOVE and well known open source visualization tools such as ParaView and VisIt.

Performance Enhancement of A Massive Scientific Data Visualization System on Virtual Reality Environment by Using Data Locality (Data Locality를 활용한 VR환경에서의 대용량 데이터 가시화 시스템의 성능 개선)

  • Lee, Se-Hoon;Kim, Min-Ah;Lee, Joong-Yeon;Hur, Young-Ju
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.11a
    • /
    • pp.284-287
    • /
    • 2012
  • GLOVE(GLObal Virtual reality visualization Environment for scientific simulation)는 컴퓨팅 자원의 성능 향상으로 데이터 양이 급속히 증가한 응용 과학과 전산 시뮬레이션 분야의 대용량 과학 데이터를 효율적으로 가시화하여 분석하기 위한 도구이다. GLOVE의 데이터 관리자인 GDM(GLOVE Data Manager)은 대용량 데이터의 분산 병렬 가시화를 위해 분산 공유 메모리를 제공하는 GA(Global Array)를 이용해 테라 바이트 단위의 데이터를 실시간으로 처리한다. 그러나 대용량 과학 데이터를 가시화 하는 과정에서 기존의 Data Locality를 고려하지 않은 데이터 접근 방식으로 인한 성능 저하를 확인했다. 본 논문은 기존 GLOVE에서 발견한 성능 저하 현상을 밝히고, 이에 대한 해결 방법을 제시한다.

Implementation of Ring Buffer based Massive VLBI Data Stream Input/Output over the Wide Area Network (광역 네트워크 상의 링 버퍼 기반 대용량 VLBI 데이터 스트림 입출력 구현)

  • Song, Min-Gyu;Kim, Hyo-Ryung;Kang, Yong-Woo;Je, Do-Heung;Wi, Seog-Oh;Lee, Sung-Mo
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.14 no.6
    • /
    • pp.1109-1120
    • /
    • 2019
  • In the field of VLBI, If the quality of the connected network between the VLBI station and the correlation center is ensured, the existing inefficiency of repeatedly storing the observation data in each station and the correlation center can be overcome. In other words, the data center can be unified with the correlation center where data analysis is performed, which can improve data processing speed and productivity. In this paper, we design a massive VLBI data system that directly transmits and stores the observation data stream obtained from the VLBI station to the correlation center via the high - speed network KREONET. Based on this system, VLBI test observations confirmed that the observation data was stored perfectly in the recording system of the correlation center without a single packet loss.

Design and Prototyping of Scientific Collaboration Platform over KREONET (KREONET 기반의 과학기술협업연구 플랫폼(RealLab) 설계 및 프로토타입 구축)

  • Kwon, Yoonjoo;Hong, Wontaek
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.4 no.9
    • /
    • pp.297-306
    • /
    • 2015
  • Cloud computing has been increasingly used in various fields due to its flexibility, scalability, cost effectiveness, etc. Recently, many scientific communities have been attempting to use cloud computing as a way to deal with difficulties in constructing and operating a research infrastructure. Especially, since they need various collaborations based on networking, such as sharing experimental data, redistributing experimental results, and so forth, cloud computing environment that supports high performance networking is required for scientific communities. To address these issues, we propose RealLab, a high performance cloud platform for collaborative research that provides virtual experimental research environment and data sharing infrastructure over KREONET/GLORIAD. Additionally, we describe some RealLab use cases for showing the swift creation of experimental environment and explain how massive experimental data can be transferred and shared among the community members.

Big Data, Business Analytics, and IoT: The Opportunities and Challenges for Business (빅데이터, 비즈니스 애널리틱스, IoT: 경영의 새로운 도전과 기회)

  • Jang, Young Jae
    • The Journal of Information Systems
    • /
    • v.24 no.4
    • /
    • pp.139-152
    • /
    • 2015
  • With the advancement of the Internet/IT technologies and the increased computation power, massive data can be collected, stored, and processed these days. The availability of large databases has brought forth a new era in which companies are hard pressed to find innovative ways to utilize immense amounts of data at their disposal. Indeed, data has opened a new age of business operations and management. There are already many cases of innovative businesses reaping success thanks to scientific decisions based on data analysis and mathematical algorithms. Big Data is a new paradigm in itself. In this article, Big Data is viewed as a new perspective rather than a new technology. This value centric definition of Big Data provides a new insight and opportunities. Moreover, the Business Analytics, which is the framework of creating tangible results in management, is introduced. Then the Internet of Things (IoT), another innovative concept of data collection and networking, is presented and how this new concept can be interpreted with Big Data in terms of the value centric perspective. The challenges and opportunities with these new concepts are also discussed.

A Pattern Matching Extended Compression Algorithm for DNA Sequences

  • Murugan., A;Punitha., K
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.8
    • /
    • pp.196-202
    • /
    • 2021
  • DNA sequencing provides fundamental data in genomics, bioinformatics, biology and many other research areas. With the emergent evolution in DNA sequencing technology, a massive amount of genomic data is produced every day, mainly DNA sequences, craving for more storage and bandwidth. Unfortunately, managing, analyzing and specifically storing these large amounts of data become a major scientific challenge for bioinformatics. Those large volumes of data also require a fast transmission, effective storage, superior functionality and provision of quick access to any record. Data storage costs have a considerable proportion of total cost in the formation and analysis of DNA sequences. In particular, there is a need of highly control of disk storage capacity of DNA sequences but the standard compression techniques unsuccessful to compress these sequences. Several specialized techniques were introduced for this purpose. Therefore, to overcome all these above challenges, lossless compression techniques have become necessary. In this paper, it is described a new DNA compression mechanism of pattern matching extended Compression algorithm that read the input sequence as segments and find the matching pattern and store it in a permanent or temporary table based on number of bases. The remaining unmatched sequence is been converted into the binary form and then it is been grouped into binary bits i.e. of seven bits and gain these bits are been converted into an ASCII form. Finally, the proposed algorithm dynamically calculates the compression ratio. Thus the results show that pattern matching extended Compression algorithm outperforms cutting-edge compressors and proves its efficiency in terms of compression ratio regardless of the file size of the data.

Visualization of Calculated Flow Fields Using Methods of Computer Graphics (컴퓨터 그래픽을 이용한 유동의 가시화)

  • Soon-Hung Han;Kyung-Ho Lee;Kyu-Ock Lee
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.29 no.4
    • /
    • pp.7-17
    • /
    • 1992
  • Developments in the emerging field of Computational Fluid Dynamics(CFD), which is made possible by the supercomputer technologies, introduce a new problem of analysing the massive amount of output produced. This problem is common to fields of computational science and engineering. Scientific visualization is to solve this problem by applying advanced technologies of computer graphics. Methods of scientific visualization are studded to visualize calculated flow fields. Different methods of scientific visualization has been surveyed, analysed and compared to select one method, iso-surface. Methods of constructing iso-surfaces from a 3-D data set have been studied. A new algorithm for constructing iso-surfaces has been developed. The algorithm can be classified as one of surface tiling methods. To develope a portable visualization system the international standard PHIGS PLUS and its implementation on X-Window system, PEX, have been selected as the development environment. A prototype of visualization system has been developed. The developed visualization system has been tried to visualize several well-known flow fields.

  • PDF

A New Approach to Marine GIS based on Uniqueness of Marine Spatial Data (해양공간 특성에 기반한 해양GIS 접근 방안 연구)

  • 박종민;서상현
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2001.10a
    • /
    • pp.183-186
    • /
    • 2001
  • For a long time, the ocean was regarded as unknown area ruled by god, consequently, not so much things were developed in scientific methods and activities. Despite a rapid development of geographic information systems(GIS) marine and ocean fields still remained in the scope of traditional tools and intuitive experiences by the late of 1990. However, land based concepts and technology models require additional customization to apply GIS effectively in marine domains, which are resulted from her dynamic, complex and seamless massive nature. This paper gives a brief review of marine spatial data characteristics and also presents strategic approaches to meet the unique marine GIS requirements.

  • PDF