• Title/Summary/Keyword: Large sized file

Search Result 22, Processing Time 0.022 seconds

Performance Analysis of Lustre File System using High Performance Storage Devices (고성능 스토리지를 이용한 Lustre 파일 시스템의 성능 분석 연구)

  • Lee, Jaehwan;Koo, Donghun;Park, Kyungmin;Kim, Jiksoo;Hwang, Soonwook
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.4
    • /
    • pp.163-169
    • /
    • 2016
  • Lustre is a scalable, distributed file system, which is popular in the field of high-performance computing. Recently, the advent of SSD has enabled high-performance storage hardware, but software development requires further improvement. In this paper, we analyzed performance of the Lustre system using SSD via extensive experimentation. We compared performance of Lustre on SSDs and HDDs in terms of file read/write throughputs and metadata access latencies. Our experimental results showed that 1) SSDs improve metadata access performance due to fast random read/write access of SSD characteristics, and 2) SSD are benefited to a greater extent under multiple threads and large numbers of small sized files.

Development of Slicing Algorithm for Rapid Prototyping Building about Big Sized Objects (대형물의 RP 작업을 위한 절단 알고리즘의 개발)

  • 채희창;강희용;유상훈;이승현;문홍렬
    • Transactions of the Korean Society of Machine Tool Engineers
    • /
    • v.11 no.2
    • /
    • pp.30-35
    • /
    • 2002
  • Nowadays, as the development period of new products becomes shorter and consumer's requirement is more various, the importance of Rapid Prototyping Technology has been rapidly increased. Rapid Prototyping makes prototypes or frictional parts directly using the 3D CAD data. But RP machines can make prototypes in limit size. For making large size prototype, we slice solid which is made of STL file, and then glue sliced solid. And if contact area of part is small, union solid will be easily destroyed for going down of adhesion. So we need to expand contact area, 1 suggest making a section into stair shape. This paper is concerned with slicing solid on STL file and improving on adhesion.

Design of High-capacity NAND Flash File System supporting Sensor Data Collection (센서 데이터 수집을 위한 대용량 NAND 플래시 파일 시스템의 설계)

  • Han, Kyoung-Hoon;Lee, Ki-Hyeok;Han, Hyung-Jin;Han, Ji-Yean;Sohn, Ki-Rack
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.7
    • /
    • pp.515-519
    • /
    • 2009
  • As the application fields of sensor nodes are getting diverse these days, it is required to have a way of collecting various data that is suitable for these application fields. In the case that the real-time surveillance over the data is unnecessary, present data collecting methods, which collect and transfer the data directly, can cause a waste of energy and data loss, A new method that store the collected data in a local storage and acquire them by query later on is required for nonreal-time applications. NAND flash has energy efficiency and large capacity so it is suitable for sensor nodes, Sensor nodes support 4-10 KBytes small sized memory and it is hard to build an effective file system since NAND Flash doesn't support overwriting NAND flash. This paper discusses an implementation of NAND Flash file system in sensor node environments. The file system makes long-term data collecting possible by reducing transmission cost. It is expected that this file system will play a central role in sensor network environments as it can be applied to various fields which call for long term data collecting.

Access efficiency of small sized files in Big Data using various Techniques on Hadoop Distributed File System platform

  • Alange, Neeta;Mathur, Anjali
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.7
    • /
    • pp.359-364
    • /
    • 2021
  • In recent years Hadoop usage has been increasing day by day. The need of development of the technology and its specified outcomes are eagerly waiting across globe to adopt speedy access of data. Need of computers and its dependency is increasing day by day. Big data is exponentially growing as the entire world is working in online mode. Large amount of data has been produced which is very difficult to handle and process within a short time. In present situation industries are widely using the Hadoop framework to store, process and produce at the specified time with huge amount of data that has been put on the server. Processing of this huge amount of data having small files & its storage optimization is a big problem. HDFS, Sequence files, HAR, NHAR various techniques have been already proposed. In this paper we have discussed about various existing techniques which are developed for accessing and storing small files efficiently. Out of the various techniques we have specifically tried to implement the HDFS- HAR, NHAR techniques.

An Attribute Replicating Vertical File Partition Method by Genetic Algorithm (유전알고리듬을 이용한 속성의 중복 허용 파일 수직분할 방법)

  • 김재련;유종찬
    • The Journal of Information Technology and Database
    • /
    • v.6 no.2
    • /
    • pp.71-86
    • /
    • 1999
  • The performance of relational database is measured by the number of disk accesses necessary to transfer data from disk to main memory. The paper proposes to vertically partition relations into fragments and to allow attribute replication to reduce the number of disk accesses. To reduce the computational time, heuristic search method using genetic algorithm is used. Genetic algorithm used employs a rank-based-sharing fitness function and elitism. Desirable parameters of genetic algorithm are obtained through experiments and used to find the solutions. Solutions of attribute replication and attribute non-replication problems are compared. Optimal solutions obtained by branch and bound method and by heuristic solutions(genetic algorithm) are also discussed. The solution method proposed is able to solve large-sized problems within acceptable time limit and shows solutions near the optimal value.

  • PDF

A Fast Editing/Writing Technique of Large-sized Multimedia Files based on the Ext4 File System (Ext4 파일 시스템 기반 대용량 멀티미디어 파일 고속 편집 저장 기법)

  • Jung, Seung Wan;Ko, Seok Young;Seo, Dae Wha
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2009.11a
    • /
    • pp.193-194
    • /
    • 2009
  • 멀티미디어 장치의 대중화로 인해 사용자는 고화질의 영상과 다양한 멀티미디어 응용 서비스에 대한 요구가 증가하고 있다. 고화질의 미디어 컨텐츠는 대용량 파일의 형태로 저장된다. 대용량 파일의 필요한 부분만을 편집하여 재 저장하기 위해서는 새로운 기법이 요구된다. 현재의 리눅스 시스템에서는 편집된 파일의 내용을 새롭게 저장하는 방법을 사용하는데 미디어 컨텐츠와 같은 대용량 파일의 경우 많은 디스크 대역폭과 시간을 요구한다. 본 논문에서는 리눅스 Ext4 파일 시스템에서 범위 지정을 통한 고속 편집 저장 방법을 제안한다. 본 기법은 실제 데이터의 디스크 I/O 없이 메타데이터 편집만을 통해 대용량 멀티미디어 파일 편집 저장을 가능하게 한다.

A CASE REPORT OF THE EXTERNAL NOSE RECONSTRUCTION USING FOREHEAD FLAP AND AURICULAR COMPOSITE GRAFTS (전두부피판과 이개복합조직이식술을 이용한 외비의 재건 치험례)

  • Park, Bong-Wook;Byun, June-Ho
    • Journal of the Korean Association of Oral and Maxillofacial Surgeons
    • /
    • v.31 no.4
    • /
    • pp.350-355
    • /
    • 2005
  • There are various surgical methods for reconstruction of the nasal defect. Among them, there is some difference in the choosing the proper reconstruction method according to defect size and position. When the defect involved the tip, the columella, and the alar, the local flaps may be preferred, because they can provide sufficient amount of tissue. However, the composite grafts from the ear have been effectively used in reconstructions of smaller sized defects of the columella and ala. We excised total external nose because of squamous cell carcinoma on the nasal tip, columella, and nasal septum. We reconstructed the nasal tip, both alae, and columella with forehead flap. After division of the regional flap, we found partial necrosis of the columella and narrowness of the nostril. So, we used chondrocutaneous auricular composite grafts for reconstruction of the columella and both nostrils. We used the file-folder designed auricular composite graft for reconstruction of columella and the wedge shaped ear helical composite grafts for widening of nostrils. 6 months later, there were no significant problems, except some mismatched dark color in the grafted alar tissues. Here, we report a successful reconstruction of large nasal defect using combined two different reconstructive methods.

MapReduce-based Localized Linear Regression for Electricity Price Forecasting (전기 가격 예측을 위한 맵리듀스 기반의 로컬 단위 선형회귀 모델)

  • Han, Jinju;Lee, Ingyu;On, Byung-Won
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • v.67 no.4
    • /
    • pp.183-190
    • /
    • 2018
  • Predicting accurate electricity prices is an important task in the electricity trading market. To address the electricity price forecasting problem, various approaches have been proposed so far and it is known that linear regression-based approaches are the best. However, the use of such linear regression-based methods is limited due to low accuracy and performance. In traditional linear regression methods, it is not practical to find a nonlinear regression model that explains the training data well. If the training data is complex (i.e., small-sized individual data and large-sized features), it is difficult to find the polynomial function with n terms as the model that fits to the training data. On the other hand, as a linear regression model approximating a nonlinear regression model is used, the accuracy of the model drops considerably because it does not accurately reflect the characteristics of the training data. To cope with this problem, we propose a new electricity price forecasting method that divides the entire dataset to multiple split datasets and find the best linear regression models, each of which is the optimal model in each dataset. Meanwhile, to improve the performance of the proposed method, we modify the proposed localized linear regression method in the map and reduce way that is a framework for parallel processing data stored in a Hadoop distributed file system. Our experimental results show that the proposed model outperforms the existing linear regression model. Specifically, the accuracy of the proposed method is improved by 45% and the performance is faster 5 times than the existing linear regression-based model.

Web based 3-D Medical Image Visualization System on the PC (웹 기반 3차원 의료모델 시각화 시스템)

  • Kim, Nam-Kug;Lee, Dong-Hyuk;Kim, Jong-Hyo;Kang, Heung-Sik;Min, Byung-Goo;Kim, Young-Ho
    • Proceedings of the KOSOMBE Conference
    • /
    • v.1997 no.11
    • /
    • pp.201-205
    • /
    • 1997
  • With the recent advance of Web and its associated technologies, information sharing on distribute computing environments has gained a great amount of attention from many researchers in many application areas, such as medicine, engineering, and business. One basic requirement of distributed medical consultation systems is that geographically dispersed, disparate participants are allowed to exchange information readily with each other. Such software also needs to be supported on a broad range of computer platforms to increase the software's accessibility. In this paper, the development of world-wide-web based medical consultation system or radiology imaging is addressed to provide the platform independence and great accessibility. The system supports sharing of 3-dimensional objects. We use VRML (Virtual Reality Modeling Language), which is the de-facto standard in 3-D modeling on the Web. 3-D objects are reconstructed from CT or MRI volume data using a VRML format, which can be viewed and manipulated easily in Web-browsers with a VRML plug-in. A Marching cubes method is used in the transformation of scanned volume data set to polygonal surfaces of VRML. A decimation algorithm is adopted to reduce the number of meshes in the resulting VRML file. 3-D volume data are often very large-sized, and hence loading the data on PC level computers requires a significant reduction of the size of the data, while minimizing the loss of the original shape information. This is also important to decrease network delays. A prototype system has been implemented (http://netopia.snu.ac.kr/-cyber/). and several sessions of experiments are carried out.

  • PDF

Performance Tests of 3D Data Models for Laser Radar Simulation (레이저레이더 시뮬레이션을 위한 3차원 데이터 모델의 성능 테스트)

  • Kim, Geun-Han;Kim, Hye-Young;Jun, Chul-Min
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.17 no.3
    • /
    • pp.97-107
    • /
    • 2009
  • Experiments using real guided weapons for the development of the LADAR(Laser radar) are not practical. Therefore, we need computing environment that can simulate the 3D detections by LADAR. Such simulations require dealing with large sized data representing buildings and terrain over large area. And they also need the information of 3D target objects, for example, material and echo rate of building walls. However, currently used 3D models are mostly focused on visualization maintained as file-based formats and do not contain such semantic information. In this study, as a solution to these problems, a method to use a spatial DBMS and a 3D model suitable for LADAR simulation is suggested. The 3D models found in previous studies are developed to serve different purposes, thus, it is not easy to choose one among them which is optimized for LADAR simulation. In this study, 4 representative 3D models are first defined, each of which are tested for different performance scenarios. As a result, one model, "Body-Face", is selected as being the most suitable model for the simulation. Using this model, a test simulation is carried out.

  • PDF