• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.03 seconds

Workflow-based Bio Data Analysis System for HPC (HPC 환경을 위한 워크플로우 기반의 바이오 데이터 분석 시스템)

  • Ahn, Shinyoung;Kim, ByoungSeob;Choi, Hyun-Hwa;Jeon, Seunghyub;Bae, Seungjo;Choi, Wan
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.97-106
    • /
    • 2013
  • Since human genome project finished, the cost for human genome analysis has decreased very rapidly. This results in the sharp increase of human genome data to be analyzed. As the need for fast analysis of very large bio data such as human genome increases, non IT researchers such as biologists should be able to execute fast and effectively many kinds of bio applications, which have a variety of characteristics, under HPC environment. To accomplish this purpose, a biologist need to define a sequence of bio applications as workflow easily because generally bio applications should be combined and executed in some order. This bio workflow should be executed in the form of distributed and parallel computing by allocating computing resources efficiently under HPC cluster system. Through this kind of job, we can expect better performance and fast response time of very large bio data analysis. This paper proposes a workflow-based data analysis system specialized for bio applications. Using this system, non-IT scientists and researchers can analyze very large bio data easily under HPC environment.

MapReduce-based Localized Linear Regression for Electricity Price Forecasting (전기 가격 예측을 위한 맵리듀스 기반의 로컬 단위 선형회귀 모델)

  • Han, Jinju;Lee, Ingyu;On, Byung-Won
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • v.67 no.4
    • /
    • pp.183-190
    • /
    • 2018
  • Predicting accurate electricity prices is an important task in the electricity trading market. To address the electricity price forecasting problem, various approaches have been proposed so far and it is known that linear regression-based approaches are the best. However, the use of such linear regression-based methods is limited due to low accuracy and performance. In traditional linear regression methods, it is not practical to find a nonlinear regression model that explains the training data well. If the training data is complex (i.e., small-sized individual data and large-sized features), it is difficult to find the polynomial function with n terms as the model that fits to the training data. On the other hand, as a linear regression model approximating a nonlinear regression model is used, the accuracy of the model drops considerably because it does not accurately reflect the characteristics of the training data. To cope with this problem, we propose a new electricity price forecasting method that divides the entire dataset to multiple split datasets and find the best linear regression models, each of which is the optimal model in each dataset. Meanwhile, to improve the performance of the proposed method, we modify the proposed localized linear regression method in the map and reduce way that is a framework for parallel processing data stored in a Hadoop distributed file system. Our experimental results show that the proposed model outperforms the existing linear regression model. Specifically, the accuracy of the proposed method is improved by 45% and the performance is faster 5 times than the existing linear regression-based model.

A Study on the Digital Filter Design for Radio Astronomy Using FPGA (FPGA를 이용한 전파천문용 디지털 필터 설계에 관한 기본연구)

  • Jung, Gu-Young;Roh, Duk-Gyoo;Oh, Se-Jin;Yeom, Jae-Hwan;Kang, Yong-Woo;Lee, Chang-Hoon;Chung, Hyun0Soo;Kim, Kwang-Dong
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.9 no.1
    • /
    • pp.62-74
    • /
    • 2008
  • In this paper, we would like to propose the design of symmetric digital filter core in order to use in the radio astronomy. The function of FIR filter core would be designed by VHDL code required at the Data Acquisition System (DAS) of Korean VLBI Network (KVN) based on the FPGA chip of Vertex-4 SX55 model of Xilinx company. The designed digital filter has the symmetric structure to increase the effectiveness of system by sharing the digital filter coefficient. The SFFU(Symmetric FIR Filter Unit) use the parallel processing method to perform the data processing efficiently by using the constrained system clock. In this paper, therefore, for the effective design of SFFU, the Unified Synthesis software ISE Foundation and Core Generator which has excellent GUI environment were used to overall IP core synthesis and experiments. Through the synthesis results of digital filter core, we verified the resource usage is less than 40% such as Slice LUT and achieved the maximum operation frequency is more than 260MHz. We also confirmed the SFFU would be well operated without error according to the SFFU simulation result using the Modelsim 6.1a of Mentor Graphics Company. To verify the function of SFFU, we carried out the additional simulation experiments using the pseudo signal to the Matlab software. From the comparison experimental results of simulation and the designed digital FIR filter, we confirmed the FIR filter was well performed with filter's basic function. So we verified the effectiveness of the designed FIR digital filter with symmetric structure using FPGA and VHDL.

  • PDF

A Study on the Design of DC Parameter Test System (DC 파라메터 검사 시스템 설계에 관한 연구)

  • 신한중;김준식
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.4 no.2
    • /
    • pp.61-69
    • /
    • 2003
  • In this paper, we developed the U parameter test system which inspects the property of DC parameter for semiconductor products. The developed system is interfaced by IBM-PC. It is consisted of CPLD part, ADC (Analogue to Digital Converter), DAC (Digital to Analogue Converter), voltage/current source, variable resistor and measurement part. In the proposed system, we have designed the constant voltage source and the constant current source in a part. The CPLD part is designed by VHBL, which it generates the control and converts the serial data to parallel data. The proposed system has two test channels and it operates VFCS mode and CFVS mode. The range of test voltage is from 0[V] to 100[V], and the range of test current is from 0[mA] to 100[mA)]. The diode is tested. The test results have a good performance.

  • PDF

3-D Hetero-Integration Technologies for Multifunctional Convergence Systems

  • Lee, Kang-Wook
    • Journal of the Microelectronics and Packaging Society
    • /
    • v.22 no.2
    • /
    • pp.11-19
    • /
    • 2015
  • Since CMOS device scaling has stalled, three-dimensional (3-D) integration allows extending Moore's law to ever high density, higher functionality, higher performance, and more diversed materials and devices to be integrated with lower cost. 3-D integration has many benefits such as increased multi-functionality, increased performance, increased data bandwidth, reduced power, small form factor, reduced packaging volume, because it vertically stacks multiple materials, technologies, and functional components such as processor, memory, sensors, logic, analog, and power ICs into one stacked chip. Anticipated applications start with memory, handheld devices, and high-performance computers and especially extend to multifunctional convengence systems such as cloud networking for internet of things, exascale computing for big data server, electrical vehicle system for future automotive, radioactivity safety system, energy harvesting system and, wireless implantable medical system by flexible heterogeneous integrations involving CMOS, MEMS, sensors and photonic circuits. However, heterogeneous integration of different functional devices has many technical challenges owing to various types of size, thickness, and substrate of different functional devices, because they were fabricated by different technologies. This paper describes new 3-D heterogeneous integration technologies of chip self-assembling stacking and 3-D heterogeneous opto-electronics integration, backside TSV fabrication developed by Tohoku University for multifunctional convergence systems. The paper introduce a high speed sensing, highly parallel processing image sensor system comprising a 3-D stacked image sensor with extremely fast signal sensing and processing speed and a 3-D stacked microprocessor with a self-test and self-repair function for autonomous driving assist fabricated by 3-D heterogeneous integration technologies.

VLSI Architecture Design of Reconstruction Filter for Morphological Image Segmentation (형태학적 영상 분할을 위한 재구성 필터의 VLSI 구조 설계)

  • Lee, Sang-Yeol;Chung, Eui-Yoon;Lee, Ho-Young;Kim, Hee-Soo;Ha, Yeong-Ho
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • v.36S no.12
    • /
    • pp.41-50
    • /
    • 1999
  • In this paper, the new VLSI architecture of a reconstruction filter for morphological image segmentation is proposed. The filter, based on the $h_{max}$ operation, simplifies the interior of each region while preserving the boundary information. The proposed architecture adopts a partitioned memory structure and an efficient image scanning strategy to reduce the operations. The proposed memory partitioning scheme makes it possible that every data required for processing can be read from each memory at a time, resulting in parallel data processing. By the extended connectivity consideration, the operation is much decreased because more simplification is achieved in scanning stage. The selective raster scan strategy endows the satisfactory noise removal capability with negligible hardware complexity increase. The proposed architecture is designed using VHDL, and functional evaluation is performed by the CAD tool, Mentor. The experiment results show that the proposed architecture can simplify image profile with less than 18% operations of the conventional method.

  • PDF

A Research on Module Arrangement of Korean Spelling Corrector to Optimize Correction Rate (교정률 최적화를 위한 한국어 철자교정기의 모듈 배열)

  • Yun Keun-Soo;Kwon Hyuk-Chul
    • Journal of KIISE:Software and Applications
    • /
    • v.32 no.5
    • /
    • pp.366-377
    • /
    • 2005
  • We find a module may that takes optimal correction rate of Korean spelling corrector. If there are a lot of module numbers of spelling corrector, it is difficult to calculate optimal correction rate of spelling corrector because permutation of N-modules is N!. This Korean spelling corrector consists of 19 modules. It is impossible to arrange 19 modules actually and the correction rate is various according to input data. We found the range of correction rate using parallel processing between modules and the optimal correction rate using sequential processing of modules. Input data that are used in an experiment is 753,191 eojeol's sets that happen in newspaper publishing company during several years. About this error set, theoretical maximum correction rate of spelling corrector is $97.28\%$ (732,764/753,191). But we got the optimal correction rate $96.62\%$ (727,750/733,191). This optimal correction rate is almost near to $99.31\%$ (727,750/732,764) of the maximum correction rate.

Design of an Efficient VLSI Architecture of SADCT Based on Systolic Array (시스톨릭 어레이에 기반한 SADCT의 효율적 VLSl 구조설계)

  • Gang, Tae-Jun;Jeong, Ui-Yun;Gwon, Sun-Gyu;Ha, Yeong-Ho
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.38 no.3
    • /
    • pp.282-291
    • /
    • 2001
  • In this paper, an efficient VLSI architecture of Shape Adaptive Discrete Cosine Transform(SADCT) based on systolic array is proposed. Since transform size in SADCT is varied according to the shape of object in each block, it are dropped that both usability of processing elements(PE´s) and throughput rate in time-recursive SADCT structure. To overcome these disadvantages, it is proposed that the architecture based on a systolic way structure which doesn´t need memory. In the proposed architecture, throughput rate is improved by consecutive processing of one-dimensional SADCT without memory and PE´s in the first column are connected to that in the last one for improvement of usability of PE. And input data are put into each column of PE in parallel according to the maximum data number in each rearranged block. The proposed architecture is described by VHDL. Also, its function is evaluated by MentorTM. Even though the hardware complexity is somewhat increased, the throughput rate is improved about twofold.

  • PDF

Prefetching Policy based on File Acess Pattern and Cache Area (파일 접근 패턴과 캐쉬 영역을 고려한 선반입 기법)

  • Lim, Jae-Deok;Hwang-Bo, Jun-Hyeong;Koh, Kwang-Sik;Seo, Dae-Hwa
    • The KIPS Transactions:PartA
    • /
    • v.8A no.4
    • /
    • pp.447-454
    • /
    • 2001
  • Various caching and prefetching algorithms have been investigated to identify and effective method for improving the performance of I/O devices. A prefetching algorithm decreases the processing time of a system by reducing the number of disk accesses when an I/O is needed. This paper proposes an AMBA prefetching method that is an extended version of the OBA prefetching method. The AMBA prefetching method will prefetching blocks continuously as long as disk bandwidth is enough. In this method, though there were excessive data request rate, we would expect efficient prefetching. And in the AMBA prefetching method, to prevent the cache pollution, it limits the number of data blocks to be prefetched within the cache area. It can be implemented in a user-level File System based on a Linux Operating System. In particular, the proposed prefetching policy improves the system performance by about 30∼40% for large files that are accessed sequentially.

  • PDF

FPGA integrated IEEE 802.15.4 ZigBee wireless sensor nodes performance for industrial plant monitoring and automation

  • Ompal, Ompal;Mishra, Vishnu Mohan;Kumar, Adesh
    • Nuclear Engineering and Technology
    • /
    • v.54 no.7
    • /
    • pp.2444-2452
    • /
    • 2022
  • The field-programmable gate array (FPGA) is gaining popularity in industrial automation such as nuclear power plant instrumentation and control (I&C) systems due to the benefits of having non-existence of operating system, minimum software errors, and minimum common reason failures. Separate functions can be processed individually and in parallel on the same integrated circuit using FPGAs in comparison to the conventional microprocessor-based systems used in any plant operations. The use of FPGAs offers the potential to minimize complexity and the accompanying difficulty of securing regulatory approval, as well as provide superior protection against obsolescence. Wireless sensor networks (WSNs) are a new technology for acquiring and processing plant data wirelessly in which sensor nodes are configured for real-time signal processing, data acquisition, and monitoring. ZigBee (IEEE 802.15.4) is an open worldwide standard for minimum power, low-cost machine-to-machine (M2M), and internet of things (IoT) enabled wireless network communication. It is always a challenge to follow the specific topology when different Zigbee nodes are placed in a large network such as a plant. The research article focuses on the hardware chip design of different topological structures supported by ZigBee that can be used for monitoring and controlling the different operations of the plant and evaluates the performance in Vitex-5 FPGA hardware. The research work presents a strategy for configuring FPGA with ZigBee sensor nodes when communicating in a large area such as an industrial plant for real-time monitoring.