• Title/Summary/Keyword: Parallel data processing

Search Result 751, Processing Time 0.029 seconds

Selective Decoding Schemes and Wireless MAC Operating in MIMO Ad Hoc Networks

  • Suleesathira, Raungrong;Aksiripipatkul, Jansilp
    • Journal of Communications and Networks
    • /
    • v.13 no.5
    • /
    • pp.421-427
    • /
    • 2011
  • Problems encountered in IEEE 802.11 medium access control (MAC) design are interferences from neighboring or hidden nodes and collision from simultaneous transmissions within the same contention floors. This paper presents the selective decoding schemes in MAC protocol for multiple input multiple output ad-hoc networks. It is able to mitigate interferences by using a developed minimum mean-squared error technique. This interference mitigation combined with the maximum likelihood decoding schemes for the Alamouti coding enables the receiver to decode and differentiate the desired data streams from co-channel data streams. As a result, it allows a pair of simultaneous transmissions to the same or different nodes which yields the network utilization increase. Moreover, the presented three decoding schemes and time line operations are optimally selected corresponding to the transmission demand of neighboring nodes to avoid collision. The selection is determined by the number of request to send (RTS) packets and the type of clear to send packets. Both theoretical channel capacity and simulation results show that the proposed selective decoding scheme MAC protocol outperforms the mitigation interference using multiple antennas and the parallel RTS processing protocols for the cases of (1) single data stream and (2) two independent data streams which are simultaneously transmitted by two independent transmitters.

Evaluation of Alignment Methods for Genomic Analysis in HPC Environment (HPC 환경의 대용량 유전체 분석을 위한 염기서열정렬 성능평가)

  • Lim, Myungeun;Jung, Ho-Youl;Kim, Minho;Choi, Jae-Hun;Park, Soojun;Choi, Wan;Lee, Kyu-Chul
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.107-112
    • /
    • 2013
  • With the progress of NGS technologies, large genome data have been exploded recently. To analyze such data effectively, the assistance of HPC technique is necessary. In this paper, we organized a genome analysis pipeline to call SNP from NGS data. To organize the pipeline efficiently under HPC environment, we analyzed the CPU utilization pattern of each pipeline steps. We found that sequence alignment is computing centric and suitable for parallelization. We also analyzed the performance of parallel open source alignment tools and found that alignment method utilizing many-core processor can improve the performance of genome analysis pipeline.

Deletion-Based Sentence Compression Using Sentence Scoring Reflecting Linguistic Information (언어 정보가 반영된 문장 점수를 활용하는 삭제 기반 문장 압축)

  • Lee, Jun-Beom;Kim, So-Eon;Park, Seong-Bae
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.3
    • /
    • pp.125-132
    • /
    • 2022
  • Sentence compression is a natural language processing task that generates concise sentences that preserves the important meaning of the original sentence. For grammatically appropriate sentence compression, early studies utilized human-defined linguistic rules. Furthermore, while the sequence-to-sequence models perform well on various natural language processing tasks, such as machine translation, there have been studies that utilize it for sentence compression. However, for the linguistic rule-based studies, all rules have to be defined by human, and for the sequence-to-sequence model based studies require a large amount of parallel data for model training. In order to address these challenges, Deleter, a sentence compression model that leverages a pre-trained language model BERT, is proposed. Because the Deleter utilizes perplexity based score computed over BERT to compress sentences, any linguistic rules and parallel dataset is not required for sentence compression. However, because Deleter compresses sentences only considering perplexity, it does not compress sentences by reflecting the linguistic information of the words in the sentences. Furthermore, since the dataset used for pre-learning BERT are far from compressed sentences, there is a problem that this can lad to incorrect sentence compression. In order to address these problems, this paper proposes a method to quantify the importance of linguistic information and reflect it in perplexity-based sentence scoring. Furthermore, by fine-tuning BERT with a corpus of news articles that often contain proper nouns and often omit the unnecessary modifiers, we allow BERT to measure the perplexity appropriate for sentence compression. The evaluations on the English and Korean dataset confirm that the sentence compression performance of sentence-scoring based models can be improved by utilizing the proposed method.

Recommendation System Using Big Data Processing Technique (빅 데이터 처리 기법을 적용한 추천 시스템에 관한 연구)

  • Yun, So-Young;Youn, Sung-Dae
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.6
    • /
    • pp.1183-1190
    • /
    • 2017
  • With the development of network and IT technology, people are searching and purchasing items they want, not bounded by places. Therefore, there are various studies on how to solve the scalability problem due to the rapidly increasing data in the recommendation system. In this paper, we propose an item-based collaborative filtering method using Tag weight and a recommendation technique using MapReduce method, which is a distributed parallel processing method. In order to improve speed and efficiency, the proposed method classifies items into categories in the preprocessing and groups according to the number of nodes. In each distributed node, data is processed by going through Map-Reduce step 4 times. In order to recommend better items to users, item tag weight is used in the similarity calculation. The experiment result indicated that the proposed method has been more enhanced the appropriacy compared to item-based method, and run efficiently on the large amounts of data.

Multiaccess Memory System supporting Local Buffer Memory System to Processing Elements (처리기에 지역 버퍼 메모리 시스템을 지원하는 다중접근기억장치)

  • Lee, Hyung
    • The Journal of the Korea Contents Association
    • /
    • v.12 no.1
    • /
    • pp.30-37
    • /
    • 2012
  • A memory system with the linear skewing scheme has been regarded as one of suitable memory systems for a single instruction, multiple data (SIMD) architecture. The memory system supports simultaneous access n data to m memory modules within various access types with a constant interval in an arbitrary position in two dimensional data array of $M{\times}N$. Although $m{\times}cells$ memory cells are physically required to support logical two dimensional $M{\times}N$ array of data by means of the memory system, at least (m-n)${\times}cells$ memory cells remain in disuse, where cells is (M-1)/q+(N-1)/$p{\times}{\lceil}M/q{\rceil}+1$. On keeping functionalities the memory system supports, $(n{\times}t){\times}N/p$ out of a number of unused memory cells, where t>0, being used as local buffer memories for n processing elements is proposed in this paper.

Ontology and Sequential Rule Based Streaming Media Event Recognition (온톨로지 및 순서 규칙 기반 대용량 스트리밍 미디어 이벤트 인지)

  • Soh, Chi-Seung;Park, Hyun-Kyu;Park, Young-Tack
    • Journal of KIISE
    • /
    • v.43 no.4
    • /
    • pp.470-479
    • /
    • 2016
  • As the number of various types of media data such as UCC (User Created Contents) increases, research is actively being carried out in many different fields so as to provide meaningful media services. Amidst these studies, a semantic web-based media classification approach has been proposed; however, it encounters some limitations in video classification because of its underlying ontology derived from meta-information such as video tag and title. In this paper, we define recognized objects in a video and activity that is composed of video objects in a shot, and introduce a reasoning approach based on description logic. We define sequential rules for a sequence of shots in a video and describe how to classify it. For processing the large amount of increasing media data, we utilize Spark streaming, and a distributed in-memory big data processing framework, and describe how to classify media data in parallel. To evaluate the efficiency of the proposed approach, we conducted an experiment using a large amount of media ontology extracted from Youtube videos.

Disk Load Balancing Scheme for High Speed Playback of Continuous Media in VOD Server (VOD서버에서 연속 매체의 고속 재생을 위한 디스크 부하 균형 정책)

  • Lee, Seung-Yong;Lee, Ho-Seok;Hong, Seong-Su
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.5
    • /
    • pp.1172-1181
    • /
    • 1997
  • A militimedia data is a data mixed of formatted data like an audio and video. Multimedia data has characteristics that it need large amount of storage,wide network bandwith andreal time responsibolity. Because of these characteristocs, the VOD server and continous media storage server have a disk stripe structure or disk stripe sructure or disk array structure(RAID).In the parallel disk access system,high-speed play-back of continous media using segment interleavung may not ensure Qos pf other cioents because of the concentrated load within some disks. The load concentration of disks is related to both the number of disks in the system and playback rate of contimous media.In this paper. we describe that high-speed playback scheme,which is independent of the number of disk and plyback rate can be achieved by technique of changing the in-teval of access to segnent location.We show the experimental result of this technique in this pater.

  • PDF

Efficient Processing of Multiple Group-by Queries in MapReduce for Big Data Analysis (맵리듀스에서 빅데이터 분석을 위한 다중 Group-by 질의의 효율적인 처리 기법)

  • Park, Eunju;Park, Sojeong;Oh, Sohyun;Choi, Hyejin;Lee, Ki Yong;Shim, Junho
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.5
    • /
    • pp.387-392
    • /
    • 2015
  • MapReduce is a framework used to process large data sets in parallel on a large cluster. A group-by query is a query that partitions the input data into groups based on the values of the specified attributes, and then evaluates the value of the specified aggregate function for each group. In this paper, we propose an efficient method for processing multiple group-by queries using MapReduce. Instead of computing each group-by query independently, the proposed method computes multiple group-by queries in stages with one or more MapReduce jobs in order to reduce the total execution cost. We compared the performance of this method with the performance of a less sophisticated method that computes each group-by query independently. This comparison showed that the proposed method offers better performance in terms of execution time.

A Study of Integral Image Hardware Design for Memory Size Efficiency (메모리 크기에 효율적인 적분영상 하드웨어 설계 연구)

  • Lee, Su-Hyun;Jeong, Yong-Jin
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.51 no.9
    • /
    • pp.75-81
    • /
    • 2014
  • The integral image is the sum of input image pixel values. It is mainly used to speed up processing of a box filter operation, such as Haar-like features. However, large memory for integral image data can be an obstacle on an embedded hardware environment with limited memory resources. Therefore, an efficient method to store the integral image is necessary. In this paper, we propose a memory size reduction hardware design for integral image. The hardware design is used two methods. It is the new integral image memory and modulo calculation for reducing integral image data. The new integral image memory has additional calculation overhead, but it is not obstacle in hardware environment that parallel processing is possible. In the Xilinx Virtex5-LX330T targeted experimental result, integral image memory can be reduced by 50% on a $640{\times}480$ 8-bit gray-scale input image.

Development of rotational pulse-echo ultrasonic propagation imaging system capable of inspecting cylindrical specimens

  • Ahmed, Hasan;Lee, Young-Jun;Lee, Jung-Ryul
    • Smart Structures and Systems
    • /
    • v.26 no.5
    • /
    • pp.657-666
    • /
    • 2020
  • A rotational pulse-echo ultrasonic propagation imager that can inspect cylindrical specimens for material nondestructive evaluations is proposed herein. In this system, a laser-generated ultrasonic bulk wave is used for inspection, which enables a clear visualization of subsurface defects with a precise reproduction of the damage shape and size. The ultrasonic waves are generated by a Q-switched laser that impinges on the outer surface of the specimen walls. The generated waves travel through the walls and their echo is detected by a Laser Doppler Vibrometer (LDV) at the same point. To obtain the optimal Signal-to-Noise Ratio (SNR) of the measured signal, the LDV requires the sensed surface to be at a right angle to the laser beam and at a predefined constant standoff distance from the laser head. For flat specimens, these constraints can be easily satisfied by performing a raster scan using a dual-axis linear stage. However, this arrangement cannot be used for cylindrical specimens owing to their curved nature. To inspect the cylindrical specimens, a circular scan technology is newly proposed for pulse-echo laser ultrasound. A rotational stage is coupled with a single-axis linear stage to inspect the desired area of the specimen. This system arrangement ensures that the standoff distance and beam incidence angle are maintained while the cylindrical specimen is being inspected. This enables the inspection of a curved specimen while maintaining the optimal SNR. The measurement result is displayed in parallel with the on-going inspection. The inspection data used in scanning are mapped from rotational coordinates to linear coordinates for visualization and post-processing of results. A graphical user interface software is implemented in C++ using a QT framework and controls all the individual blocks of the system and implements the necessary image processing, scan calculations, data acquisition, signal processing and result visualization.