• Title/Summary/Keyword: Map-Reduce

Search Result 849, Processing Time 0.029 seconds

A MapReduce-Based Distributed Data Mining Approach to Next Place Prediction for Mobile Users (이동 사용자의 다음 장소 예측을 위한 맵리듀스 기반의 분산 데이터 마이닝)

  • Kim, Jong-Hwan;Lee, Seok-Jun;Kim, In-Cheol
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2014.04a
    • /
    • pp.777-780
    • /
    • 2014
  • 본 논문에서는 휴대용 기기 사용자들의 이동 궤적을 기록한 대용량의 GPS 위치 데이터 집합으로부터 각 사용자의 이동 패턴 모델을 학습해내고, 이 모델을 적용하여 각 사용자의 다음 방문 장소를 효율적으로 예측할 수 있는 맵리듀스 기반의 분산 데이터 마이닝 시스템을 소개한다. 본 시스템은 크게 사용자별 이동 패턴 모델을 학습하는 후단부와 실시간으로 다음 방문 장소를 예측하는 전단부로 구성된다. 이 중에서 후단부는 주요 장소 추출, 이동 궤적 변환, 이동 패턴 모델 학습 등 총 3개의 맵리듀스 작업 모듈들로 구성된다. 이에 반해, 본 시스템의 전단부는 이동 경로 후보군 생성, 다음 장소 예측 등 총 2개의 맵리듀스 작업 모듈들로 구성된다. 그리고 본 시스템을 구성하는 각각의 작어마다 분산처리를 극대화할 수 있도록 맵과 리듀스 함수를 설계하였다. 끝으로, 대용량의 GeoLife 벤치마크 데이터 집합을 이용하여 본 논문에서 소개한 시스템의 예측 성능을 분석하기 위한 실험을 수행하였고, 이를 통해 본 시스템의 높은 성능을 확인할 수 있었다.

Speech Query Recognition for Tamil Language Using Wavelet and Wavelet Packets

  • Iswarya, P.;Radha, V.
    • Journal of Information Processing Systems
    • /
    • v.13 no.5
    • /
    • pp.1135-1148
    • /
    • 2017
  • Speech recognition is one of the fascinating fields in the area of Computer science. Accuracy of speech recognition system may reduce due to the presence of noise present in speech signal. Therefore noise removal is an essential step in Automatic Speech Recognition (ASR) system and this paper proposes a new technique called combined thresholding for noise removal. Feature extraction is process of converting acoustic signal into most valuable set of parameters. This paper also concentrates on improving Mel Frequency Cepstral Coefficients (MFCC) features by introducing Discrete Wavelet Packet Transform (DWPT) in the place of Discrete Fourier Transformation (DFT) block to provide an efficient signal analysis. The feature vector is varied in size, for choosing the correct length of feature vector Self Organizing Map (SOM) is used. As a single classifier does not provide enough accuracy, so this research proposes an Ensemble Support Vector Machine (ESVM) classifier where the fixed length feature vector from SOM is given as input, termed as ESVM_SOM. The experimental results showed that the proposed methods provide better results than the existing methods.

Application of Hyperion Hyperspectral Remote Sensing Data for Wildfire Fuel Mapping

  • Yoon, Yeo-Sang;Kim, Yong-Seung
    • Korean Journal of Remote Sensing
    • /
    • v.23 no.1
    • /
    • pp.21-32
    • /
    • 2007
  • Fire fuel map is one of the most critical factors for planning and managing the fire hazard and risk. However, fuel mapping is extremely difficult because fuel properties vary at spatial scales, change depending on the seasonal situations and are affected by the surrounding environment. Remote sensing has potential to reduce the uncertainty in mapping fuels and offers the best approach for improving our abilities. Especially, Hyperspectral sensor have a great potential for mapping vegetation properties because of their high spectral resolution. The objective of this paper is to evaluate the potential of mapping fuel properties using Hyperion hyperspectral remote sensing data acquired in April, 2002. Fuel properties are divided into four broad categories: 1) fuel moisture, 2) fuel green live biomass, 3) fuel condition and 4) fuel types. Fuel moisture and fuel green biomass were assessed using canopy moisture, derived from the expression of liquid water in the reflectance spectrum of plants. Fuel condition was assessed using endmember fractions from spectral mixture analysis (SMA). Fuel types were classified by fuel models based on the results of SMA. Although Hyperion imagery included a lot of sensor noise and poor performance in liquid water band, the overall results showed that Hyperion imagery have good potential for wildfire fuel mapping.

A Virtual Machine Remapping Scheme for Reducing Relocation Time on a Cloud Cluster (클라우드 클러스터에서 가상머신 재배치시간을 단축하기 위한 재매핑 기법)

  • Kim, Chang-Hyeon;Kim, Jun-Sang;Jeon, Chang-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.19 no.11
    • /
    • pp.1-7
    • /
    • 2014
  • In this paper, we propose a virtual machine(VM) remapping scheme that reduces VM relocation time on a cloud cluster. The proposed scheme finds VMs that should be migrated in sequence from a given VM map, and exchanges destinations of some VMs among them to reduce the VM relocation time. The VMs, the destinations of which will be exchanged, are chosen based on the amount of physical machine's available resources and migration completion time. The exchange of destinations is repeated until the VM relocation time cannot be shortened any further. Through a simulation, we show that the proposed scheme reduces VM relocation time by 42.7% in maximum.

A Study on Performance Improvement of Distributed Computing Framework using GPU (GPU를 활용한 분산 컴퓨팅 프레임워크 성능 개선 연구)

  • Song, Ju-young;Kong, Yong-joon;Shim, Tak-kil;Shin, Eui-seob;Seong, Kee-kin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.04a
    • /
    • pp.499-502
    • /
    • 2012
  • 빅 데이터 분석의 시대가 도래하면서 대용량 데이터의 특성과 계산 집약적 연산의 특성을 동시에 가지는 문제 해결에 대한 요구가 늘어나고 있다. 대용량 데이터 처리의 경우 각종 분산 파일 시스템과 분산/병렬 컴퓨팅 기술들이 이미 많이 사용되고 있으며, 계산 집약적 연산 처리의 경우에도 GPGPU 활용 기술의 발달로 보편화되는 추세에 있다. 하지만 대용량 데이터와 계산 집약적 연산 이 두 가지 특성을 모두 가지는 문제를 처리하기 위해서는 많은 제약 사항들을 해결해야 하는데, 본 논문에서는 이에 대한 대안으로 분산 컴퓨팅 프레임워크인 Hadoop MapReduce와 Nvidia의 GPU 병렬 컴퓨팅 아키텍처인 CUDA 흘 연동하는 방안을 제시하고, 이를 밀집행렬(dense matrix) 연산에 적용했을 때 얻을 수 있는 성능 개선 효과에 대해 소개하고자 한다.

Adaptive Wireless Network Coding for Infrastructure Wireless Mesh Networks

  • Carrillo, Ernesto;Ramos, Victor
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.7
    • /
    • pp.3470-3493
    • /
    • 2019
  • IEEE 802.11s-based infrastructure Wireless Mesh Networks (iWMNs) are envisaged as a promising solution to provide ubiquitous wireless Internet access. The limited network capacity is a problem mainly caused by the medium contention between mesh users and the mesh access points (MAPs), which gets worst when the mesh clients employ the Transmission Control Protocol (TCP). To mitigate this problem, we use wireless network coding (WNC) in the MAPs. The aim of this proposal is to take advantage of the network topology around the MAPs, to alleviate the contention and maximize the use of the network capacity. We evaluate WNC when is used in MAPs. We model the formation of coding opportunities and, using computer simulations, we evaluate the formation of such coding opportunities. The results show that as the users density grows, the coding opportunities increase up to 70%; however, at the same time, the coding delay increments significantly. In order to reduce such delay, we propose to adaptively adjust the time that a packet can wait to catch a coding opportunity in an MAP. We assess the performance of moving-average estimation methods to forecast this adaptive sojourn time. We show that using moving-average estimation methods can significantly decrease the coding delay since they consider the traffic density conditions.

An Efficient Parallel Construction Scheme of An R-Tree using Hadoop (Hadoop을 이용한 R-트리의 효율적인 병렬 구축 기법)

  • Cong, Viet-Ngu Huynh;Kim, Jongmin;Kwon, Oh-Heum;Song, Ha-Joo
    • Journal of Korea Multimedia Society
    • /
    • v.22 no.2
    • /
    • pp.231-241
    • /
    • 2019
  • Bulk-loading an R-tree can be a good approach to build an efficient one. However, it takes a lot of time to bulk-load an R-tree for huge amount of data. In this paper, we propose a parallel R-tree construction scheme based on a Hadoop framework. The proposed scheme divides the data set into a number of partitions for which local R-trees are built in parallel via Map-Reduce operations. Then the local R-trees are merged into an global R-tree that covers the whole data set. While generating the partitions, it considers the spatial distribution of the data into account so that each partition has nearly equal amounts of data. Therefore, the proposed scheme gives an efficient index structure while reducing the construction time. Experimental tests show that the proposed scheme builds an R-tree more efficiently than the existing approaches.

Vascular Morphometric Changes During Tumor Growth and Chemotherapy in a Murine Mammary Tumor Model Using OCT Angiography: a Preliminary Study

  • Kim, Hoonsup;Eom, Tae Joong;Kim, Jae Gwan
    • Current Optics and Photonics
    • /
    • v.3 no.1
    • /
    • pp.54-65
    • /
    • 2019
  • To develop a biomarker predicting tumor treatment efficacy is helpful to reduce time, medical expenditure, and efforts in oncology therapy. In clinics, microvessel density using immunohistochemistry has been proposed as an indicator that correlates with both tumor size and metastasis of cancer. In the preclinical study, we hypothesized that vascular morphometrics using optical coherence tomography angiography (OCTA) could be potential indicators to estimate the treatment efficacy of breast cancer. To verify this hypothesis, a 13762-MAT-B-III rat breast tumor was grown in a dorsal skinfold window chamber which was applied to a nude mouse, and the change in vascular morphology was longitudinally monitored during tumor growth and metronomic cyclophosphamide treatment. Based on the daily OCTA maximum intensity projection map, multiple vessel parameters (vessel skeleton density, vessel diameter index, fractal dimension, and lacunarity) were compared with the tumor size in no tumor, treated tumor, and untreated tumor cases. Although each case has only one animal, we found that the vessel skeleton density (VSD), vessel diameter index and fractal dimension (FD) tended to be positively correlated with tumor size while lacunarity showed a partially negative correlation. Moreover, we observed that the changes in the VSD and FD are prior to the morphological change of the tumor. This feasibility study would be helpful in evaluating the tumor vascular response to treatment in preclinical settings.

Implementation of a Raspberry-Pi-Sensor Network (라즈베리파이 센서 네트워크 구현)

  • Moon, Sangook
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.10a
    • /
    • pp.915-916
    • /
    • 2014
  • With the upcoming era of internet of things, the study of sensor network has been paid attention. Raspberry pi is a tiny versatile computer system which is able to act as a sensor node in hadoop cluster network. In this paper, we deployed 5 Raspberry pi's to construct an experimental testbed of hadoop sensor network with 5-node map-reduce hadoop software framework. We compared and analyzed the network architecture in terms of efficiency, resource management, and throughput using various parameters. We used a learning machine with support vector machine as test workload. In our experiments, Raspberry pi fulfilled the role of distributed computing sensor node in the sensor network.

  • PDF

Distributed Support Vector Machines for Localization on a Sensor Newtork (센서 네트워크에서 위치 측정을 위한 분산 지지 벡터 머신)

  • Moon, Sangook
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.10a
    • /
    • pp.944-946
    • /
    • 2014
  • Localization of a sensor network node using machine learning has been recently studied. It is easy for Support vector machines algorithm to implement in high level language enabling parallelism. In this paper, we realized Support vector machine using python language and built a sensor network cluster with 5 Pi's. We also established a Hadoop software framework to employ MapReduce mechanism. We modified the existing Support vector machine algorithm to fit into the distributed hadoop architecture system for localization of a sensor node. In our experiment, we implemented the test sensor network with a variety of parameters and examined based on proficiency, resource evaluation, and processing time.

  • PDF