• Title/Summary/Keyword: Map-Reduce

Search Result 852, Processing Time 0.03 seconds

Distributed Processing Environment for Outlier Removal to Analyze Big Data (대용량 데이터 분석을 위한 이상치 제거용 분산처리 환경)

  • Hong, Yejin;Na, Eunhee;Jung, Yonghwan;Kim, Yangwoo
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2016.07a
    • /
    • pp.73-74
    • /
    • 2016
  • IoT 데이터는 비정형 데이터로 가공되고 분석하였을 때 비로소 가치를 갖기에 전 세계적으로 빅데이터 기술에 관심이 집중되고 있다. IoT 데이터 중 많은 부분을 차치하는 센서 데이터는 수집이 용이하고 활용범위가 넓기 때문에 여러 분야에서 사용되고 있다. 하지만 센서가 정상적으로 작동하지 못한 경우에는 실제와는 다른 값인 이상치를 포함하여 왜곡된 결과가 도출되어 활용할 수 없는 경우가 생긴다. 따라서 본 논문에서는 정확한 결과를 도출하기 위하여 수집된 원자료의 데이터를 분석하기 전에 이상치 탐지 및 제거를 하고자 한다. 또한 점점 늘어나고 있는 대용량 데이터를 신속하게 처리하기 위하여 메모리 접근방식인 스파크를 사용한 분산처리환경에서 이상치 탐지 및 제거하는 것을 제안한다. 맵리듀스 기반의 이상치 탐지 및 제거는 총 4단계로 나누어 구현하였으며 제안한 기법의 성능 평가를 위해 총 3가지 환경에서 비교하여 실험하였다. 실험을 통해 데이터의 용량이 커질수록 분산처리환경에서 스파크를 사용하여 처리하는 방식이 가장 빠를 것 이라는 결과를 얻었다.

  • PDF

A Study on the Auto-MTPT Algorithm to Make the Speed-based Current-map of IPMSM for Traction of Inwheel (인휠 구동용 IPMSM의 속도 기반 전류맵 작성을 위한 Auto-MTPT 알고리즘)

  • Park, Gui-Yeol;Park, Jung-Woo;Hwang, Yo-Han;Shin, Duck-Woong;Moon, Chae-Joo
    • The Transactions of the Korean Institute of Power Electronics
    • /
    • v.21 no.5
    • /
    • pp.411-417
    • /
    • 2016
  • Theoretical IPMSM control technique is complicated, and reliability is low because of the changing parameters. Further, in case of general look-up table designing method which obtains torque characteristics (according to current and speed) or torque characteristics (according to magnetic flux through the entire control region), obtaining a precise result can be difficult and has the disadvantage taking too much time to establish a current look-up table. In this paper, the new auto maximum torque point tracking (MTPT) algorithm that automatically finds the optimum stator d - q axis electric current reference through the entire speed region is devised; consequently, it could establish a 3D look-up table with torque characteristics according to current and speed. In case of using the devised auto MTPT algorithm, the result value detailed was obtained in comparison with the generalized look-up design technique, and checked to reduce the current look-up table establishment time.

3D Navigation Real Time RSSI-based Indoor Tracking Application

  • Lee, Boon-Giin;Lee, Young-Sook;Chung, Wan-Young
    • Journal of Ubiquitous Convergence Technology
    • /
    • v.2 no.2
    • /
    • pp.67-77
    • /
    • 2008
  • Representation of various types of information in an interactive virtual reality environment on mobile devices had been an attractive and valuable research in this new era. Our main focus is presenting spatial indoor location sensing information in 3D perception in mind to replace the traditional 2D floor map using handheld PDA. Designation of 3D virtual reality by Virtual Reality Modeling Language (VRML) demonstrates its powerful ability in providing lots of useful positioning information for PDA user in real-time situation. Furthermore, by interpolating portal culling algorithm would reduce the 3D graphics rendering time on low power processing PDA significantly. By fully utilizing the CC2420 chipbased sensor nodes, wireless sensor network was established to locate user position based on Received Signal Strength Indication (RSSI) signals. Implementation of RSSI-based indoor tracking method is low-cost solution. However, due to signal diffraction, shadowing and multipath fading, high accuracy of sensing information is unable to obtain even though with sophisticated indoor estimation methods. Therefore, low complexity and flexible accuracy refinement algorithm was proposed to obtain high precision indoor sensing information. User indoor position is updated synchronously in virtual reality to real physical world. Moreover, assignment of magnetic compass could provide dynamic orientation information of user current viewpoint in real-time.

  • PDF

Next-Generation Sequencing and Epigenomics Research: A Hammer in Search of Nails

  • Sarda, Shrutii;Hannenhalli, Sridhar
    • Genomics & Informatics
    • /
    • v.12 no.1
    • /
    • pp.2-11
    • /
    • 2014
  • After the initial enthusiasm of the human genome project, it became clear that without additional data pertaining to the epigenome, i.e., how the genome is marked at specific developmental periods, in different tissues, as well as across individuals and species-the promise of the genome sequencing project in understanding biology cannot be fulfilled. This realization prompted several large-scale efforts to map the epigenome, most notably the Encyclopedia of DNA Elements (ENCODE) project. While there is essentially a single genome in an individual, there are hundreds of epigenomes, corresponding to various types of epigenomic marks at different developmental times and in multiple tissue types. Unprecedented advances in next-generation sequencing (NGS) technologies, by virtue of low cost and high speeds that continue to improve at a rate beyond what is anticipated by Moore's law for computer hardware technologies, have revolutionized molecular biology and genetics research, and have in turn prompted innovative ways to reduce the problem of measuring cellular events involving DNA or RNA into a sequencing problem. In this article, we provide a brief overview of the epigenome, the various types of epigenomic data afforded by NGS, and some of the novel discoveries yielded by the epigenomics projects. We also provide ample references for the reader to get in-depth information on these topics.

A Study on Real Time Control of Resin Transfer Molding (RTM 공정의 실시간 제어에 관한 연구)

  • Jeon Young Jae;Um Moon Kwang;Byun Joon Hyung;Lee Woo Il
    • Composites Research
    • /
    • v.18 no.4
    • /
    • pp.35-43
    • /
    • 2005
  • In resin transfer molding(RTM), race-track effects and non-uniform fiber volume fraction may cause undesirable resin flow patterns and thus result in dry spots, which affect the mechanical properties of the finished parts. In this study, a real time RTM control strategy to reduce these unfavorable effects is proposed. This control rule is accomplished by means of the permeability mapping and pressure regulation. Through numerical simulations, the validity of the proposed scheme is demonstrated.

Learning System for Big Data Analysis based on the Raspberry Pi Board (라즈베리파이 보드 기반의 빅데이터 분석을 위한 학습 시스템)

  • Kim, Young-Geun;Jo, Min-Hui;Kim, Won-Jung
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.11 no.4
    • /
    • pp.433-440
    • /
    • 2016
  • In order to construct a system for big data processing, one needs to configure the node by using network equipments to connect multiple computers or establish cloud environments through virtual hosts on a single computer. However, there are many restrictions on constructing the big data analysis system including complex system configuration and cost. These constraints are becoming a major obstacle to professional manpower training for big data areas which is emerging as one of the most important national competitiveness. As a result, for professional manpower training of big data areas, this paper proposes a Raspberry Pi Board based educational big data processing system which is capable of practical training at an affordable price.

An Extraction Method of Sentiment Infromation from Unstructed Big Data on SNS (SNS상의 비정형 빅데이터로부터 감성정보 추출 기법)

  • Back, Bong-Hyun;Ha, Ilkyu;Ahn, ByoungChul
    • Journal of Korea Multimedia Society
    • /
    • v.17 no.6
    • /
    • pp.671-680
    • /
    • 2014
  • Recently, with the remarkable increase of social network services, it is necessary to extract interesting information from lots of data about various individual opinions and preferences on SNS(Social Network Service). The sentiment information can be applied to various fields of society such as politics, public opinions, economics, personal services and entertainments. To extract sentiment information, it is necessary to use processing techniques that store a large amount of SNS data, extract meaningful data from them, and search the sentiment information. This paper proposes an efficient method to extract sentiment information from various unstructured big data on social networks using HDFS(Hadoop Distributed File System) platform and MapReduce functions. In experiments, the proposed method collects and stacks data steadily as the number of data is increased. When the proposed functions are applied to sentiment analysis, the system keeps load balancing and the analysis results are very close to the results of manual work.

Development of G-code generating software for 3D printer in Hadoop (Hadoop에서 3D 프린팅용 G-code 생성 소프트웨어 개발)

  • Lee, Kyuyoung;Nam, Kiwon;Kim, Gunyoung;Kim, Sungsuk;Yang, Sun-Ok
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.04a
    • /
    • pp.78-80
    • /
    • 2017
  • 3D 프린터를 이용하여 출력을 하기 위해서는 3D 모델 데이터를 G-code로 변환하는 과정을 수행해야 한다. 일반적으로 3D 모델은 STL 파일 형식으로 저장되는데, 이 파일은 대개 삼각형 형식인 페이셋들의 좌표 데이터를 포함하고 있다. 만약 3D 모델의 크기가 커지거나 정밀도가 높아진다면, 페이셋의 수가 매우 많아지게 되고, 결과적으로 3D 모델에서 G-code로 변환하는 시간이 길어지게 된다. 본 논문에서는 널리 활용되고 있는 Hadoop에서 변환 소프트웨어를 개발하고자 하였다. Hadoop은 마스터 노드와 여러 데이터 노드들이 Map-Reduce 방식으로 작업을 수행한다. 이러한 노드들은 하둡 파일시스템(HDFS)을 공유할 수 있어 작업을 효율적으로 수행할 수 있다. 이에 본 논문에서는 이 시스템의 기능을 활용하여 기존에 개발된 분산 알고리즘을 변형한 후 이를 구현하고자 한다.

Protopine reduces the inflammatory activity of lipopolysaccharide-stimulated murine macrophages

  • Bae, Deok-Sung;Kim, Young-Hoon;Pan, Cheol-Ho;Nho, Chu-Won;Samdan, Javzan;Yansan, Jamyansan;Lee, Jae-Kwon
    • BMB Reports
    • /
    • v.45 no.2
    • /
    • pp.108-113
    • /
    • 2012
  • Protopine is an isoquinoline alkaloid contained in plants in northeast Asia. In this study, we investigated whether protopine derived from Hypecoum erectum L could suppress lipopolysaccharide (LPS)-induced inflammatory responses in murine macrophages (Raw 264.7 cells). Protopine was found to reduce nitric oxide (NO), cyclooxygenase-2 (COX-2), and prostaglandin $E_2$ ($PGE_2$) production by LPS-stimulated Raw 264.7 cells, without a cytotoxic effect. Pre-treatment of Raw 264.7 cells with protopine reduced the production of pro-inflammatory cytokines. These inhibitory effects were caused by blocking phosphorylation of mitogen-activated protein kinases (MAP kinases) and also blocking activation of a nuclear factor kappa-light-chain-enhancer of activated B cells (NF-${\kappa}B$).

Prediction of Chip Forms using Neural Network and Experimental Design Method (신경회로망과 실험계획법을 이용한 칩형상 예측)

  • 한성종;최진필;이상조
    • Journal of the Korean Society for Precision Engineering
    • /
    • v.20 no.11
    • /
    • pp.64-70
    • /
    • 2003
  • This paper suggests a systematic methodology to predict chip forms using the experimental design technique and the neural network. Significant factors determined with ANOVA analysis are used as input variables of the neural network back-propagation algorithm. It has been shown that cutting conditions and cutting tool shapes have distinct effects on the chip forms, so chip breaking. Cutting tools are represented using the Z-map method, which differs from existing methods using some chip breaker parameters. After training the neural network with selected input variables, chip forms are predicted and compared with original chip forms obtained from experiments under same input conditions, showing that chip forms are same at all conditions. To verify the suggested model, one tool not used in training the model is chosen and input to the model. Under various cutting conditions, predicted chip forms agree well with those obtained from cutting experiments. The suggested method could reduce the cost and time significantly in designing cutting tools as well as replacing the“trial-and-error”design method.