• Title/Summary/Keyword: Map Balancing

Search Result 42, Processing Time 0.024 seconds

A Parallel HDFS and MapReduce Functions for Emotion Analysis (감성분석을 위한 병렬적 HDFS와 맵리듀스 함수)

  • Back, BongHyun;Ryoo, Yun-Kyoo
    • Journal of the Korea society of information convergence
    • /
    • v.7 no.2
    • /
    • pp.49-57
    • /
    • 2014
  • Recently, opinion mining is introduced to extract useful information from SNS data and to evaluate the true intention of users. Opinion mining are required several efficient techniques to collect and analyze a large amount of SNS data and extract meaningful data from them. Therefore in this paper, we propose a parallel HDFS(Hadoop Distributed File System) and emotion functions based on Mapreduce to extract some emotional information of users from various unstructured big data on social networks. The experiment results have verified that the proposed system and functions perform faster than O(n) for data gathering time and loading time, and maintain stable load balancing for memory and CPU resources.

  • PDF

Reconstitution of CB Trie for the Efficient Hangul Retrieval (효율적인 한글 탐색을 위한 CB 트라이의 재구성)

  • Jung, Kyu-Cheol
    • Convergence Security Journal
    • /
    • v.7 no.4
    • /
    • pp.29-34
    • /
    • 2007
  • This paper proposes RCB(Reduced Compact Binary) trie to correct faults of CB(Compact Binary) trie. First, in the case of CB trie, a compact structure was tried for the first time, but as the amount of data was increasing, that of inputted data gained and much difficulty was experienced in insertion due to the dummy nods used in balancing trees. On the other hand, if the HCB trie realized hierarchically, given certain depth to prevent the map from increasing on the right, reached the depth, the method for making new trees and connecting to them was used. Eventually, fast progress could be made in the inputting and searching speed, but this had a disadvantage of the storage space becoming bigger because of the use of dummy nods like CB trie and of many tree links. In the case of RCB trie in this thesis, a capacity is increased by about 60% by completely cutting down dummy nods.

  • PDF

Dynamic Load Management Method for Spatial Data Stream Processing on MapReduce Online Frameworks (맵리듀스 온라인 프레임워크에서 공간 데이터 스트림 처리를 위한 동적 부하 관리 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.8
    • /
    • pp.535-544
    • /
    • 2018
  • As the spread of mobile devices equipped with various sensors and high-quality wireless network communications functionsexpands, the amount of spatio-temporal data generated from mobile devices in various service fields is rapidly increasing. In conventional research into processing a large amount of real-time spatio-temporal streams, it is very difficult to apply a Hadoop-based spatial big data system, designed to be a batch processing platform, to a real-time service for spatio-temporal data streams. This paper extends the MapReduce online framework to support real-time query processing for continuous-input, spatio-temporal data streams, and proposes a load management method to distribute overloads for efficient query processing. The proposed scheme shows a dynamic load balancing method for the nodes based on the inflow rate and the load factor of the input data based on the space partition. Experiments show that it is possible to support efficient query processing by distributing the spatial data stream in the corresponding area to the shared resources when load management in a specific area is required.

A Study on Small-sized Index Structure and Fast Retrieval Method Using The RCB trio (RCB트라이를 이용한 빠른 검색과 소용량 색인 구조에 관한 연구)

  • Jung, Kyu-Cheol
    • Journal of the Korea Society of Computer and Information
    • /
    • v.12 no.4
    • /
    • pp.11-19
    • /
    • 2007
  • This paper proposes RCB(Reduced Compact Binary) tie to correct faults of both CB(Compact Binary) tie and HCB(Hierarchical Compact Binary) trie. First, in the case of CB trie, a compact structure was tried for the first time, but as the amount of data was increasing, that of inputted data gained and much difficulty was experienced in insertion due to the dummy nods used in balancing trees. On the other hand, if the HCB trie realized hierarchically, given certain depth to prevent the map from increasing on the right, reached the depth, the method for making new trees and connecting to them was used. Eventually, fast progress could be made in the inputting and searching speed, but this had a disadvantage of the storage space becoming bigger because of the use of dummy nods like CB trie and of many tree links. In the case of RCB trie in this thesis, the tree-map could be reduced by about 35% by completely cutting down dummy nods and the whole size by half, compared with the HCB trie.

  • PDF

A Portfolio Model for National IT R&D Strategy Project Selection Methods

  • Ryu, Dong-Hyun;Lee, Woo-Jin
    • Journal of information and communication convergence engineering
    • /
    • v.9 no.5
    • /
    • pp.491-499
    • /
    • 2011
  • In this paper, we offer a new strategic portfolio model for national IT R&D project selection in Korea. A risk and return (R-R) portfolio model was developed using an objectively quantified index on the two axes of risk and return, in order to select a strategic project and allocate resources in compliance with a national IT R&D strategy. We strategize using the R-R portfolio model to solve the non-strategy and subjectivity problems of the existing national R&D project selection model. We also use the quantified evaluation index of the IT technology road map (TRM) and the technical level reports (TLR) for the subjectivity of project selection, and try to discover the weights using the analytic hierarchy process (AHP). In addition, we intend to maximize the chance for a successful national IT R&D project, by selecting a strategic portfolio project and balancing the allocation of resources effectively and objectively.

Service Resource, Capability and Performance: an Exploratory Study on Hotel Industry (호텔 서비스 자원에 따른 운영역량과 성과의 차이에 관한 연구)

  • Cho, Jungeun
    • Journal of Korean Society for Quality Management
    • /
    • v.41 no.4
    • /
    • pp.513-525
    • /
    • 2013
  • Purpose: The purpose of this paper are to propose a strategic map for hotel industry through analyzing the relationship between service resource, operational capabilities, and performance. Methods: A phone survey was conducted among Korean hotels, and 102 data sets were collected. Measurement items are assessed using both cognitive and objective scales. Results: As results, 'superior group', which is superior in both physical resources and human resources, is excellent in all capabilities and also in room occupancy rate. On the other hands, 'inferior group', which is inferior in both physical resources and human resources, shows lower achievements is in most areas except speed. In addition, physical superior group is better than human superior group in most capabilities except speed, but human superior group shows better results than physical superior group in both room occupancy rate and customer satisfaction. Conclusions: Through the empirical analysis, the conclusions attained are as follows; First, human resources affect customer satisfaction more directly that physical resources. Second, the balancing between physical resources and human resources has an importance to improve operational capabilities.

Wafer Map Defect Pattern Classification with Progressive Pseudo-Labeling Balancing (점진적 데이터 평준화를 이용한 반도체 웨이퍼 영상 내 결함 패턴 분류)

  • Do, Jeonghyeok;Kim, Munchurl
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2020.11a
    • /
    • pp.248-251
    • /
    • 2020
  • 전 반도체 제조 및 검사 공정 과정을 자동화하는 스마트 팩토리의 실현에 있어 제품 검수를 위한 검사 장비는 필수적이다. 하지만 딥 러닝 모델 학습을 위한 데이터 처리 과정에서 엔지니어가 전체 웨이퍼 영상에 대하여 결함 항목 라벨을 매칭하는 것은 현실적으로 불가능하기 때문에 소량의 라벨 (labeled) 데이터와 나머지 라벨이 없는 (unlabeled) 데이터를 적절히 활용해야 한다. 또한, 웨이퍼 영상에서 결함이 발생하는 빈도가 결함 종류별로 크게 차이가 나기 때문에 빈도가 적은 (minor) 결함은 잡음처럼 취급되어 올바른 분류가 되지 않는다. 본 논문에서는 소량의 라벨 데이터와 대량의 라벨이 없는 데이터를 동시에 활용하면서 결함 사이의 발생 빈도 불균등 문제를 해결하는 점진적 데이터 평준화 (progressive pseudo-labeling balancer)를 제안한다. 점진적 데이터 평준화를 이용해 분류 네트워크를 학습시키는 경우, 기존의 테스트 정확도인 71.19%에서 6.07%-p 상승한 77.26%로 약 40%의 라벨 데이터가 추가된 것과 같은 성능을 보였다.

  • PDF

Efficient and Robust Correspondence Detection between Unbalanced Stereo Images

  • Kim, Yong-Ho;Kim, Jong-Su;Lee, Sangkeun;Choi, Jong-Soo
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.1 no.3
    • /
    • pp.161-170
    • /
    • 2012
  • This paper presents an efficient and robust approach for determining the correspondence between unbalanced stereo images. The disparity vectors were used instead of feature points, such as corners, to calculate a correspondence relationship. For a faster and optimal estimation, the vectors were classified into several regions, and the homography of each region was calculated using the RANSAC algorithm. The correspondence image was calculated from the images transformed by each homography. Although it provided good results under normal conditions, it was difficult to obtain reliable results in an unbalanced stereo pair. Therefore, a balancing method is also proposed to minimize the unbalance effects using the histogram specification and structural similarity index. The experimental results showed that the proposed approach outperformed the baseline algorithms with respect to the speed and peak-signal-to-noise ratio. This work can be applied to practical fields including 3D depth map acquisition, fast stereo coding, 2D-to-3D conversion, etc.

  • PDF

Dynamic Load Balancing Mechanism for MMORPG (MMORPG를 위한 동적 부하 균등화 기법)

  • Lim, Chae-Gyun;Rho, Kyung-Taeg
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.9 no.5
    • /
    • pp.199-203
    • /
    • 2009
  • Recently, Massively Multiplayer Online Role-Playing Games (MMORPGs) has become increasingly popular, which continue to increase the number of game player. The volume of the game world also has been extended on a large scale. Existing Map-based distributed server architecture divides the game world into the rectangular regions and allocates the registered player in each region to the server responsible for that region. Players tend to concentrate in certain regions of the game world, which makes some special server overloaded. This paper proposes to change the boundary between servers to solve such a unbalanced load problem. Our proposed method first finds the overloaded server and then searches for its neighboring lightest loaded server to share with the overload evenly. Finally we implemented performance evaluation to demonstrate the efficiency of this approach.

  • PDF

Reconstitution of Compact Binary trie for the Efficient Retrieval of Hangul UniCODE Text (한글 유니코드 텍스트의 효율적인 탐색을 위한 컴팩트 바이너리 트라이의 재구성)

  • Jung, Kyu Cheol;Lee, Jong Chan;Park, Sang Joon;Kim, Byung Gi
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.5 no.2
    • /
    • pp.21-28
    • /
    • 2009
  • This paper proposes RCBT(Reduced Compact Binary trie) to correct faults of CBT (Compact Binary trie). First, in the case of CBT, a compact structure was tried for the first time, but as the amount of data was increasing, that of inputted data gained and much difficulty was experienced in insertion due to the dummy nodes used in balancing trees. On the other hand, if the HCBT realized hierarchically, given certain depth to prevent the map from increasing onthe right, reached the depth, the method for making new trees and connecting to them was used. Eventually, fast progress could be made in the inputting and searching speed, but this had a disadvantage of the storage space becoming bigger because of the use of dummy nods like CBT and of many tree links. In the case of RCBT in this thesis, a capacity is increased by about 60% by completely cutting down dummy nods.