DOI QR코드

DOI QR Code

하둡 환경에 적합한 클러스터 그룹 기반 속성 정보를 이용한 빅 데이터 관리 기법

Big Data Management Scheme using Property Information based on Cluster Group in adopt to Hadoop Environment

  • 한군희 (백석대학교 정보통신공학과) ;
  • 정윤수 (목원대학교 정보통신융합공학부)
  • Han, Kun-Hee (Dept. of Information Communication & Engineeringe, Baeseok University) ;
  • Jeong, Yoon-Su (Dept. of Information Communication & Engineeringe, Mokwon University)
  • 투고 : 2015.07.19
  • 심사 : 2015.09.20
  • 발행 : 2015.09.28

초록

소셜 네트워크 기술이 발달하면서 빅 데이터 서비스에 대한 관심이 증가하고 있다. 그러나, 중앙 서버가 아닌 분산 서버에 저장된 데이터를 손쉽게 검색 및 추출하기 위한 기술은 부족한 실정이다. 본 논문에서는 빅 데이터 서비스를 제공하는 컨텐츠 서버와 관리 서버에서 사용자가 원하는 정보의 처리시간을 최소화하기 위한 빅 데이터 관리 기법을 제안하다. 제안 기법은 빅 데이터의 종류, 기능, 특성에 따라 데이터를 그룹으로 분류한 후 분류된 그룹내 데이터를 속성정보와 연계하여 해쉬체인에 적용한다. 또한, 분산 서버에 저장된 데이터를 최단 시간에 추출하기 위해서 데이터 인덱스 정보(DII, Data Index Information)를 그룹화하여 데이터에 부여된 다중의 속성 정보를 분류하여 데이터의 처리 속도를 향상시킨다. 실험 결과, 클러스터 그룹 수에 따른 데이터의 평균 검색 시간은 평균 14.6% 향상되었고, 키워드 수에 따른 데이터 처리시간은 평균 13% 단축되었다.

Social network technology has been increasing interest in the big data service and development. However, the data stored in the distributed server and not on the central server technology is easy enough to find and extract. In this paper, we propose a big data management techniques to minimize the processing time of information you want from the content server and the management server that provides big data services. The proposed method is to link the in-group data, classified data and groups according to the type, feature, characteristic of big data and the attribute information applied to a hash chain. Further, the data generated to extract the stored data in the distributed server to record time for improving the data index information processing speed of the data classification of the multi-attribute information imparted to the data. As experimental result, The average seek time of the data through the number of cluster groups was increased an average of 14.6% and the data processing time through the number of keywords was reduced an average of 13%.

키워드

참고문헌

  1. H. Hu, Y. Wen, T. S. Chua, X. Li, "Toward Scalable Systems for Big Data Anaqlytics: A Technology Tutorial", IEEE Access, vol. 2, pp. 652-687, 2014. https://doi.org/10.1109/ACCESS.2014.2332453
  2. P. Russom, "Big Data Analytics", TDWI Research Fourth Quarter, pp. 6, Dec. 2011.
  3. V. Gadepally, J. Kepner. "Big data dimensional analysis", 2014 IEEE High Performance Extreme Computing Conference(HPEC) pp. 1-6, Sep. 2014.
  4. Y. Demchenko, C. De Laat, P. Membrey, "Defining architecture components of the Big data Ecosystem", 2014 International conference on Collaboration Technologies and Systems(CTS), pp.104-112, May, 2014.
  5. J. Manyika, M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh, A. H. Byers, "Big Data: The Next Frontier for Innovation, Competition and Productivity", Mckinsey Global Institute, pp. 1-137. 2011.
  6. S. Abdul-Rahman, A. A. Bakar, Z. -A, Mohamed-Hussein, "Optimizing Big Data in Bioinformatics with Swarm Algorithms", 2013 IEEE 16th International Conference on Computational Science and Engineering (CSE), pp. 1091-1095, Dec. 2013.
  7. A. Chong, T. D. Gedeon, L. T. Koczy, "Hierarchical fuzzy classifier for bioinformatics data", 2003. Proceedings. Seventh International Symposium on Signal Processing and Its Applications, pp. 45-48, July 2003.
  8. E. Ahmed, "Resource capability discovery and description management system for bioinformatics Data and service Integration - an experiment with gene regulatory networks", 2008. ICCIT 2008. 11th International Conference on Computer and Information Technology, pp. 56-61, Dec. 2008.
  9. Jiang Peiyong, Sun Xiaoxi, E.Z. Chen, Sun Kun, R. W. K. Chiu, Y. M. D. Lo, Sun Hao, "Methy-Pipe: An integrated bioinformatics data analysis pipeline for whole genome methylome analysis", 2010 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW), pp. 585-590, Dec. 2010.
  10. A. Katal, M. Wazid, R. H. Goudar, "Big data: Issues, challenges, tools and Good practices ", 2013 Sixth International Conference on Contemporary Computing(IC3), pp. 404-409, Aug. 2013.
  11. Y. C. Jung. "Big Data revolution and media policy issues", KISDI Premium Report, Vol. 12, No. 2, pp. 1-22, 2012.
  12. S. H. Kim, N. U. Kim, t. M. Chung, "Attribute Relationship Evaluation Methodology for Big Data Seucrity", 2013 International Conference on IT Convergence and Security(ICITCS), pp. 1-4, Dec. 2013.
  13. S. Y. Son, "Big data, online marketing and privacy protection", KISDI Premium Report, Vol. 13, No. 1, pp.1-26, 2013.
  14. J. T. Kim, B. J. Oh, J. Y. Park, "Standard Trends for the BigData Technologies", 2013 Electronics and Telecommunications Trends, Vol. 28, No. 1, pp. 92-99, 2013.
  15. M. Paryasto, A. Alamsyah, B. Rahardjo, Kuspriyanto, "Big-data security management issues", 2014 2nd International Conference on Information and Communication Technology(ICoICT), pp. 59-63, May, 2014.