• Title/Summary/Keyword: Big Data Distributed Processing System

Search Result 114, Processing Time 0.024 seconds

Technique for Concurrent Processing Graph Structure and Transaction Using Topic Maps and Cassandra (토픽맵과 카산드라를 이용한 그래프 구조와 트랜잭션 동시 처리 기법)

  • Shin, Jae-Hyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.1 no.3
    • /
    • pp.159-168
    • /
    • 2012
  • Relation in the new IT environment, such as the SNS, Cloud, Web3.0, has become an important factor. And these relations generate a transaction. However, existing relational database and graph database does not processe graph structure representing the relationships and transactions. This paper, we propose the technique that can be processed concurrently graph structures and transactions in a scalable complex network system. The proposed technique simultaneously save and navigate graph structures and transactions using the Topic Maps data model. Topic Maps is one of ontology language to implement the semantic web(Web 3.0). It has been used as the navigator of the information through the association of the information resources. In this paper, the architecture of the proposed technique was implemented and design using Cassandra - one of column type NoSQL. It is to ensure that can handle up to Big Data-level data using distributed processing. Finally, the experiments showed about the process of storage and query about typical RDBMS Oracle and the proposed technique to the same data source and the same questions. It can show that is expressed by the relationship without the 'join' enough alternative to the role of the RDBMS.

Big Data based Tourist Attractions Recommendation - Focus on Korean Tourism Organization Linked Open Data - (빅데이터 기반 관광지 추천 시스템 구현 - 한국관광공사 LOD를 중심으로 -)

  • Ahn, Jinhyun;Kim, Eung-Hee;Kim, Hong-Gee
    • Management & Information Systems Review
    • /
    • v.36 no.4
    • /
    • pp.129-148
    • /
    • 2017
  • Conventional exhibition management information systems recommend tourist attractions that are close to the place in which an exhibition is held. Some recommended attractions by the location-based recommendation could be meaningless when nothing is related to the exhibition's topic. Our goal is to recommend attractions that are related to the content presented in the exhibition, which can be coined as content-based recommendation. Even though human exhibition curators can do this, the quality is limited to their manual task and knowledge. We propose an automatic way of discovering attractions relevant to an exhibition of interests. Language resources are incorporated to discover attractions that are more meaningful. Because a typical single machine is unable to deal with such large-scale language resources efficiently, we implemented the algorithm on top of Apache Spark, which is a well-known distributed computing framework. As a user interface prototype, a web-based system is implemented that provides users with a list of relevant attractions when users are browsing exhibition information, available at http://bike.snu.ac.kr/WARP. We carried out a case study based on Korean Tourism Organization Linked Open Data with Korean Wikipedia as a language resource. Experimental results are demonstrated to show the efficiency and effectiveness of the proposed system. The effectiveness was evaluated against well-known exhibitions. It is expected that the proposed approach will contribute to the development of both exhibition and tourist industries by motivating exhibition visitors to become active tourists.

  • PDF

E-Discovery Process Model and Alternative Technologies for an Effective Litigation Response of the Company (기업의 효과적인 소송 대응을 위한 전자증거개시 절차 모델과 대체 기술)

  • Lee, Tae-Rim;Shin, Sang-Uk
    • Journal of Digital Convergence
    • /
    • v.10 no.8
    • /
    • pp.287-297
    • /
    • 2012
  • In order to prepare for the introduction of the E-Discovery system from the United States and to cope with some causable changes of legal systems, we propose a general E-Discovery process and essential tasks of the each phase. The proposed process model is designed by the analysis of well-known projects such as EDRM, The Sedona Conference, which are advanced research for the standardization of E-Discovery task procedures and for the supply of guidelines to hands-on workers. In addition, Machine Learning Algorithms, Open-source libraries for the Information Retrieval and Distributed Processing technologies based on the Hadoop for big data are introduced and its application methods on the E-Discovery work scenario are proposed. All this information will be useful to vendors or people willing to develop the E-Discovery service solution. Also, it is very helpful to company owners willing to rebuild their business process and it enables people who are about to face a major lawsuit to handle a situation effectively.

A Study on the Seepage Behavior of Embankment with Weak Zone using Numerical Analysis and Model Test (취약대를 가진 모형제방의 침투거동에 관한 연구)

  • Park, Mincheol;Im, Eunsang;Lee, Seokyoung;Han, Heuisoo
    • Journal of the Korean GEO-environmental Society
    • /
    • v.17 no.7
    • /
    • pp.5-13
    • /
    • 2016
  • This research is focused on the seepage behavior of embankment which had the weak zone with big permeability. The distributed TDR (Time Domain Reflectometer) and point sensors such as settlement gauge, pore water pressuremeter, vertical total stressmeter, and FDR (Frequency Domain Reflectometer) sensor were used to measure the seepage characteristics and embankment behavior. Also, the measured data were compared to the data of 2-D and 3-D numerical analysis. The dimension of model embankment was 7 m length, 5 m width and 1.5 m height, which is composed of fine-grained sands and the water level of embankment was 1.3 m height. The seepage behavior of measuring and numerical analysis were very similar, it means that the proper sensing system can monitor the real-time safety of embankment. The result by 2-D and 3-D numerical analysis showed similar saturation processing, however in case of weak zone, the phreatic lines of 2-D showed faster movement than that of 3-D analysis, and finally they converged.