• Title/Summary/Keyword: Query process

Search Result 523, Processing Time 0.023 seconds

Design and Implementation of an Efficient Web Services Data Processing Using Hadoop-Based Big Data Processing Technique (하둡 기반 빅 데이터 기법을 이용한 웹 서비스 데이터 처리 설계 및 구현)

  • Kim, Hyun-Joo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.1
    • /
    • pp.726-734
    • /
    • 2015
  • Relational databases used by structuralizing data are the most widely used in data management at present. However, in relational databases, service becomes slower as the amount of data increases because of constraints in the reading and writing operations to save or query data. Furthermore, when a new task is added, the database grows and, consequently, requires additional infrastructure, such as parallel configuration of hardware, CPU, memory, and network, to support smooth operation. In this paper, in order to improve the web information services that are slowing down due to increase of data in the relational databases, we implemented a model to extract a large amount of data quickly and safely for users by processing Hadoop Distributed File System (HDFS) files after sending data to HDFSs and unifying and reconstructing the data. We implemented our model in a Web-based civil affairs system that stores image files, which is irregular data processing. Our proposed system's data processing was found to be 0.4 sec faster than that of a relational database system. Thus, we found that it is possible to support Web information services with a Hadoop-based big data processing technique in order to process a large amount of data, as in conventional relational databases. Furthermore, since Hadoop is open source, our model has the advantage of reducing software costs. The proposed system is expected to be used as a model for Web services that provide fast information processing for organizations that require efficient processing of big data because of the increase in the size of conventional relational databases.

The Effect of Deal-Proneness in the Searching Pattern on the Purchase Probability of Customer in Online Travel Services (소비자 키워드광고 탐색패턴에 나타난 촉진지향성이 온라인 여행상품 구매확률에 미치는 영향)

  • Kim, Hyun Gyo;Lee, Dong Il
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.39 no.1
    • /
    • pp.29-48
    • /
    • 2014
  • The recent keyword advertising does not reflect the individual customer searching pattern because it is focused on each keyword at the aggregate level. The purpose of this research is to observe processes of customer searching patterns. To be specific, individual deal-proneness is mainly concerned. This study incorporates location as a control variable. This paper examines the relationship between customers' searching patterns and probability of purchase. A customer searching session, which is the collection of sequence of keyword queries, is utilized as the unit of analysis. The degree of deal-proneness is measured using customer behavior which is revealed by customer searching keywords in the session. Deal-proneness measuring function calculates the discount of deal prone keyword leverage in accordance with customer searching order. Location searching specificity function is also calculated by the same logic. The analyzed data is narrowed down to the customer query session which has more than two keyword queries. The number of the data is 218,305 by session, which is derived from Internet advertising agency's (COMAS) advertisement managing data and the travel business advertisement revenue data from advertiser's. As a research result, there are three types of the deal-prone customer. At first, there is an unconditional active deal-proneness customer. It is the customer who has lower deal-proneness which means that he/she utilizes deal-prone keywords in the last phase. He/she starts searching a keyword like general ones and then finally purchased appropriate products by utilizing deal-prone keywords in the last time. Those two types of customers have the similar rates of purchase. However, the last type of the customer has middle deal-proneness; who utilizes deal-prone keywords in the middle of the process. This type of a customer closely gets into the information by employing deal-prone keywords but he/she could not find out appropriate alternative then would modify other keywords to look for other alternatives. That is the reason why the purchase probability in this case would be decreased Also, this research confirmed that there is a loyalty effect using location searching specificity. The customer who has higher trip loyalty for specificity location responds to selected promotion rather than general promotion. So, this customer has a lower probability to purchase.

Automatic Generation of DB Images for Testing Enterprise Systems (전사적 응용시스템 테스트를 위한 DB이미지 생성에 관한 연구)

  • Kwon, Oh-Seung;Hong, Sa-Neung
    • Journal of Intelligence and Information Systems
    • /
    • v.17 no.4
    • /
    • pp.37-58
    • /
    • 2011
  • In general, testing DB applications is much more difficult than testing other types of software. The fact that the DB states as much as the input data influence and determine the procedures and results of program testing is one of the decisive reasons for the difficulties. In order to create and maintain proper DB states for testing, it not only takes a lot of time and efforts, but also requires extensive IT expertise and business knowledge. Despite the difficulties, there are not enough research and tools for the needed help. This article reports the result of research on automatic creation and maintenance of DB states for testing DB applications. As its core, this investigation develops an automation tool which collects relevant information from a variety of sources such as log, schema, tables and messages, combines collected information intelligently, and creates pre- and post-Images of database tables proper for application tests. The proposed procedures and tool are expected to be greatly helpful for overcoming inefficiencies and difficulties in not just unit and integration tests but including regression tests. Practically, the tool and procedures proposed in this research allows developers to improve their productivity by reducing time and effort required for creating and maintaining appropriate DB sates, and enhances the quality of DB applications since they are conducive to a wider variety of test cases and support regression tests. Academically, this research deepens our understanding and introduces new approach to testing enterprise systems by analyzing patterns of SQL usages and defining a grammar to express and process the patterns.

Technique for Concurrent Processing Graph Structure and Transaction Using Topic Maps and Cassandra (토픽맵과 카산드라를 이용한 그래프 구조와 트랜잭션 동시 처리 기법)

  • Shin, Jae-Hyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.1 no.3
    • /
    • pp.159-168
    • /
    • 2012
  • Relation in the new IT environment, such as the SNS, Cloud, Web3.0, has become an important factor. And these relations generate a transaction. However, existing relational database and graph database does not processe graph structure representing the relationships and transactions. This paper, we propose the technique that can be processed concurrently graph structures and transactions in a scalable complex network system. The proposed technique simultaneously save and navigate graph structures and transactions using the Topic Maps data model. Topic Maps is one of ontology language to implement the semantic web(Web 3.0). It has been used as the navigator of the information through the association of the information resources. In this paper, the architecture of the proposed technique was implemented and design using Cassandra - one of column type NoSQL. It is to ensure that can handle up to Big Data-level data using distributed processing. Finally, the experiments showed about the process of storage and query about typical RDBMS Oracle and the proposed technique to the same data source and the same questions. It can show that is expressed by the relationship without the 'join' enough alternative to the role of the RDBMS.

Efficient Rotation-Invariant Boundary Image Matching Using the Envelope-based Lower Bound (엔빌로프 기반 하한을 사용한 효율적인 회전-불변 윤곽선 이미지 매칭)

  • Kim, Sang-Pil;Moon, Yang-Sae;Hong, Sun-Kyong
    • The KIPS Transactions:PartD
    • /
    • v.18D no.1
    • /
    • pp.9-22
    • /
    • 2011
  • In this paper we present an efficient solution to rotation?invariant boundary image matching. Computing the rotation-invariant distance between image time-series is a time-consuming process since it requires a lot of Euclidean distance computations for all possible rotations. In this paper we propose a novel solution that significantly reduces the number of distance computations using the envelope-based lower bound. To this end, we first present how to construct a single envelope from a query sequence and how to obtain a lower bound of the rotation-invariant distance using the envelope. We then show that the single envelope-based lower bound can reduce a number of distance computations. This approach, however, may cause bad performance since it may incur a larger lower bound by considering all possible rotated sequences in a single envelope. To solve this problem, we present a concept of rotation interval, and using the rotation interval we generalize the envelope-based lower bound by exploiting multiple envelopes rather than a single envelope. We also propose equi-width and envelope minimization divisions as the method of determining rotation intervals in the multiple envelope approach. Experimental results show that our envelope-based solutions outperform existing solutions by one or two orders of magnitude.

Application of Geographic Information Systems for Effective Management of University Forests (대학연습림의 효율적 관리를 위한 지리정보시스템의 활용방안)

  • Kwon, Taeho;Kim, Taekyun
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.2 no.3
    • /
    • pp.81-90
    • /
    • 1999
  • The functional change of university forest have led to need more complicated techniques for forest management strategies, and more information about forest and natural environment. Therefore the systematic tools, like the so-called Forest Information System to which apply the techniques of geographic information system, are eagerly required for collecting, editing, managing, analyzing the various data about forest and environment, and for supporting the decision-making process. The digital mapping, which could be a primary step to construct the Forest Information System, was carried out using the many kinds of thematic spatial data referring to the Seongju Experimental Forest of Taegu University. As a result, various digital maps including forest type, soil type and so on were constructed. And then we made an user-interface system to link the attributive data in management plan to the thematic spatial data. This system was regarded as the effective tool capable of the more rapid query, analysis and update of related data for systematic management of university forest. Moreover, it would be a useful tool of decision-making in devising, assessing and operating the plan of forest management and development. But there would be much room for supplementation and improvement to make the more convenient and powerful system for the external demands, therefore more concerns and efforts in collecting, revising and updating the data is continuously required.

  • PDF

A Dual Processing Load Shedding to Improve The Accuracy of Aggregate Queries on Clustering Environment of GeoSensor Data Stream (클러스터 환경에서 GeoSensor 스트림 데이터의 집계질의의 정확도 향상을 위한 이중처리 부하제한 기법)

  • Ji, Min-Sub;Lee, Yeon;Kim, Gyeong-Bae;Bae, Hae-Young
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.1
    • /
    • pp.31-40
    • /
    • 2012
  • u-GIS DSMSs have been researched to deal with various sensor data from GeoSensors in ubiquitous environment. Also, they has been more important for high availability. The data from GeoSensors have some characteristics that increase explosively. This characteristic could lead memory overflow and data loss. To solve the problem, various load shedding methods have been researched. Traditional methods drop the overloaded tuples according to a particular criteria in a single server. Tuple deletion sensitive queries such as aggregation is hard to satisfy accuracy. In this paper a dual processing load shedding method is suggested to improve the accuracy of aggregation in clustering environment. In this method two nodes use replicated stream data for high availability. They process a stream in two nodes by using a characteristic they share stream data. Stream data are synchronized between them with a window as a unit. Then, processed results are merged. We gain improved query accuracy without data loss.

Field Mapping based on Virtual Office for Real time GIS in Field Survey for Natural Environment (자연환경조사에서 실시간 GIS구현을 위한 가상사무실 기반의 필드멥핑)

  • 엄정섭;김희두
    • Spatial Information Research
    • /
    • v.9 no.1
    • /
    • pp.51-72
    • /
    • 2001
  • It is frequently pointed out that the conventional field survey for natural environment has may limitations in terms of positional accuracy, real-time GIS data acquisition, and economic efficiency. The aim of this research was to develop an on site real-time mapping technique that enables the surveyor to input data in the field. The idea is based upon the recent trends in the field of Telecommunication and Information Technology that uses a GPS, wireless network computing, moving computing, etc. A virtual office approach has been adopted, in which a portable computer is linked to a GPS and field workers record data on the computer at the site and analyse data on site. This field mapping system has shown to be much less susceptible to the positional accuracy than that of th conventional approach. The Graphic User Interface, in particular, were ideally suited to combining positional information with attribute data which changes with every survey points. This interface allows users to interactively display and query GIS layers reproduced from the past survey results. The GIS database stored in the virtual office will serve to carry out a highly reliable survey since it could play a crucial role in identifying temporal and spatial changes occurred in the site. It is expected that integrated utilization of field data among the related agencies would be increased much more than before since the virtual office survey would be a powerful tool to ensure geometric fidelity in GIS database creation process. This paper also discusses the limitations and future direction of the present prototype research.

  • PDF

Design and Implementation of Web GIS Server Using Node.js (Node.js를 활용한 웹GIS 서버의 설계와 구현)

  • Jun, Sang Hwan;Doh, Kyoung Tae
    • Spatial Information Research
    • /
    • v.21 no.3
    • /
    • pp.45-53
    • /
    • 2013
  • Web GIS, based on the latest web-technology, has evolved to provide efficient and accurate spatial information to users. Furthermore, Web GIS Server has improved the performance constantly to respond user web requests and to offer spatial information service. This research aims to create a designed and implemented Web GIS Server that is named as Nodemap which uses the emergent technology, Node.js, which has been issued for an event-oriented, non-blocking I/O model framework for coding JavaScript on the server development. Basically, NodeMap is Web GIS Server that supports OGC implementation specification. It is designed to process GIS data by using DBMS, which supports spatial index and standard spatial query function. And NodeMap uses Node-Canvas module supported HTML5 canvas to render spatial information on tile map. Lastly, NodeMap uses Express module based connect module framework. NodaMap performance demonstration confirmed a possibility of applying Node.js as a (next/future) Web GIS Server development technology through the benchmarking. Having completed its quality test of NodeMap, this study has shown the compatibility and potential for Node.js as a Web GIS server development technology, and has shown the bright future of internet GIS service.

Grid-based Trajectory Cloaking Method for protecting Trajectory privacy in Location-based Services (위치기반서비스에서 개인의 궤적 정보를 보호하기 위한 그리드 기반 궤적 클로킹 기법)

  • Youn, Ji-hye;Song, Doo-hee;Cai, Tian-yuan;Park, Kwang-jin
    • Journal of Internet Computing and Services
    • /
    • v.18 no.5
    • /
    • pp.31-38
    • /
    • 2017
  • Recently with the rapid development of LBS (Location-based Services) technology, approaches of protecting user's location have gained tremendous attentions. For using LBS, users need to forward their real locations to LBS server. However, if the user sends his/her real location to LBS server, the server will have the all the information about user in LBS. Moreover, if the user opens it to LBS server for a long time, the trajectory of user may be released. In this paper, we propose GTC (Grid-based Trajectory Cloaking) method to address the privacy issue. Different from existing approaches, firstly the GTC method sets the predicting trajectory and divides the map into $2^n*2^n$ grid. After that we will generate cloaking regions according to user's desired privacy level. Finally the user sends them to LBS server randomly. The GTC method can make the cost of process less than sequential trajectory k-anonymity. Because of confusing the departure and destination, LBS server could not know the user's trajectory any more. Thus, we significantly improve the privacy level. evaluation results further verify the effectiveness and efficiency of our GTC method.