• Title/Summary/Keyword: group-by query

Search Result 49, Processing Time 0.029 seconds

Efficient Processing of Multiple Group-by Queries in MapReduce for Big Data Analysis (맵리듀스에서 빅데이터 분석을 위한 다중 Group-by 질의의 효율적인 처리 기법)

  • Park, Eunju;Park, Sojeong;Oh, Sohyun;Choi, Hyejin;Lee, Ki Yong;Shim, Junho
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.5
    • /
    • pp.387-392
    • /
    • 2015
  • MapReduce is a framework used to process large data sets in parallel on a large cluster. A group-by query is a query that partitions the input data into groups based on the values of the specified attributes, and then evaluates the value of the specified aggregate function for each group. In this paper, we propose an efficient method for processing multiple group-by queries using MapReduce. Instead of computing each group-by query independently, the proposed method computes multiple group-by queries in stages with one or more MapReduce jobs in order to reduce the total execution cost. We compared the performance of this method with the performance of a less sophisticated method that computes each group-by query independently. This comparison showed that the proposed method offers better performance in terms of execution time.

How Query by humming, a Music Information Retrieval System, is Being Used in the Music Education Classroom

  • Bradshaw, Brian
    • Journal of Multimedia Information System
    • /
    • v.4 no.3
    • /
    • pp.99-106
    • /
    • 2017
  • This study does a qualitative and quantitative analysis of how music by humming is being used by music educators in the classroom. Music by humming is part division of music information retrieval. In order to define what a music information retrieval system is first I need to define what it is. Berger and Lafferty (1999) define information retrieval as "someone doing a query to a retrieval system, a user begins with an information need. This need is an ideal document- perfect fit for the user, but almost certainly not present in the retrieval system's collection of documents. From this ideal document, the user selects a group of identifying terms. In the context of traditional IR, one could view this group of terms as akin to expanded query." Music Information Retrieval has its background in information systems, data mining, intelligent systems, library science, music history and music theory. Three rounds of surveys using question pro where completed. The study found that there were variances in knowledge, training and level of awareness of query by humming, music information retrieval systems. Those variance relationships where based on music specialty, level that they teach, and age of the respondents.

GOPES: Group Order-Preserving Encryption Scheme Supporting Query Processing over Encrypted Data

  • Lee, Hyunjo;Song, Youngho;Chang, Jae-Woo
    • Journal of Information Processing Systems
    • /
    • v.14 no.5
    • /
    • pp.1087-1101
    • /
    • 2018
  • As cloud computing has become a widespread technology, malicious attackers can obtain the private information of users that has leaked from the service provider in the outsourced databases. To resolve the problem, it is necessary to encrypt the database prior to outsourcing it to the service provider. However, the most existing data encryption schemes cannot process a query without decrypting the encrypted databases. Moreover, because the amount of the data is large, it takes too much time to decrypt all the data. For this, Programmable Order-Preserving Secure Index Scheme (POPIS) was proposed to hide the original data while performing query processing without decryption. However, POPIS is weak to both order matching attacks and data count attacks. To overcome the limitations, we propose a group order-preserving data encryption scheme (GOPES) that can support efficient query processing over the encrypted data. Since GOPES can preserve the order of each data group by generating the signatures of the encrypted data, it can provide a high degree of data privacy protection. Finally, it is shown that GOPES is better than the existing POPIS, with respect to both order matching attacks and data count attacks.

A Query Result Integrity Assurance Scheme Using an Order-preserving Encryption Scheme in the Database Outsourcing Environment (데이터베이스 아웃소싱 환경에서 순서 보존 암호화 기법을 이용한 질의 결과 무결성 검증 기법)

  • Jang, Miyoung;Chang, Jae Woo
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.97-106
    • /
    • 2015
  • Recently, research on database encryption for data protection and query result authentication methods has been performed more actively in the database outsourcing environment. Existing database encryption schemes are vulnerable to order matching and counting attack of intruders who have background knowledge of the original database domain. Existing query result integrity auditing methods suffer from the transmission overhead of verification object. To resolve these problems, we propose a group-order preserving encryption index and a query result authentication method based on the encryption index. Our group-order preserving encryption index groups the original data for data encryption and support query processing without data decryption. We generate group ids by using the Hilbert-curve so that we can protect the group information while processing a query. Finally, our periodic function based data grouping and query result authentication scheme can reduce the data size of the query result verification. Through performance evaluation, we show that our method achieves better performance than an existing bucket-based verification scheme, it is 1.6 times faster in terms of query processing time and produces verification data that is 20 times smaller.

A Method of Chinese and Thai Cross-Lingual Query Expansion Based on Comparable Corpus

  • Tang, Peili;Zhao, Jing;Yu, Zhengtao;Wang, Zhuo;Xian, Yantuan
    • Journal of Information Processing Systems
    • /
    • v.13 no.4
    • /
    • pp.805-817
    • /
    • 2017
  • Cross-lingual query expansion is usually based on the relationship among monolingual words. Bilingual comparable corpus contains relationships among bilingual words. Therefore, this paper proposes a method based on these relationships to conduct query expansion. First, the word vectors which characterize the bilingual words are trained using Chinese and Thai bilingual comparable corpus. Then, the correlation between Chinese query words and Thai words are computed based on these word vectors, followed with selecting the Thai candidate expansion terms via the correlative value. Then, multi-group Thai query expansion sentences are built by the Thai candidate expansion words based on Chinese query sentence. Finally, we can get the optimal sentence using the Chinese and Thai query expansion method, and perform the Thai query expansion. Experiment results show that the cross-lingual query expansion method we proposed can effectively improve the accuracy of Chinese and Thai cross-language information retrieval.

A Query Randomizing Technique for breaking 'Filter Bubble'

  • Joo, Sangdon;Seo, Sukyung;Yoon, Youngmi
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.12
    • /
    • pp.117-123
    • /
    • 2017
  • The personalized search algorithm is a search system that analyzes the user's IP, cookies, log data, and search history to recommend the desired information. As a result, users are isolated in the information frame recommended by the algorithm. This is called 'Filter bubble' phenomenon. Most of the personalized data can be deleted or changed by the user, but data stored in the service provider's server is difficult to access. This study suggests a way to neutralize personalization by keeping on sending random query words. This is to confuse the data accumulated in the server while performing search activities with words that are not related to the user. We have analyzed the rank change of the URL while conducting the search activity with 500 random query words once using the personalized account as the experimental group. To prove the effect, we set up a new account and set it as a control. We then searched the same set of queries with these two accounts, stored the URL data, and scored the rank variation. The URLs ranked on the upper page are weighted more than the lower-ranked URLs. At the beginning of the experiment, the difference between the scores of the two accounts was insignificant. As experiments continue, the number of random query words accumulated in the server increases and results show meaningful difference.

Evaluating the Performance Quality of Open Source Database Management Systems (오픈소스 DBMS의 성능 품질 평가)

  • Min, Meekyung
    • Journal of Korean Society for Quality Management
    • /
    • v.45 no.4
    • /
    • pp.933-942
    • /
    • 2017
  • Purpose: The purpose of this paper is to evaluate the performance quality of the open source DBMSs. Performance quality is defined as processing time for Join queries. Query processing time is measured and compared in the most widely used open source DBMSs and commercial DBMS. Methods: By varying the number of tuples of two relations to be joined, the average processing time(seconds) of a Join query in each DBMS was obtained experimentally. ANOVA and Tukey HSD test were used in order to compare the performance quality of DBMSs. Results: There was a significant difference between the performance qualities of the three DBMSs at all experimental levels where the number of tuples was 100, 1,000, 2,000, 10,000, and 50,000. As a result of the Tukey HSD test, two open source DBMSs (MariaDB, MySQL) were classified in the same group only at the tuple level of 100. The commercial DBMS (MS-SQL Server) belonged to another group. At level of more than 1,000 tuples, all three DBMSs belonged to different groups. Conclusion: Within the open source DBMS group, MariaDB showed the better performance quality except for a small number of tuples. Thus the results show that MariaDB can be the alternative to MySQL which is currently most widely used. Between open source DBMS and commercial DBMS groups, MS-SQL Server always shows the best performance quality, but the less number of tuples, the less the difference.

Research on supporting the group by clause reflecting XML data characteristics in XQuery (XQuery에서의 XML 데이터 특성을 고려한 group by 지원을 위한 질의 표현 기법에 대한 연구)

  • Lee Min-Soo;Cho Hye-Young;Oh Jung-Sun;Kim Yun-Mi;Song Soo-Kyung
    • The KIPS Transactions:PartD
    • /
    • v.13D no.4 s.107
    • /
    • pp.501-512
    • /
    • 2006
  • XML is the most popular platform-independent data expression which is used to communicate between loosely coupled heterogeneous systems such as B2B Applications or Workflow systems. The powerful query language XQuery has been developed to support diverse needs for querying XML documents. XQuery is designed to configure results from diverse data sources into a uniquely structured query result. Therefore, it became the standard for the XML query language. Although the latest XQuery supports heavy search functions including iterations, the grouping mechanism for data is too primitive and makes the query expression difficult and complex. Therefore, this work is focused on supporting the groupby clause in the query expression to process XQuery grouping. We suggest it to be a more efficient way to process grouping for restructuring and aggregation functions on XML data. We propose an XQuery EBNF that includes the groupby clause and implemented an XQuery processing system with grouping functions based on the eXist Native XML Database.

Robust Syntactic Annotation of Corpora and Memory-Based Parsing

  • Hinrichs, Erhard W.
    • Proceedings of the Korean Society for Language and Information Conference
    • /
    • 2002.02a
    • /
    • pp.1-1
    • /
    • 2002
  • This talk provides an overview of current work in my research group on the syntactic annotation of the T bingen corpus of spoken German and of the German Reference Corpus (Deutsches Referenzkorpus: DEREKO) of written texts. Morpho-syntactic and syntactic annotation as well as annotation of function-argument structure for these corpora is performed automatically by a hybrid architecture that combines robust symbolic parsing with finite-state methods ("chunk parsing" in the sense Abney) with memory-based parsing (in the sense of Daelemans). The resulting robust annotations can be used by theoretical linguists, who lire interested in large-scale, empirical data, and by computational linguists, who are in need of training material for a wide range of language technology applications. To aid retrieval of annotated trees from the treebank, a query tool VIQTORYA with a graphical user interface and a logic-based query language has been developed. VIQTORYA allows users to query the treebanks for linguistic structures at the word level, at the level of individual phrases, and at the clausal level.

  • PDF

Design and Implementation of Load Balancing Method for Efficient Spatial Query Processing in Clustering Environment (클러스터링 환경에서 효율적인 공간 질의 처리를 위한 로드 밸런싱 기법의 설계 및 구현)

  • 김종훈;이찬구;정현민;정미영;배영호
    • Journal of Korea Multimedia Society
    • /
    • v.6 no.3
    • /
    • pp.384-396
    • /
    • 2003
  • Hybrid query processing method is used for preventing server overload that is created by heavy user connection in Web GIS. In Hybrid query processing method, both server and client participate in spatial query processing. But, Hybrid query processing method is restricted in scalability of server and it can't be fundamentally solution for server overload. So, it is necessary for Web GIS to be brought in web clustering technique. In this thesis, we propose load-balancing method that uses proximity of query region. In this paper, we create tile groups that have relation each tile in same group is very close, and forward client request to the server that can have maximum rate of buffer reuse with considering characteristic of spatial query. With out load balancing method, buffet in server is optimized for exploring spatial index tree and increase rate of buffer reuse, so it can be reduced amount of disk access and increase system performance.

  • PDF