• The Transactions of the Korea Information Processing Society
• /
• v.6 no.2
• /
• pp.315-325
• /
• 1999

Indexing techniques for distributed resources have much effect on an information service system based on distributed environment like digital library. There is a centralized indexing technique, a distributed technique, and a mixed technique for distributed indexing techniques. In this paper, we propose new distributed indexing technique using EIF(extended Inverted File) structure that mix the centralized technique and t도 distributed technique. And we propose management techniques using EIF structure and retrieval technique using EIF structure. This distributed indexing technique proposed is able to fast process retrieval request and reduce network overload and select servers relevant to query terms. This paper investigated performance of a proposed distributed indexing technique.

• International Journal of Internet, Broadcasting and Communication
• /
• v.4 no.1
• /
• pp.13-17
• /
• 2012

Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

• The KIPS Transactions:PartD
• /
• v.19D no.2
• /
• pp.151-160
• /
• 2012

Most of the distributed high-dimensional indexing structures provide a reasonable search performance especially when the dataset is uniformly distributed. However, in case when the dataset is clustered or skewed, the search performances gradually degrade as compared with the uniformly distributed dataset. We propose a method of improving the k-nearest neighbor search performance for the distributed vector approximation-tree based on the strongly clustered or skewed dataset. The basic idea is to compute volumes of the leaf nodes on the top-tree of a distributed vector approximation-tree and to assign different number of bits to them in order to assure an identification performance of vector approximation. In other words, it can be done by assigning more bits to the high-density clusters. We conducted experiments to compare the search performance with the distributed hybrid spill-tree and distributed vector approximation-tree by using the synthetic and real data sets. The experimental results show that our proposed scheme provides consistent results with significant performance improvements of the distributed vector approximation-tree for strongly clustered or skewed datasets.

• Journal of KIISE:Databases
• /
• v.37 no.5
• /
• pp.228-237
• /
• 2010

Although conventional index structures provide various nearest-neighbor search algorithms for high-dimensional data, there are additional requirements to increase search performances as well as to support index scalability for large scale data. To support these requirements, we propose a distributed high-dimensional indexing structure based on cluster systems, called a Distributed Vector Approximation-tree (DVA-tree), which is a two-level structure consisting of a hybrid spill-tree and VA-files. We also describe the algorithms used for constructing the DVA-tree over multiple machines and performing distributed k-nearest neighbors (NN) searches. To evaluate the performance of the DVA-tree, we conduct an experimental study using both real and synthetic datasets. The results show that our proposed method contributes to significant performance advantages over existing index structures on difference kinds of datasets.

• Journal of KIISE:Databases
• /
• v.29 no.2
• /
• pp.128-137
• /
• 2002

In this paper, we propose the LPC+-file for efficient indexing of high-dimensional image data. With the proliferation of multimedia data, there Is an increasing need to support the indexing and retrieval of high-dimensional image data. Recently, the LPC-file (5) that based on vector approximation has been developed for indexing high-dimensional data. The LPC-file gives good performance especially when the dataset is uniformly distributed. However, compared with for the uniformly distributed dataset, its performance degrades when the dataset is clustered. We improve the performance of the LPC-file for the strongly clustered image dataset. The basic idea is to adaptively partition the data space to find subspaces with high-density clusters and to assign more bits to them than others to increase the discriminatory power of the approximation of vectors. The total number of bits used to represent vector approximations is rather less than that of the LPC-file since the partitioned cells in the LPC+-file share the bits. An empirical evaluation shows that the LPC+-file results in significant performance improvements for real image data sets which are strongly clustered.

• Spatial Information Research
• /
• v.19 no.5
• /
• pp.63-74
• /
• 2011

Recently, there has been much interest in the spatial query which energy-efficiently acquires sensor readings from sensor nodes inside specified geographical area of interests. The centralized approach which performs the spatial query at a server after acquiring all sensor readings, though simple, it incurs high wireless transmission cost in accessing all sensor nodes. In order to remove the high wireless transmission cost, various in-network spatial indexing schemes have been proposed. They have focused on reducing the transmission cost by performing distributed spatial filtering on sensor nodes. However, these in-network spatial indexing schemes have a problem which cannot optimize both the spatial filtering and the wireless routing among sensor nodes, because these schemes have been developed by simply applying the existing spatial indexing schemes into the in-network environment. Therefore, we propose a new distributed spatial indexing scheme of the GR-tree. The GR-tree which form s a MBR-based tree structure, can reduce the wireless transmission cost by optimizing both the efficient spatial filtering and the wireless routing. Finally, we compare the existing spatial indexing scheme through extensive experiments and clarify our approach's distinguished features.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.9 no.5
• /
• pp.1807-1824
• /
• 2015

As the technologies advance, mobile peer-to-peer (MP2P) networks or systems become one of the major ways to share resources and information. On such a system, the information retrieval (IR), including the development of scalable infrastructures for indexing, becomes more complicated due to a huge increase on the amount of information and rapid information change. To keep the systems on MP2P networks more reliable and consistent, the index structures need to be updated frequently. For a semantic IR system, the index structure is even more complicated than a classic IR system and generally has higher update cost. The most well-known indexing technique used in semantic IR systems is Latent Semantic Indexing (LSI), of which the index structure is generated by singular value decomposition (SVD). Although LSI performs well, updating the index structure is not easy and time consuming. In an MP2P environment, which is fully distributed and dynamic, the update becomes more challenging. In this work, we consider how to update the sematic index generated by LSI and keep the index consistent in the whole MP2P network. The proposed Concept Space Update (CSU) protocol, based on distributed 2-Phase locking strategy, can effectively achieve the objectives in terms of two measurements: coverage speed and update cost. Using the proposed effective synchronization mechanism with the efficient updates on the SVD, re-computing the whole index on the P2P overlay can be avoided and the consistency can be achieved. Simulated experiments are also performed to validate our analysis on the proposed CSU protocol. The experimental results indicate that CSU is effective on updating the concept space with LSI/SVD index structure in MP2P semantic IR systems.

• Journal of Platform Technology
• /
• v.11 no.6
• /
• pp.89-105
• /
• 2023

With the development of information technology, most of the information has been converted into digital information, leading to the Big Data era. The demand for search platform has increased to enhance accessibility and usability of information in the databases. Big data search software platforms consist of two main components: (1) an indexing component to generate and store data indices for a fast and efficient data search and (2) a searching component to look up the given data fast. As an amount of data has explosively increased, data indexing performance has become a key performance bottleneck of big data search platforms. Though many companies adopted big data search platforms, relatively little research has been made to improve indexing performance. This research study employs Elasticsearch platform, one of the most famous enterprise big data search platforms, and builds physical clusters of 3 nodes to investigate optimal indexing performance configurations. Our comprehensive experiments and studies demonstrate that the proposed optimal Elasticsearch configuration achieves high indexing performance by an average of 3.13 times.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.05a
• /
• pp.45-48
• /
• 2013

In this paper, we proposed an Indexing Structure System (ISS) based on Apache Hadoop in Wireless Sensor Network (WSN). Nowadays sensors data continuously keep growing that need to control. Data constantly update in order to provide the newest information to users. While data keep growing, data retrieving and storing are face some challenges. So by using the ISS, we can maximize processing quality and minimize data retrieving time. In order to design ISS, Indexing Types have to be defined depend on each sensor type. After identifying, each sensor goes through the Indexing Structure Processing (ISP) in order to be indexed. After ISP, indexed data are streaming and storing in Hadoop Distributed File System (HDFS) across a number of separate machines. Indexed data are split and run by MapReduce tasks. Data are sorted and grouped depend on sensor data object categories. Thus, while users send the requests, all the queries will be filter from sensor data object and managing the task by MapReduce processing framework.

• Journal of Communications and Networks
• /
• v.13 no.4
• /
• pp.400-407
• /
• 2011

Various air indexing and data scheduling schemes for wireless broadcast of spatial data have been developed for energy efficient query processing. The existing schemes are not effective when the clients' data access patterns are skewed to some items. It is because the schemes are based on flat broadcast that does not take the popularity of the data items into consideration. In this paper, thus, we propose a data scheduling scheme letting the popular items appear more frequently on the channel, and grid-based distributed index for non-flat broadcast (GDIN) for window query processing. The proposed GDIN allows quick and energy efficient processing of window query, matching the clients' linear channel access pattern and letting the clients access only the queried data items. The simulation results show that the proposed GDIN significantly outperforms the existing schemes in terms of access time, tuning time, and energy efficiency.