• Title/Summary/Keyword: Parallel Search

Search Result 318, Processing Time 0.037 seconds

k-NN Join Based on LSH in Big Data Environment

  • Ji, Jiaqi;Chung, Yeongjee
    • Journal of information and communication convergence engineering
    • /
    • v.16 no.2
    • /
    • pp.99-105
    • /
    • 2018
  • k-Nearest neighbor join (k-NN Join) is a computationally intensive algorithm that is designed to find k-nearest neighbors from a dataset S for every object in another dataset R. Most related studies on k-NN Join are based on single-computer operations. As the data dimensions and data volume increase, running the k-NN Join algorithm on a single computer cannot generate results quickly. To solve this scalability problem, we introduce the locality-sensitive hashing (LSH) k-NN Join algorithm implemented in Spark, an approach for high-dimensional big data. LSH is used to map similar data onto the same bucket, which can reduce the data search scope. In order to achieve parallel implementation of the algorithm on multiple computers, the Spark framework is used to accelerate the computation of distances between objects in a cluster. Results show that our proposed approach is fast and accurate for high-dimensional and big data.

Implementation of a parallel traversal scheme for O(n!) search space exploiting cost constraint (비용 제약조건을 이용한 병렬 O(n!) 서치 스페이스 탐색 기법의 구현)

  • Lee, Junghoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2010.11a
    • /
    • pp.1501-1502
    • /
    • 2010
  • DualCore 혹은 MultiCore 플랫폼의 보급에 따라 높은 시간복잡도를 갖는 응용들도 사용자의 컴퓨터나 단말에서 수행되어 다양한 서비스를 제공할 수 있게 되었다. 본 논문에서는 관광 스케줄을 효율적으로 결정하기 위한 다중목적지 방문 문제에 대해 이중 쓰레드에 기반한 서치 스페이스 탐색 알고리즘을 구현한다. 이는 Traveling Salesman Problem의 한 종류로서 O(n!) 시간 복잡도를 갖고 있으며 검색시의 독립성때문에 각 쓰레드는 병렬적으로 최적의 스케줄을 탐색할 수 있다. 또 현재까지 발견된 최적값을 기반으로 부분 경로의 비용이 이미 최적값을 넘는 경우는 하위 탐색을 제거하여 상당한 성능의 향상을 가져온다. 2.4 GHz Intel(R) Core DuoCPU와 3 GB 메모리로 구성된 플랫폼 상에서 구현된 서비스는 11개의 목적지에 대한 방문 스케줄을 생성함에 있어서 단일 쓰레드 버전은 14.196초, 이중 쓰레드 버전은 6.411초, 제약조건을 포함한 이중 쓰레드 버전은 0.14초에 최적의 스케줄을 찾아낼 수 있다.

A Study for Parallelizing Sequential Algorithms of Search Engine in Parallel Information Retrieval System (병렬 정보검색 시스템의 순차적인 검색엔진 알고리즘의 병렬화를 위한 연구)

  • Kim, Seok Young;Park, Mi-Young;Park, Hyuk-Ro;Chung, In Sang
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2007.11a
    • /
    • pp.693-696
    • /
    • 2007
  • 대규모 데이터를 효율적으로 검색하기 위한 병렬 정보검색 시스템에서는 하드웨어 확장으로 인한 병렬화로 시스템 전체의 작업 처리량을 증가시켰다. 그러나 병렬 시스템 상에서 수행되는 검색엔진의 알고리즘들은 여전히 순차적으로 수행되기 때문에, 사용자의 개별적인 질의처리 시간은 단축되지 않는다. 본 연구는 검색엔진의 병렬화를 위하여 사용자 질의처리 과정과 역색인 파일처리 과정의 순차 알고리즘들을 조사하여 병렬화의 필요성과 가능성을 평가한다. 이러한 평가는 병렬 정보검색 시스템에서 수행되는 순차 알고리즘들의 효과적이고 체계적인 병렬화를 도모하고, 보다 효율적인 병렬 정보검색 시스템의 구축을 가능하게 한다.

A Design of 4×4 Block Parallel Interpolation Motion Compensation Architecture for 4K UHD H.264/AVC Decoder (4K UHD급 H.264/AVC 복호화기를 위한 4×4 블록 병렬 보간 움직임보상기 아키텍처 설계)

  • Lee, Kyung-Ho;Kong, Jin-Hyeung
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.5
    • /
    • pp.102-111
    • /
    • 2013
  • In this paper, we proposed a $4{\times}4$ block parallel architecture of interpolation for high-performance H.264/AVC Motion Compensation in 4K UHD($3840{\times}2160$) video real time processing. To improve throughput, we design $4{\times}4$ block parallel interpolation. For supplying the $9{\times}9$ reference data for interpolation, we design 2D cache buffer which consists of the $9{\times}9$ memory arrays. We minimize redundant storage of the reference pixel by applying the Search Area Stripe Reuse scheme(SASR), and implement high-speed plane interpolator with 3-stage pipeline(Horizontal Vertical 1/2 interpolation, Diagonal 1/2 interpolation, 1/4 interpolation). The proposed architecture was simulated in 0.13um standard cell library. The maximum operation frequency is 150MHz. The gate count is 161Kgates. The proposed H.264/AVC Motion Compensation can support 4K UHD at 72 frames per second by running at 150MHz.

The Motion Estimator Implementation with Efficient Structure for Full Search Algorithm of Variable Block Size (다양한 블록 크기의 전역 탐색 알고리즘을 위한 효율적인 구조를 갖는 움직임 추정기 설계)

  • Hwang, Jong-Hee;Choe, Yoon-Sik
    • Journal of the Institute of Electronics Engineers of Korea SD
    • /
    • v.46 no.11
    • /
    • pp.66-76
    • /
    • 2009
  • The motion estimation in video encoding system occupies the biggest part. So, we require the motion estimator with efficient structure for real-time operation. And for motion estimator's implementation, it is desired to design hardware module of an exclusive use that perform the encoding process at high speed. This paper proposes motion estimation detection block(MED), 41 SADs(Sum of Absolute Difference) calculation block, minimum SAD calculation and motion vector generation block based on parallel processing. The parallel processing can reduce effectively the amount of the operation. The minimum SAD calculation and MED block uses the pre-computation technique for reducing switching activity of the input signal. It results in high-speed operation. The MED and 41 SADs calculation blocks are composed of adder tree which causes the problem of critical path. So, the structure of adder tree has changed the most commonly used ripple carry adder(RCA) with carry skip adder(CSA). It enables adder tree to operate at high speed. In addition, as we enabled to easily control key variables such as control signal of search range from the outside, the efficiency of hardware structure increased. Simulation and FPGA verification results show that the delay of MED block generating the critical path at the motion estimator is reduced about 19.89% than the conventional strukcture.

Design and Implemention of Real-time web Crawling distributed monitoring system (실시간 웹 크롤링 분산 모니터링 시스템 설계 및 구현)

  • Kim, Yeong-A;Kim, Gea-Hee;Kim, Hyun-Ju;Kim, Chang-Geun
    • Journal of Convergence for Information Technology
    • /
    • v.9 no.1
    • /
    • pp.45-53
    • /
    • 2019
  • We face problems from excessive information served with websites in this rapidly changing information era. We find little information useful and much useless and spend a lot of time to select information needed. Many websites including search engines use web crawling in order to make data updated. Web crawling is usually used to generate copies of all the pages of visited sites. Search engines index the pages for faster searching. With regard to data collection for wholesale and order information changing in realtime, the keyword-oriented web data collection is not adequate. The alternative for selective collection of web information in realtime has not been suggested. In this paper, we propose a method of collecting information of restricted web sites by using Web crawling distributed monitoring system (R-WCMS) and estimating collection time through detailed analysis of data and storing them in parallel system. Experimental results show that web site information retrieval is applied to the proposed model, reducing the time of 15-17%.

Optimal Satellite Constellation Design for Korean Navigation Satellite System (한국형 위성항법시스템을 위한 위성군집궤도 최적 설계)

  • Kim, Han Byeol;Kim, Heung Seob
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.39 no.3
    • /
    • pp.1-9
    • /
    • 2016
  • NSS (Navigation satellite system) provides the information for determining the position, velocity and time of users in real time using satellite-networking, and is classified into GNSS (Global NSS) and RNSS (Regional NSS). Although GNSS services for global users, the exactitude of provided information is dissatisfied with the degree required in modern systems such as unmanned system, autonomous navigation system for aircraft, ship and others, air-traffic control system. Especially, due to concern about the monopoly status of the countries operating it, some other countries have already considered establishing RNSS. The RNSS services for users within a specific area, however, it not only gives more precise information than those from GNSS, but also can be operated independently from the NSS of other countries. Thus, for Korean RNSS, this paper suggests the methodology to design the satellite constellation considering the regional features of Korean Peninsula. It intends to determine the orbits and the arrangement of navigation satellites for minimizing PDOP (Position dilution of precision). PGA (Parallel Genetic Algorithm) geared to solve this nonlinear optimization problem is proposed and STK (System tool kit) software is used for simulating their space flight. The PGA is composed of several GAs and iterates the process that they search the solution for a problem during the pre-specified generations, and then mutually exchange the superior solutions investigated by each GA. Numerical experiments were performed with increasing from four to seven satellites for Korean RNSS. When the RNSS was established by seven satellites, the time ratio that PDOP was measured to less than 5 (i.e. better than 'Good' level on the meaning of the PDOP value) was found to 94.3% and PDOP was always kept at 10 or less (i.e. better than 'Moderate' level).

Analysis of GPU-based Parallel Shifted Sort Algorithm by comparing with General GPU-based Tree Traversal (일반적인 GPU 트리 탐색과의 비교실험을 통한 GPU 기반 병렬 Shifted Sort 알고리즘 분석)

  • Kim, Heesu;Park, Taejung
    • Journal of Digital Contents Society
    • /
    • v.18 no.6
    • /
    • pp.1151-1156
    • /
    • 2017
  • It is common to achieve lower performance in traversing tree data structures in GPU than one expects. In this paper, we analyze the reason of lower-than-expected performance in GPU tree traversal and present that the warp divergences is caused by the branch instructions ("if${\ldots}$ else") which appear commonly in tree traversal CUDA codes. Also, we compare the parallel shifted sort algorithm which can reduce the number of warp divergences with a kd-tree CUDA implementation to show that the shifted sort algorithm can work faster than the kd-tree CUDA implementation thanks to less warp divergences. As the analysis result, the shifted sort algorithm worked about 16-fold faster than the kd-tree CUDA implementation for $2^{23}$ query points and $2^{23}$ data points in $R^3$ space. The performance gaps tend to increase in proportion to the number of query points and data points.

A Study of th Optimum of closed ${CO}_{2}$ Gas Turbine Process for Nuclear Energy Power Plant(II) - For Optimal Design of Heat Exchanger- (원자력 발전소에 대한 밀폐 ${CO}_{2}$ 가스터빈 프로세스의 최적화 연구 (II) -열교환기의 설계에 관하여 -)

  • 이찬규;이종원
    • Transactions of the Korean Society of Mechanical Engineers
    • /
    • v.14 no.1
    • /
    • pp.251-258
    • /
    • 1990
  • Optimal design of heat exchanger for closed CO$_{2}$ gas turbine plant of three processes selected from the result of cycle analysis have been discussed previously paper(I) has been carried out under specified inlet and outlet conditions. Independent variables such as number of parallel connection, tube diameter, shell side and tube side pressure loss as well as dependent variables such as shell diameter, number of tubes, number of serial connections were all characterized according to the standardization or so. Search method was used to construct a computer simulation together with the calculation of heat transfer rate by logarithmic mean temperature difference method. Strength analysis of major parts was carried to examine their dimensions satisfying heat transfer and pressure loss requirements.

A Simulation Method For Virtual Situations Through Seamless Integration Of Independent Events Via Autonomous And Independent Agents

  • Park, Jong Hee;Choi, Jun Seong
    • International Journal of Contents
    • /
    • v.14 no.3
    • /
    • pp.7-16
    • /
    • 2018
  • The extent and depth of the event plan determines the scope of pedagogical experience in situations and consequently the quality of immersive learning based on our simulated world. In contrast to planning in conventional narrative-based systems mainly pursuing dramatic interests, planning in virtual world-based pedagogical systems strive to provide realistic experiences in immersed situations. Instead of story plot comprising predetermined situations, our inter-event planning method aims at simulating diverse situations that each involve multiple events coupled via their associated agents' conditions and meaningful associations between events occurring in a background world. The specific techniques to realize our planning method include, two-phase planning based on inter-event search and intra-event decomposition (down to the animated action level); autonomous and independent agents to behave proactively with their own belief and planning capability; full-blown background world to be used as the comprehensive stage for all events to occur in; coupling events via realistic association types including deontic associations as well as conventional causality; separation of agents from event roles; temporal scheduling; and parallel and concurrent event progression mechanism. Combining all these techniques, diverse exogenous events can be derived and seamlessly (i.e., semantically meaningfully) integrated with the original event to form a wide scope of situations providing chances of abundant pedagogical experiences. For effective implementation of plan execution, we devise an execution scheme based on multiple priority queues, particularly to realize concurrent progression of many simultaneous events to simulate its corresponding reality. Specific execution mechanisms include modeling an action in terms of its component motions, adjustability of priority for agent across different events, and concurrent and parallel execution method for multiple actions and its expansion for multiple events.