• Title/Summary/Keyword: Streams Computing

Search Result 71, Processing Time 0.025 seconds

Load Shedding for Temporal Queries over Data Streams

  • Al-Kateb, Mohammed;Lee, Byung-Suk
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.4
    • /
    • pp.294-304
    • /
    • 2011
  • Enhancing continuous queries over data streams with temporal functions and predicates enriches the expressive power of those queries. While traditional continuous queries retrieve only the values of attributes, temporal continuous queries retrieve the valid time intervals of those values as well. Correctly evaluating such queries requires the coalescing of adjacent timestamps for value-equivalent tuples prior to evaluating temporal functions and predicates. For many stream applications, the available computing resources may be too limited to produce exact query results. These limitations are commonly addressed through load shedding and produce approximated query results. There have been many load shedding mechanisms proposed so far, but for temporal continuous queries, the presence of coalescing makes theses existing methods unsuitable. In this paper, we propose a new accuracy metric and load shedding algorithm that are suitable for temporal query processing when memory is insufficient. The accuracy metric uses a combination of the Jaccard coefficient to measure the accuracy of attribute values and $\mathcal{PQI}$ interval orders to measure the accuracy of the valid time intervals in the approximate query result. The algorithm employs a greedy strategy combining two objectives reflecting the two accuracy metrics (i.e., value and interval). In the performance study, the proposed greedy algorithm outperforms a conventional random load shedding algorithm by up to an order of magnitude in its achieved accuracy.

Transformation of Continuous Aggregation Join Queries over Data Streams

  • Tran, Tri Minh;Lee, Byung-Suk
    • Journal of Computing Science and Engineering
    • /
    • v.3 no.1
    • /
    • pp.27-58
    • /
    • 2009
  • Aggregation join queries are an important class of queries over data streams. These queries involve both join and aggregation operations, with window-based joins followed by an aggregation on the join output. All existing research address join query optimization and aggregation query optimization as separate problems. We observe that, by putting them within the same scope of query optimization, more efficient query execution plans are possible through more versatile query transformations. The enabling idea is to perform aggregation before join so that the join execution time may be reduced. There has been some research done on such query transformations in relational databases, but none has been done in data streams. Doing it in data streams brings new challenges due to the incremental and continuous arrival of tuples. These challenges are addressed in this paper. Specifically, we first present a query processing model geared to facilitate query transformations and propose a query transformation rule specialized to work with streams. The rule is simple and yet covers all possible cases of transformation. Then we present a generic query processing algorithm that works with all alternative query execution plans possible with the transformation, and develop the cost formulas of the query execution plans. Based on the processing algorithm, we validate the rule theoretically by proving the equivalence of query execution plans. Finally, through extensive experiments, we validate the cost formulas and study the performances of alternative query execution plans.

A Review of Window Query Processing for Data Streams

  • Kim, Hyeon Gyu;Kim, Myoung Ho
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.4
    • /
    • pp.220-230
    • /
    • 2013
  • In recent years, progress in hardware technology has resulted in the possibility of monitoring many events in real time. The volume of incoming data may be so large, that monitoring all individual data might be intractable. Revisiting any particular record can also be impossible in this environment. Therefore, many database schemes, such as aggregation, join, frequent pattern mining, and indexing, become more challenging in this context. This paper surveys the previous efforts to resolve these issues in processing data streams. The emphasis is on specifying and processing sliding window queries, which are supported in many stream processing engines. We also review the related work on stream query processing, including synopsis structures, plan sharing, operator scheduling, load shedding, and disorder control.

A Design and Implementation of PULL-based Data Transfer Method on Distributed Multimedia Streams (분산 멀티미디어 스트림상의 PULL 방식 데이터 전송기법 설계 및 구현)

  • Kim, Jeong-Hun;Lee, Seung-Hyun;Park, Yong-Hee;Lee, Jo-Won;Lim, Young-Hwan
    • Annual Conference of KIPS
    • /
    • 2000.04a
    • /
    • pp.1178-1183
    • /
    • 2000
  • 기존의 MuX 시스템은 분산환경에서 멀티미디어 데이터를 전송하기 위해서 PUSH 방식의 데이터 전송기법만을 제공한다. PUSH 방식의 데이터 전송기법은 자료 발생매체 중심의 데이터 전송기법으로 멀티미디어 데이터가 발생할 때 바로 자료 사용 매체로 전송하는 기법이다. PUSH 방식 데이터 전송 기법은 네트웍 대역폭이 크고 시스템 자원이 풍부한 경우 간단히 연출 QoS를 만족시킬 수 있다. 그러나 네트웍의 대역폭이 좁은 시스템에서는 PUSH 방식 데이터 전송기법으로 연출 QoS를 만족시키는데 한계가 있다. 이러한 문제를 해결하기 위해 PULL 방식 데이터 전송기법을 제안하였다. PULL 방식 데이터 전송기법은 자료 사용 매체 중심의 데이터 전송 기법으로 자료 사용 매체의 요구시점에 요구한 만큼의 데이터를 발생시켜 그 매체로 전송하는 방법이다.

  • PDF

Finding high utility old itemsets in web-click streams (웹 클릭 스트림에서 고유용 과거 정보 탐색)

  • Chang, Joong-Hyuk
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.17 no.4
    • /
    • pp.521-528
    • /
    • 2016
  • Web-based services are used widely in many computer application fields due to the increasing use of PCs and mobile devices. Accordingly, topics on the analysis of access logs generated in the application fields have been researched actively to support personalized services in the field, and analyzing techniques based on the weight differentiation of information in access logs have been proposed. This paper outlines an analysis technique for web-click streams, which is useful for finding high utility old item sets in web-click streams, whose data elements are generated at a rapid rate. Using the technique, interesting information can be found, which is difficult to find in conventional techniques for analyzing web-click streams and is used effectively in target marketing. The proposed technique can be adapted widely to analyzing the data generated in a range of computing application fields, such as IoT environments, bio-informatics, etc., which generated data as a form of data streams.

GeoSensor Data Stream Processing System for u-GIS Computing (u-GIS 컴퓨팅을 위한 GeoSensor 데이터 스트림 처리 시스템)

  • Chung, Weon-Il;Shin, Soong-Sun;Back, Sung-Ha;Lee, Yeon;Lee, Dong-Wook;Kim, Kyung-Bae;Lee, Chung-Ho;Kim, Ju-Wan;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.1
    • /
    • pp.9-16
    • /
    • 2009
  • In ubiquitous spatial computing environments, GeoSensor generates sensor data streams including spatial information as well as various conventional sensor data from RFID, WSN, Web CAM, Digital Camera, CCTV, and Telematics units. This GeoSensor enables the revitalization of various ubiquitous USN technologies and services on geographic information. In order to service the u-GIS applications based on GeoSensors, it is indispensable to efficiently process sensor data streams from GeoSensors of a wide area. In this paper, we propose a GeoSensor data stream processing system for u-GIS computing over real-time stream data from GeoSensors with geographic information. The proposed system provides efficient gathering, storing, and continuous query processing of GeoSensor data stream, and also makes it possible to develop diverse u-GIS applications meet each user requirements effectively.

  • PDF

A Design and Implementation of Distributed Mail Server for Multimedia Mail on Distributed Multimedia Streams (분산 멀티미디어 스트림 상의 멀티 미디어 메일을 위한 분산 메일 서버 설계 및 구현)

  • Park, Yong-Hee;Chon, Sung-Mi;Ko, Hee-Seon;Lim, Young-Hwan
    • Annual Conference of KIPS
    • /
    • 2001.04a
    • /
    • pp.111-114
    • /
    • 2001
  • 기존의 멀티미디어 메일은 대용량 메일이라 개인 사용자에게 그 부담이 크지 않을 수가 없었다. 본 연구에서는 멀티미디어 스트림엔진인 Essence 를 이용하여 동영상 멀티미디어 메일을 전송하는데 필요한 Essence 내에 데이터베이스를 지원하기 위한 DBMedium 개발과 데이터베이스 라이브러리를 개발하기 위한 일련의 과정을 제안하고 있다.

  • PDF

A Study on Stream Reactor for the event processing of multimedia streams in context-based (컨텍스트 기반에서의 멀티미디어 스트림의 사건처리를 위한 Stream Reactor연구)

  • Park, Yong-Hee;Kang, Tae-Sung;Lim, Young-Hwan
    • Annual Conference of KIPS
    • /
    • 2000.04a
    • /
    • pp.1166-1171
    • /
    • 2000
  • 기존의 멀티미디어 연구의 실현에 있어 가장 큰 문제라 할 수 있던 성능의 문제가 하드웨어의 급속한 발달로 해결되어 감에 따라 멀티미디어 및 제반 관련기술도 함께 발전되었으며 이에 기반한 multimedia stream에서의 event를 검출하기 위한 다양한 연구들이 진행되어 왔다. 그러나 지금까지의 연구는 주로 전송 및 저장, 검색에 집중되어 연구되어 왔으며 영상인식 등의 Vision관련 연구에서는 멀티미디어 스트리밍 기술과의 연동을 고려하지 않은 연구를 수행함에 따라 검출 가능한 event가 있다고 하더라도 응용영역에 종속적인 인테페이스만을 고려함에 따라 사용자가 이를 기술(記述, description)하거나, 사용자에게 검출 가능한 event를 제시하기 위해 일반화된 방법이 제시되어 있지 않았다. 본 연구에서는 사용자가 검출을 원하는 event를 기술하는 방법과, 시스템에서 검출 가능한 event를 제시하기 위한 방법을 제안하고, 제시되는 방법이 응용영역에 독립적이기 위해 요구되는 사항들과 객체 단위인 이벤트/행위와 처리기 사이의 인터페이스에 관하여 정의한 후 기본적인 동작방식을 제안한다.

  • PDF

The Design and Implementation of Automatic Control System of Living Environment Based on Ubiquitous Sensor Network (유비쿼터스 센서 네트워크 기반의 생활환경 자동제어 시스템 설계 및 구현)

  • Yun, Ji-Hoon;Moon, Seung-Jin
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.14 no.1
    • /
    • pp.1-6
    • /
    • 2008
  • The ubiquitous sensor network technique is widely applied to variety of information fields such as home automations, logistics, traffic controls, public administrations, health and environment monitoring and etc. It is particularly useful in the areas where energy consumption is minimal and where continuous monitoring of the surrounding environments, which generates streams of data, are required. In this study, we have designed and implemented a living environment automatic control system which collects the streams of temperature, humidity, light and noise data of a simulated house setting in real-time fashion, then controls the home environment based on the collected data according to the users favorites. In order to differentiate the proposed system from the currently existing similar system, we have demonstrated not only the feasibility of collecting data using sensor network in the controlled environment but also the ability to control the various household equipments through wireless communications.

Applying Formal Methods to Modeling and Analysis of Real-time Data Streams

  • Kapitanova, Krasimira;Wei, Yuan;Kang, Woo-Chul;Son, Sang-H.
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.1
    • /
    • pp.85-110
    • /
    • 2011
  • Achieving situation awareness is especially challenging for real-time data stream applications because they i) operate on continuous unbounded streams of data, and ii) have inherent realtime requirements. In this paper we showed how formal data stream modeling and analysis can be used to better understand stream behavior, evaluate query costs, and improve application performance. We used MEDAL, a formal specification language based on Petri nets, to model the data stream queries and the quality-of-service management mechanisms of RT-STREAM, a prototype system for data stream management. MEDAL's ability to combine query logic and data admission control in one model allows us to design a single comprehensive model of the system. This model can be used to perform a large set of analyses to help improve the application's performance and quality of service.