• Title/Summary/Keyword: Massive Data Processing

Search Result 231, Processing Time 0.024 seconds

Fire detection in video surveillance and monitoring system using Hidden Markov Models (영상감시시스템에서 은닉마코프모델을 이용한 불검출 방법)

  • Zhu, Teng;Kim, Jeong-Hyun;Kang, Dong-Joong;Kim, Min-Sung;Lee, Ju-Seoup
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2009.04a
    • /
    • pp.35-38
    • /
    • 2009
  • The paper presents an effective method to detect fire in video surveillance and monitoring system. The main contribution of this work is that we successfully use the Hidden Markov Models in the process of detecting the fire with a few preprocessing steps. First, the moving pixels detected from image difference, the color values obtained from the fire flames, and their pixels clustering are applied to obtain the image regions labeled as fire candidates; secondly, utilizing massive training data, including fire videos and non-fire videos, creates the Hidden Markov Models of fire and non-fire, which are used to make the final decision that whether the frame of the real-time video has fire or not in both temporal and spatial analysis. Experimental results demonstrate that it is not only robust but also has a very low false alarm rate, furthermore, on the ground that the HMM training which takes up the most time of our whole procedure is off-line calculated, the real-time detection and alarm can be well implemented when compared with the other existing methods.

Rating Individual Food Items of Restaurant Menu based on Online Customer Reviews using Text Mining Technique (신뢰성있는 온라인 고객 리뷰 텍스트 마이닝 기반 식당 개별 음식 아이템 평가)

  • Syed, Muzamil Hussain;Chung, Sun-Tae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2020.05a
    • /
    • pp.389-392
    • /
    • 2020
  • The growth in social media, blogs and restaurant listing directories have led to increasing customer reviews about restaurants, their quality of food items and services available on the internet. These user reviews offer a massive amount of valuable information that can be used for various decision-making purposes. Currently, most food recommendation sites provide recommendation scores about restaurants rather than food items of the restaurant and the provided recommendation scores may be biased since they are calculated only from user reviews listed only in their sites. Usually, people wants a reliable recommendation about foods, not restaurant. In this paper, we present a reliable Korean food items rating method; we first extract food items by applying NER technique to restaurant reviews collected from many Korean restaurant recommendation web sites, blogs and web data. Then, we apply lexicon-based sentiment analysis on collected user reviews and predict people's opinions as sentiment polarity scores (+1 for positive; -1 for negative; 0 for neutral). Finally, by taking average of all calculated polarity scores about a food item, we obtain a rating to individual menu items of the restaurant. The proposed food item rating is more reliable since it does not depend on reviews of only one site.

Implementation of Pixel Subword Parallel Processing Instructions for Embedded Parallel Processors (임베디드 병렬 프로세서를 위한 픽셀 서브워드 병렬처리 명령어 구현)

  • Jung, Yong-Bum;Kim, Jong-Myon
    • The KIPS Transactions:PartA
    • /
    • v.18A no.3
    • /
    • pp.99-108
    • /
    • 2011
  • Processor technology is currently continued to parallel processing techniques, not by only increasing clock frequency of a single processor due to the high technology cost and power consumption. In this paper, a SIMD (Single Instruction Multiple Data) based parallel processor is introduced that efficiently processes massive data inherent in multimedia. In addition, this paper proposes pixel subword parallel processing instructions for the SIMD parallel processor architecture that efficiently operate on the image and video pixels. The proposed pixel subword parallel processing instructions store and process four 8-bit pixels on the partitioned four 12-bit registers in a 48-bit datapath architecture. This solves the overflow problem inherent in existing multimedia extensions and reduces the use of many packing/unpacking instructions. Experimental results using the same SIMD-based parallel processor architecture indicate that the proposed pixel subword parallel processing instructions achieve a speedup of $2.3{\times}$ over the baseline SIMD array performance. This is in contrast to MMX-type instructions (a representative Intel multimedia extension), which achieve a speedup of only $1.4{\times}$ over the same baseline SIMD array performance. In addition, the proposed instructions achieve $2.5{\times}$ better energy efficiency than the baseline program, while MMX-type instructions achieve only $1.8{\times}$ better energy efficiency than the baseline program.

Development of Network based Gravity and Magnetic data Processing System (네트워크에 기반한 중력.자력 자료의 처리기술 개발 연구)

  • Kwon, Byung-Doo;Lee, Heui-Soon;Oh, Seok-Hoon;Chung, Ho-Joon;Rim, Hyoung-Rae
    • Journal of the Korean Geophysical Society
    • /
    • v.3 no.4
    • /
    • pp.235-244
    • /
    • 2000
  • We studied basic ideas of a network based Gravity/Magnetic data processing server/client system which provides functions of data processing, forward modeling, inversion and data process on Data Base. This Java technology was used to provide facilities, socket communication and JDBC(Java Database Connectivity) technology to produce an effective and practical client application. The server computers are linked by network to process the MPI parallelized computing. This can provide useful devices of the geophysical process and modeling that usually require massive computing performance and time. Since this system can be accessed by lots of users, it can provides the consistent and confident results through the verified processing programs. This system also makes it possible to get results and outputs through internet when their local machines are connected to the network. It can help many users who want to omit the jobs of system administration and to process data during their field works.

  • PDF

Important Facility Guard System Using Edge Computing for LiDAR (LiDAR용 엣지 컴퓨팅을 활용한 중요시설 경계 시스템)

  • Jo, Eun-Kyung;Lee, Eun-Seok;Shin, Byeong-Seok
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.11 no.10
    • /
    • pp.345-352
    • /
    • 2022
  • Recent LiDAR(Light Detection And Ranging) sensor is used for scanning object around in real-time. This sensor can detect movement of the object and how it has changed. As the production cost of the sensors has been decreased, LiDAR begins to be used for various industries such as facility guard, smart city and self-driving car. However, LiDAR has a large input data size due to its real-time scanning process. So another way for processing a large amount of data are needed in LiDAR system because it can cause a bottleneck. This paper proposes edge computing to compress massive point cloud for processing quickly. Since laser's reflection range of LiDAR sensor is limited, multiple LiDAR should be used to scan a large area. In this reason multiple LiDAR sensor's data should be processed at once to detect or recognize object in real-time. Edge computer compress point cloud efficiently to accelerate data processing and decompress every data in the main cloud in real-time. In this way user can control LiDAR sensor in the main system without any bottleneck. The system we suggest solves the bottleneck which was problem on the cloud based method by applying edge computing service.

An Integrated Method of Iterative and Incremental Requirement Analysis for Large-Scale Systems (시스템 요구사항 분석을 위한 순환적-점진적 복합 분석방법)

  • Park, Jisung;Lee, Jaeho
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.4
    • /
    • pp.193-202
    • /
    • 2017
  • Development of Intelligent Systems involves effective integration of large-scaled knowledge processing and understanding, human-machine interaction, and intelligent services. Especially, in our project for development of a self-growing knowledge-based system with inference methodologies utilizing the big data technology, we are building a platform called WiseKB as the central knowledge base for storing massive amount of knowledge and enabling question-answering by inferences. WiseKB thus requires an effective methodology to analyze diverse requirements convoluted with the integration of various components of knowledge representation, resource management, knowledge storing, complex hybrid inference, and knowledge learning, In this paper, we propose an integrated requirement analysis method that blends the traditional sequential method and the iterative-incremental method to achieve an efficient requirement analysis for large-scale systems.

Design and Implementation of a Multi-level VOD Server System (복합 다단계 주문형 비디오 서버의 설계 및 구현)

  • Suh, Duk-Rok;Gang, Dae-Hyeok;Kim, Su-Jeong;Lee, Won-Seok;Lee, Jeong-Su
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.3
    • /
    • pp.685-697
    • /
    • 1997
  • A Viedo-on-Demand(VOD) service is a multimedia service that is realized by the rapid advance of computer and data communication techmologies. Basically, a VOD system is composed of a server and a number fo clients.The server stores and manages a large amount of digital moving picture data. Each slient sends an on-line requwet to the server and receives data for real-time displying. Many researches are performed on a VOD server using hard disks for the permanent video fata storage. However, a hard disk is less reliable and requires more storage cost than a massive storage device. Due to these resasons, a multi-level VOD system is proposed for using teh jukeboxes of optical disks as the permanent video storage,device. In this paper, we prpose the necessary software modules and protocols between the server and its clients for the implementation of the multi-level VOD server.

  • PDF

A Sliding Window Technique for Open Data Mining over Data Streams (개방 데이터 마이닝에 효율적인 이동 윈도우 기법)

  • Chang Joong-Hyuk;Lee Won-Suk
    • The KIPS Transactions:PartD
    • /
    • v.12D no.3 s.99
    • /
    • pp.335-344
    • /
    • 2005
  • Recently open data mining methods focusing on a data stream that is a massive unbounded sequence of data elements continuously generated at a rapid rate are proposed actively. Knowledge embedded in a data stream is likely to be changed over time. Therefore, identifying the recent change of the knowledge quickly can provide valuable information for the analysis of the data stream. This paper proposes a sliding window technique for finding recently frequent itemsets, which is applied efficiently in open data mining. In the proposed technique, its memory usage is kept in a small space by delayed-insertion and pruning operations, and its mining result can be found in a short time since the data elements within its target range are not traversed repeatedly. Moreover, the proposed technique focused in the recent data elements, so that it can catch out the recent change of the data stream.

Developing dirty data cleansing service between SOA-based services (SOA 기반 서비스 사이의 오류 데이터 정제 서비스 개발)

  • Ji, Eun-Mi;Choi, Byoung-Ju;Lee, Jung-Won
    • The KIPS Transactions:PartD
    • /
    • v.14D no.7
    • /
    • pp.829-840
    • /
    • 2007
  • Dirty Data Cleansing technique so far have aimed to integrate large amount of data from various sources and manage data quality resided in DB so that it enables to extract meaningful information. Prompt response to varying environment is required in order to persistently survive in rapidly changing business environment and the age of limitless competition. As system requirement is recently getting complexed, Service Oriented Architecture is proliferated for the purpose of integration and implementation of massive distributed system. Therefore, SOA necessarily needs Data Exchange among services through Data Cleansing Technique. In this paper, we executed quality management of XML data which is transmitted through events between services while they are integrated as a sole system. As a result, we developed Dirty Data Cleansing Service based on SOA as focusing on data cleansing between interactive services rather than cleansing based on detection of data error in DB already integrated.

Distributed Edge Computing for DNA-Based Intelligent Services and Applications: A Review (딥러닝을 사용하는 IoT빅데이터 인프라에 필요한 DNA 기술을 위한 분산 엣지 컴퓨팅기술 리뷰)

  • Alemayehu, Temesgen Seyoum;Cho, We-Duke
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.12
    • /
    • pp.291-306
    • /
    • 2020
  • Nowadays, Data-Network-AI (DNA)-based intelligent services and applications have become a reality to provide a new dimension of services that improve the quality of life and productivity of businesses. Artificial intelligence (AI) can enhance the value of IoT data (data collected by IoT devices). The internet of things (IoT) promotes the learning and intelligence capability of AI. To extract insights from massive volume IoT data in real-time using deep learning, processing capability needs to happen in the IoT end devices where data is generated. However, deep learning requires a significant number of computational resources that may not be available at the IoT end devices. Such problems have been addressed by transporting bulks of data from the IoT end devices to the cloud datacenters for processing. But transferring IoT big data to the cloud incurs prohibitively high transmission delay and privacy issues which are a major concern. Edge computing, where distributed computing nodes are placed close to the IoT end devices, is a viable solution to meet the high computation and low-latency requirements and to preserve the privacy of users. This paper provides a comprehensive review of the current state of leveraging deep learning within edge computing to unleash the potential of IoT big data generated from IoT end devices. We believe that the revision will have a contribution to the development of DNA-based intelligent services and applications. It describes the different distributed training and inference architectures of deep learning models across multiple nodes of the edge computing platform. It also provides the different privacy-preserving approaches of deep learning on the edge computing environment and the various application domains where deep learning on the network edge can be useful. Finally, it discusses open issues and challenges leveraging deep learning within edge computing.