• 제목/요약/키워드: IoT Big data

검색결과 406건 처리시간 0.03초

Design of Anomaly Detection System Based on Big Data in Internet of Things (빅데이터 기반의 IoT 이상 장애 탐지 시스템 설계)

  • Na, Sung Il;Kim, Hyoung Joong
    • Journal of Digital Contents Society
    • /
    • 제19권2호
    • /
    • pp.377-383
    • /
    • 2018
  • Internet of Things (IoT) is producing various data as the smart environment comes. The IoT data collection is used as important data to judge systems's status. Therefore, it is important to monitor the anomaly state of the sensor in real-time and to detect anomaly data. However, it is necessary to convert the IoT data into a normalized data structure for anomaly detection because of the variety of data structures and protocols. Thus, we can expect a good quality effect such as accurate analysis data quality and service quality. In this paper, we propose an anomaly detection system based on big data from collected sensor data. The proposed system is applied to ensure anomaly detection and keep data quality. In addition, we applied the machine learning model of support vector machine using anomaly detection based on time-series data. As a result, machine learning using preprocessed data was able to accurately detect and predict anomaly.

Impact of Road Traffic Characteristics on Environmental Factors Using IoT Urban Big Data (IoT 도시빅데이터를 활용한 도로교통특성과 유해환경요인 간 영향관계 분석)

  • Park, Byeong hun;Yoo, Dayoung;Park, Dongjoo;Hong, Jungyeol
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • 제20권5호
    • /
    • pp.130-145
    • /
    • 2021
  • As part of the Smart Seoul policy, the importance of using big urban data is being highlighted. Furthermore interest in the impact of transportation-related urban environmental factors such as PM10 and noise on citizen's quality of life is steadily increasing. This study established the integrated DB by matching IoT big data with transportation data, including traffic volume and speed in the microscopic Spatio-temporal scope. This data analyzed the impact of a spatial unit in the road-effect zone on environmental risk level. In addition, spatial units with similar characteristics of road traffic and environmental factors were clustered. The results of this study can provide the basis for systematically establishing environmental risk management of urban spatial units such as PM10 or PM2.5 hot-spot and noise hot-spot.

System Design for Real-Time Data Transmission in Web-based Open IoT System (웹 기반 개방형 IoT 환경에서 실시간 데이터 전송을 위한 시스템 설계)

  • Phyo, Gyung-soo;Park, Jin-tae;Moon, Il-young
    • Journal of Advanced Navigation Technology
    • /
    • 제20권6호
    • /
    • pp.562-567
    • /
    • 2016
  • IoT is attracting attention as the development of the Internet and the spread of smart devices are rapidly increasing worldwide. As IoT is integrated into everyday life, the market is getting bigger. So, experts predict that IoT devices will grow to more than one trillion in a decade. Techniques related to IoT are also being developed steadily, and studies are underway to develop IoT in various fields. However, vendors launching IoT services do not interact with data from other platforms. Therefore, it is limited to growing into a big market by facing the obstacle called the silo phenomenon. To solve this problem, web technology attracts attention. Web technology can interact with data regardless of platform, and it can not only develop various services using the data, but also reduce unnecessary costs for developers. In this paper, we have studied a web - based open IoT system that can transmit data independently in real time to the IoT platform.

Real-time IoT Big Data Analysis Platform Requirements (실시간 IoT Big Data 분석 플랫폼 요건)

  • Kang, Sun-Kyoung;Lee, Hyun-Chang;Shin, Seong-Yoon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 한국정보통신학회 2017년도 춘계학술대회
    • /
    • pp.165-166
    • /
    • 2017
  • It is demanding to receive information of data in real time anywhere and analyze it with meaningful data. Research on the platform for such analysis is actively underway. In this paper, we try to find out what are important factors in solving the problems of collecting and analyzing IoT data in real time. How much better than existing data collection methods and analytical methods can be the basis for judging the value of the data. It is important to accurately collect and store data more quickly and quickly from many sensors in real time in real time, and analytical methods that can derive values from the stored data. Therefore, an important requirement of the analysis platform in the IoT environment is to process large amount of data in real time and to centralize and manage it.

  • PDF

Outlier Detection Based on MapReduce for Analyzing Big Data (대용량 데이터 분석을 위한 맵리듀스 기반의 이상치 탐지)

  • Hong, Yejin;Na, Eunhee;Jung, Yonghwan;Kim, Yangwoo
    • Journal of Internet Computing and Services
    • /
    • 제18권1호
    • /
    • pp.27-35
    • /
    • 2017
  • In near future, IoT data is expected to be a major portion of Big Data. Moreover, sensor data is expected to be major portion of IoT data, and its' research is actively carried out currently. However, processed results may not be trusted and used if outlier data is included in the processing of sensor data. Therefore, method for detection and deletion of those outlier data before processing is studied in this paper. Moreover, we used Spark which is memory based distributed processing environment for fast processing of big sensor data. The detection and deletion of outlier data consist of four stages, and each stage is implemented with Mapper and Reducer operation. The proposed method is compared in three different processing environments, and it is expected that the outlier detection and deletion performance is best in the distributed Spark environment as data volume is increasing.

Application cases of IoT using big data and its' direction of improvement. (빅데이터를 이용한 IoT 활용사례와 발전방향)

  • Cho, Young-Ju;Kim, Jin-Hyuk;So, Yoon-Jeong;No, Chang-Hee;Kwon, Soon-Pil
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 한국컴퓨터정보학회 2017년도 제55차 동계학술대회논문집 25권1호
    • /
    • pp.67-70
    • /
    • 2017
  • 빅데이터는 인터넷의 발달로 인하여 규모를 가늠할 수 없을 정도로 많은 양의 정보가 생산되어지는 데이터를 말한다. 빅데이터를 활용하는 다양한 사례 중 기업에서 활용된 사례로 사람들의 구매데이터를 분석하여 개인의 구매 취향을 분석한 뒤 구매자가 원하는 제품을 빠르게 안내하는 시스템이 있으며, 정부에서 빅데이터를 활용된 사례로는 사회복지 자금이 대상자들에게 제대로 지급되고 있는지 판단하여 부정수급자를 원천봉쇄 하는 시스템이 있다. 이처럼 빅테이터가 인간의 삶에 긍정적인 영향을 주는 등 다양한 분야에서 활용되고 있는데 본 논문에서는 빅데이터를 이용한 IoT의 활용사례를 알아보고, 긍정적 사례와 부정적인 활용사례를 분석한 뒤 그 발전방향에 대해 제시하고자 한다.

  • PDF

A Normal Network Behavior Profiling Method Based on Big Data Analysis Techniques (Hadoop/Hive) (빅데이터 분석 기술(Hadoop/Hive) 기반 네트워크 정상행위 규정 방법)

  • Kim, SungJin;Kim, Kangseok
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • 제27권5호
    • /
    • pp.1117-1127
    • /
    • 2017
  • With the advent of Internet of Things (IoT), the number of devices connected to Internet has rapidly increased, but the security for IoT is still vulnerable. It is difficult to integrate existing security technologies due to generating a large amount of traffic by using different protocols to use various IoT devices according to purposes and to operate in a low power environment. Therefore, in this paper, we propose a normal network behavior profiling method based on big data analysis techniques. The proposed method utilizes a Hadoop/Hive for Big Data analytics and an R for statistical computing. Also we verify the effectiveness of the proposed method through a simulation.

Designing Cost Effective Open Source System for Bigdata Analysis (빅데이터 분석을 위한 비용효과적 오픈 소스 시스템 설계)

  • Lee, Jong-Hwa;Lee, Hyun-Kyu
    • Knowledge Management Research
    • /
    • 제19권1호
    • /
    • pp.119-132
    • /
    • 2018
  • Many advanced products and services are emerging in the market thanks to data-based technologies such as Internet (IoT), Big Data, and AI. The construction of a system for data processing under the IoT network environment is not simple in configuration, and has a lot of restrictions due to a high cost for constructing a high performance server environment. Therefore, in this paper, we will design a development environment for large data analysis computing platform using open source with low cost and practicality. Therefore, this study intends to implement a big data processing system using Raspberry Pi, an ultra-small PC environment, and open source API. This big data processing system includes building a portable server system, building a web server for web mining, developing Python IDE classes for crawling, and developing R Libraries for NLP and visualization. Through this research, we will develop a web environment that can control real-time data collection and analysis of web media in a mobile environment and present it as a curriculum for non-IT specialists.

Application Analysis of Smart Tourism Management Model under the Background of Big Data and IOT

  • Gangmin Weng;Jingyu Zhang
    • Journal of Information Processing Systems
    • /
    • 제19권3호
    • /
    • pp.347-354
    • /
    • 2023
  • The rapid development of information technology has accelerated the application of big data and the Internet of Things in various industries. Big data has a great potential in the development of smart tourism. With the help of innovation in emerging technologies such as big data and Internet of Things, smart tourism has a better possibility to surpass traditional tourism. Therefore, this article provides a theoretical support to this process. It has explored the innovative management model of big data and IoT in smart tourism and evaluate their effects on promoting tourism. It offers a reference for the integration and innovation of the tourism theory system. Before big data technology, the development of Internet boosted online tourism. However, tourism marketing is still inefficient due to a lack of understanding about tourists. After many practical explorations of big data technology, tourism websites begin to adopt big data technology in their daily operations. With the changes in tourists' preferences and needs, further innovation and research are needed to help smart tourism keep up with the changes in the market and create more competitive products and services. Innovation serves as the driving force for enterprises to occupy the market and develop.

Real-Time IoT Big-data Processing for Stream Reasoning (스트림-리즈닝을 위한 실시간 사물인터넷 빅-데이터 처리)

  • Yun, Chang Ho;Park, Jong Won;Jung, Hae Sun;Lee, Yong Woo
    • Journal of Internet Computing and Services
    • /
    • 제18권3호
    • /
    • pp.1-9
    • /
    • 2017
  • Smart Cities intelligently manage numerous infrastructures, including Smart-City IoT devices, and provide a variety of smart-city applications to citizen. In order to provide various information needed for smart-city applications, Smart Cities require a function to intelligently process large-scale streamed big data that are constantly generated from a large number of IoT devices. To provide smart services in Smart-City, the Smart-City Consortium uses stream reasoning. Our stream reasoning requires real-time processing of big data. However, there are limitations associated with real-time processing of large-scale streamed big data in Smart Cities. In this paper, we introduce one of our researches on cloud computing based real-time distributed-parallel-processing to be used in stream-reasoning of IoT big data in Smart Cities. The Smart-City Consortium introduced its previously developed smart-city middleware. In the research for this paper, we made cloud computing based real-time distributed-parallel-processing available in the cloud computing platform of the smart-city middleware developed in the previous research, so that we can perform real-time distributed-parallel-processing with them. This paper introduces a real-time distributed-parallel-processing method and system for stream reasoning with IoT big data transmitted from various sensors of Smart Cities and evaluate the performance of real-time distributed-parallel-processing of the system where the method is implemented.