• Title/Summary/Keyword: NoSQL System

Search Result 82, Processing Time 0.021 seconds

Comparison research of the Spatial Indexing Methods for ORDBMS in Embedded Systems (임베디드 시스템의 객체 관계형 DBMS에 적합한 공간 인덱스 방법 비교 연구)

  • Lee, Min-Woo;Park, Soo-Hong
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.8 no.1
    • /
    • pp.63-74
    • /
    • 2005
  • The telematics device, which is a typical embedded system on the transportation or vehicle, requires the embedded spatial DBMS based on RTOS (Real Time Operating System) for processing the huge spatial data in real time. This spatial DBMS can be developed very easily by SQL3 functions of the ORDBMS such as UDT (user-defined type) and UDF (user-defined function). However, developing index suitable for the embedded spatial DBMS is very difficult. This is due to the fact that there is no built-in SQL3 functions to construct spatial indexes. In this study, we compare and analyze both Generalized Search Tree and Relational Indexing methods which are suggested as common ways of developing User-Defined Indexes nowadays. Two implementations of R-Tree based on each method were done and region query performance test results were evaluated for suggesting a suitable indexing method of an embedded spatial DBMS, especially for telematics devices.

  • PDF

Development of an Agricultural Data Middleware to Integrate Multiple Sensor Networks for an Farm Environment Monitoring System

  • Kim, Joonyong;Lee, Chungu;Kwon, Tae-Hyung;Park, Geonhwan;Rhee, Joong-Yong
    • Journal of Biosystems Engineering
    • /
    • v.38 no.1
    • /
    • pp.25-32
    • /
    • 2013
  • Purpose: The objective of this study is to develop a data middleware for u-IT convergence in agricultural environment monitoring, which can support non-standard data interfaces and solve the compatibility problems of heterogenous sensor networks. Methods: Six factors with three different interfaces were chosen as target data among the environmental monitoring factors for crop cultivation. PostgresSQL and PostGIS were used for database and the data middleware was implemented by Python programming language. Based on hierarchical model design and key-value type table design, the data middleware was developed. For evaluation, 2,000 records of each data access interface were prepared. Results: Their execution times of File I/O interface, SQL interface and HTTP interface were 0.00951 s/record, 0.01967 s/record and 0.0401 s/record respectively. And there was no data loss. Conclusions: The data middleware integrated three heterogenous sensor networks with different data access interfaces.

The Bigdata Processing Environment Building for the Learning System (학습 시스템을 위한 빅데이터 처리 환경 구축)

  • Kim, Young-Geun;Kim, Seung-Hyun;Jo, Min-Hui;Kim, Won-Jung
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.9 no.7
    • /
    • pp.791-797
    • /
    • 2014
  • In order to create an environment for Apache Hadoop for parallel distributed processing system of Bigdata, by connecting a plurality of computers, or to configure the node, using the configuration of the virtual nodes on a single computer it is necessary to build a cloud fading environment. However, be constructed in practice for education in these systems, there are many constraints in terms of cost and complex system configuration. Therefore, it is possible to be used as training for educational institutions and beginners in the field of Bigdata processing, development of learning systems and inexpensive practical is urgent. Based on the Raspberry Pi board, training and analysis of Big data processing, such as Hadoop and NoSQL is now the design and implementation of a learning system of parallel distributed processing of possible Bigdata in this study. It is expected that Bigdata parallel distributed processing system that has been implemented, and be a useful system for beginners who want to start a Bigdata and education.

Based on MQTT and Node-RED Implementation of a Smart Farm System that stores MongoDB (MQTT와 Node-RED를 기반한 MongoDB로 저장 하는 스마트 팜 시스템 구현)

  • Hong-Jin Park
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.16 no.5
    • /
    • pp.256-264
    • /
    • 2023
  • Smart farm technology using IoT is one of the technologies that can increase productivity and improve the quality of agricultural products in agriculture, which is facing difficulties due to the decline in rural population, lack of rural manpower due to aging, and increase in diseases and pests due to climate change. . Smart farms using existing IoT simply monitor farms, implement smart plant growers, and have automatic greenhouse opening and closing systems. This paper implements a smart farm system based on MQTT, an industry standard protocol for the Internet of Things, and Node-RED, a representative development middleware for the Internet of Things. First, data is extracted from Arduino sensors, and data is collected and transmitted from IoT devices using the MQTT protocol. Then, Node-RED is used to process MQTT messages and store the sensing data in real time in MongoDB, a representative NoSQL, to store the data. Through this smart farm system, farm managers can use a computer or mobile phone to check sensing information on the smart farm in real time, anytime, anywhere, without restrictions on time and space.

The suggestion of new big data platform for the strengthening of privacy and enabled of big data (개인정보 보안강화 및 빅데이터 활성화를 위한 새로운 빅데이터 플랫폼 제시)

  • Song, Min-Gu
    • Journal of Digital Convergence
    • /
    • v.14 no.12
    • /
    • pp.155-164
    • /
    • 2016
  • In this paper, we investigate and analyze big data platform published at home and abroad. The results had a problem with personal information security on each platform. In particular, there was a vulnerability in the encryption of personal information stored in big data representative of HBase NoSQL DB that is commonly used for big data platform. However, data encryption and decryption cause the system load. In this paper, we propose a method of encryption with HBase, encryption and decryption systems, and methods for applying the personal information management system (PMIS) for each step of the way and big data platform to reduce the load on the network to communicate. And we propose a new big data platform that reflects this. Therefore, the proposed Big Data platform will greatly contribute to the activation of Big Data used to obtain personal information security and system performance efficiency.

IEC 61850 Based IoT Gateway Platform for Interworking to Microgrid Operational System (마이크로그리드 운영 시스템 연계를 위한 IEC 61850 기반 IoT 게이트웨이 플랫폼)

  • Park, Jeewon;Song, ByungKwen;Shin, InJae
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.4 no.2
    • /
    • pp.67-73
    • /
    • 2018
  • There are many types of power facilities such as transformers, switches, and energy storage devices in the micro grid environment. However, with the development of IoT technology, opportunities to acquire sensor information such as temperature, pressure, and humidity are provided. In the existing micro grid environment, the communication protocols such as MMS transport protocol in IEC 61850 standard is applied in accordance with the integrated operation between the power facilities and the platform. Therefore, to accommodate IoT data, a gateway technology that can link IoT data to a data collection device (FEP) based on IEC 61850 is required. In this paper, we propose IEC 61850 based IoT gateway platform prototype for microgrid operating system linkage. The gateway platform consists of an IoT protocol interface module (MQTT, CoAP, AMQP) and database, IEC 61850 server. For databases, We used open source based NoSQL databases, Hbase and MongoDB, to store JSON data. We verified the interoperability between the IoT protocol and the IEC 61850 protocol using Sisco's MMS EASY Lite.

MLOps workflow language and platform for time series data anomaly detection

  • Sohn, Jung-Mo;Kim, Su-Min
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.19-27
    • /
    • 2022
  • In this study, we propose a language and platform to describe and manage the MLOps(Machine Learning Operations) workflow for time series data anomaly detection. Time series data is collected in many fields, such as IoT sensors, system performance indicators, and user access. In addition, it is used in many applications such as system monitoring and anomaly detection. In order to perform prediction and anomaly detection of time series data, the MLOps platform that can quickly and flexibly apply the analyzed model to the production environment is required. Thus, we developed Python-based AI/ML Modeling Language (AMML) to easily configure and execute MLOps workflows. Python is widely used in data analysis. The proposed MLOps platform can extract and preprocess time series data from various data sources (R-DB, NoSql DB, Log File, etc.) using AMML and predict it through a deep learning model. To verify the applicability of AMML, the workflow for generating a transformer oil temperature prediction deep learning model was configured with AMML and it was confirmed that the training was performed normally.

A Study on Developing and Refining a Large Citation Service System

  • Kim, Kwang-Young;Kim, Hwan-Min
    • International Journal of Knowledge Content Development & Technology
    • /
    • v.3 no.1
    • /
    • pp.65-80
    • /
    • 2013
  • Today, citation index information is used as an outcome scale of spreading technology and encouraging research. Article citation information is an important factor to determine the authority of the relevant author. Google Scholar uses the article citation information to organize academic article search results with a rank algorithm. For an accurate analysis of such important citation index information, large amounts of bibliographic data are required. Therefore, this study aims to build a fast and efficient system for large amounts of bibliographic data, and to design and develop a system for quickly analyzing cited information for that data. This study also aims to use and analyze citation data to be a basic element for providing various advanced services to the academic article search system.

The Model of Network Packet Analysis based on Big Data (빅 데이터 기반의 네트워크 패킷 분석 모델)

  • Choi, Bomin;Kong, Jong-Hwan;Han, Myung-Mook
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.23 no.5
    • /
    • pp.392-399
    • /
    • 2013
  • Due to the development of IT technology and the information age, a dependency of the network over the most of our lives have grown to a greater extent. Although it provides us to get various useful information and service, it also has negative effectiveness that can provide network intruder with vulnerable roots. In other words, we need to urgently cope with theses serious security problem causing service disableness or system connected to network obstacle with exploiting various packet information. Many experts in a field of security are making an effort to develop the various security solutions to respond against these threats, but existing solutions have a lot of problems such as lack of storage capacity and performance degradation along with the massive increase of packet data volume. Therefore we propose the packet analysis model to apply issuing Big Data technology in the field of security. That is, we used NoSQL which is technology of massive data storage to collect the packet data growing massive and implemented the packet analysis model based on K-means clustering using MapReudce which is distributed programming framework, and then we have shown its high performance by experimenting.

Study of MongoDB Architecture by Data Complexity for Big Data Analysis System (빅데이터 분석 시스템 구현을 위한 데이터 구조의 복잡성에 따른 MongoDB 환경 구성 연구)

  • Hyeopgeon Lee;Young-Woon Kim;Jin-Woo Lee;Seong Hyun Lee
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.16 no.5
    • /
    • pp.354-361
    • /
    • 2023
  • Big data analysis systems apply NoSQL databases like MongoDB to store, process, and analyze diverse forms of large-scale data. MongoDB offers scalability and fast data processing speeds through distributed processing and data replication, depending on its configuration. This paper investigates the suitable MongoDB environment configurations for implementing big data analysis systems. For performance evaluation, we configured both single-node and multi-node environments. In the multi-node setup, we expanded the number of data nodes from two to three and measured the performance in each environment. According to the analysis, the processing speeds for complex data structures with three or more dimensions are approximately 5.75% faster in the single-node environment compared to an environment with two data nodes. However, a setting with three data nodes processes data about 25.15% faster than the single-node environment. On the other hand, for simple one-dimensional data structures, the multi-node environment processes data approximately 28.63% faster than the single-node environment. Further research is needed to practically validate these findings with diverse data structures and large volumes of data.