• Title/Summary/Keyword: NoSQL System

Search Result 82, Processing Time 0.026 seconds

General-purpose Transaction Management Technique for Data Stability of NoSQL on Distributed File System (분산 파일 시스템 기반 NoSQL의 데이터 안정성을 위한 범용 트랜잭션 관리 기법)

  • Kwon, Younghyun;Yun, Do-hyun;Park, Hojin
    • Journal of Digital Contents Society
    • /
    • v.16 no.2
    • /
    • pp.299-306
    • /
    • 2015
  • In this paper, we research to secure stability of data storing/searching on NoSQL implemented on Distributed File System. When implementing NoSQL on Distributed File System, we faced that random write on Distributed File System is almost impossible. To solve this problem, a concept of Intermediate-File was employed, and then it has been achieved that our system resist any failure circumstance. Additionally, since we discovered its performance cannot be as fast as general File System, by redefining the file block unit for our NoSQL system, we have prevented a slowdown in system performance. As a result, we are able to develop highly scalable NoSQL as Distributed File System, which fulfills basic conditions of transaction: Atomicity, Consistency, Isolation, and Performance.

Improving Database System Performance by Applying NoSQL

  • Choi, Yong-Lak;Jeon, Woo-Seong;Yoon, Seok-Hwan
    • Journal of Information Processing Systems
    • /
    • v.10 no.3
    • /
    • pp.355-364
    • /
    • 2014
  • Internet accessibility has been growing due to the diffusion of smartphones in today's society. Therefore, people can generate data anywhere and are confronted with the challenge that they should process a large amount of data. Since the appearance of relational database management system (RDBMS), most of the recent information systems are built by utilizing it. RDBMS uses foreign-keys to avoid data duplication. The transactions in the database use attributes, such as atomicity, consistency, isolation, durability (ACID), which ensures that data integrity and processing results are stably managed. The characteristic of RDBMS is that there is high data reliability. However, this results in performance degradation. Meanwhile, from among these information systems, some systems only require high-performance rather than high reliability. In this case, if we only consider performance, the use of NoSQL provides many advantages. It is possible to reduce the maintenance cost of the information system that continues to increase in the use of open source software based NoSQL. And has a huge advantage that is easy to use NoSQL. Therefore, in this study, we prove that the leverage of NoSQL will ensure high performance than RDBMS by applying NoSQL to database systems that implement RDBMS.

Design of the Protection System for NoSQL Injection Attack (NoSQL Injection 공격 방어 시스템 설계)

  • Jung, Yong-Hwan;Jo, Jin-O;Gil, Joon-Min;Choi, Jang-Won
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.867-869
    • /
    • 2015
  • 최근 폭발적으로 증가하는 데이터양과 데이터 특성들로 인해 관계형 데이터베이스는 빅데이터 처리에 어려움이 발생하기 시작했으며, 이러한 빅데이터의 신속한 처리를 위해 비정형 데이터 분산처리, 병렬 처리 등 특정한 영역에서 우수한 성능을 보이는 NoSQL 데이터베이스의 활용이 증가하고 있다. 기존 관계형 데이터베이스에서 악의적인 SQL Injection 공격이 시스템에 치명적인 피해를 주는 것과 마찬가지로, 다른 쿼리 언어를 사용하는 NoSQL 데이터베이스에서도 Injection 공격에 대한 취약점이 여전히 존재할 뿐만 아니라, NoSQL 데이터베이스가 비교적 최신 기술이기 때문에 개발자들은 Injection 공격에 대한 인식이 부족한 실정이다. 본 논문에서는 NoSQL 데이터베이스에 대한 대표적인 2가지 Injection 공격 방법을 소개하고, "NoSQL Injection Defender(NID)"라 명명한 Injection 공격 방어 시스템의 설계 방안을 제시하고자 한다.

Interoperability between NoSQL and RDBMS via Auto-mapping Scheme in Distributed Parallel Processing Environment (분산병렬처리 환경에서 오토매핑 기법을 통한 NoSQL과 RDBMS와의 연동)

  • Kim, Hee Sung;Lee, Bong Hwan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.11
    • /
    • pp.2067-2075
    • /
    • 2017
  • Lately big data processing is considered as an emerging issue. As a huge amount of data is generated, data processing capability is getting important. In processing big data, both Hadoop distributed file system and unstructured date processing-based NoSQL data store are getting a lot of attention. However, there still exists problems and inconvenience to use NoSQL. In case of low volume data, MapReduce of NoSQL normally consumes unnecessary processing time and requires relatively much more data retrieval time than RDBMS. In order to address the NoSQL problem, in this paper, an interworking scheme between NoSQL and the conventional RDBMS is proposed. The developed auto-mapping scheme enables to choose an appropriate database (NoSQL or RDBMS) depending on the amount of data, which results in fast search time. The experimental results for a specific data set shows that the database interworking scheme reduces data searching time by 35% at the maximum.

Applications of Open-source NoSQL Database Systems for Astronomical Spatial and Temporal Data

  • Shin, Min-Su
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.42 no.2
    • /
    • pp.88.3-89
    • /
    • 2017
  • We present our experiences with open-source NoSQL database systems in analyzing spatial and temporal astronomical data. We conduct experiments of using Redis in-memory NoSQL database system by modifying and exploiting its support of geohash for astronmical spatial data. Our experiment focuses on performance, cost, difficulty, and scalability of the database system. We also test OpenTSDB as a possible NoSQL database system to process astronomical time-series data. Our experiments include ingesting, indexing, and querying millions or billions of astronomical time-series measurements. We choose our KMTNet data and the public VVV (VISTA Variables in the Via Lactea) catalogs as test data. We discuss issues in using these NoSQL database systems in astronomy.

  • PDF

Transaction Processing Method for NoSQL Based Column

  • Kim, Jeong-Joon
    • Journal of Information Processing Systems
    • /
    • v.13 no.6
    • /
    • pp.1575-1584
    • /
    • 2017
  • As interest in big data has increased recently, NoSQL, a solution for storing and processing big data, is getting attention. NoSQL supports high speed, high availability, and high scalability, but is limited in areas where data integrity is important because it does not support multiple row transactions. To overcome these drawbacks, many studies are underway to support multiple row transactions in NoSQL. However, existing studies have a disadvantage that the number of transactions that can be processed per unit of time is low and performance is degraded. Therefore, in this paper, we design and implement a multi-row transaction system for data integrity in big data environment based on HBase, a column-based NoSQL which is widely used recently. The multi-row transaction system efficiently performs multi-row transactions by adding columns to manage transaction information for every user table. In addition, it controls the execution, collision, and recovery of multiple row transactions through the transaction manager, and it communicates with HBase through the communication manager so that it can exchange information necessary for multiple row transactions. Finally, we performed a comparative performance evaluation with HAcid and Haeinsa, and verified the superiority of the multirow transaction system developed in this paper.

NoSQL-based Sensor Web System for Fine Particles Analysis Services (미세먼지 분석 서비스를 위한 NoSQL 기반 센서 웹 시스템)

  • Kim, Jeong-Joon;Kwak, Kwang-Jin;Park, Jeong-Min
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.2
    • /
    • pp.119-125
    • /
    • 2019
  • Recently, it has become a social problem due to fine particles. There are more people wearing masks, weather alerts and disaster notices. Research and policy are actively underway. Meteorologically, the biggest damage caused by fine particles is the inversion layer phenomenon. In this study, we designed a system to warn fine Particles by analyzing inversion layer and wind direction. This weather information system proposes a system that can efficiently perform scalability and parallel processing by using OGC sensor web enablement system and NoSQL storage for sensor control and data exchange.

Performance Analysis of RDBMS and MongoDB through YCSB in Medical Data Processing System Based HL7 FHIR (HL7 FHIR 기반 의료 데이터 처리 시스템에서 YCSB를 통한 RDBMS와 MongoDB의 성능 분석 연구)

  • Jeon, Dong-cheol;Lee, Byung Mun;Hwang, Heejoung
    • Journal of Korea Multimedia Society
    • /
    • v.21 no.8
    • /
    • pp.934-941
    • /
    • 2018
  • There are some limits on cost and efficiency for large amount of data in RDBMS, and NoSQL is starting to gain popularity. In medical institutions, data forms are different between organizations, and that makes difficulty for interoperability between organizations. In this paper we focused on performance issues between RDMBS and NoSQL in medical documents. We had built two different environment and had experiment comparative analysis of NoSQL with RDBMS based on medical data. We used medical HL7 FHIR as a medical data standard. Also YCSB benchmark tool was used for performance comparison. Experiments shows that NoSQL has better performance in large amounts of medical data processing systems that have over 10,000~100,000 records.

NoSQL-based User Behavior Detection System in Cloud Computing Environment (NoSQL 기반 클라우드 사용자 행동 탐지 시스템 설계)

  • Ahn, Kwang-Min;Lee, Bong-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.804-807
    • /
    • 2012
  • Cloud service provider has to protect client's information securely since all the resources are offered by the service provider, and a large number of users share the resources. In this paper, a NoSQL-based anomaly detection system is proposed in order to enhance the security of mobile cloud services. The existing integrated security management system that uses a relational database can not be used for real-time processing of data since security log from a variety of security equipment and data from cloud node have different data format with unstructured features. The proposed system can resolve the emerging security problem because it provides real time processing and scalability in distributed processing environment.

  • PDF

Comparative Evaluation of Data Processing Performance between MySQL and Redis (MySQL과 Redis의 데이터 처리 성능 비교 평가)

  • Hyeok Bang;Seo-Hyeon Kim;Sanghoon Jeon
    • Journal of Internet Computing and Services
    • /
    • v.25 no.3
    • /
    • pp.35-41
    • /
    • 2024
  • As online activities have rapidly increased due to recent digital changes and the impact of COVID-19, the importance of large-scale data processing and maintenance is increasing. This study compares the performance of the two main types of databases widely used for data storage and management: Relational Database Management Systems (RDBMS) and Non-Relational Databases (NoSQL). Specifically, we measured and evaluated the execution time of data insertion, query, and deletion functions using MySQL, a representative example of RDBMS, and Redis, a representative example of NoSQL. The experimental results showed that Redis showed performance about 5.84 times faster in data insertion, 6.61 times faster in query, and 12.33 times faster in deletion than MySQL. These results demonstrate that Redis provides superior performance, especially in environments requiring large-scale data processing and maintenance. Therefore, companies and online service providers can choose NoSQL databases such as Redis to ensure more efficient data management solutions. We hope this study will be an essential reference when selecting a database based on data processing performance.