• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.12 no.2
• /
• pp.127-134
• /
• 2019

With the advancement of IT technology, the amount of data generated has been growing exponentially every year. As an alternative to this, research on distributed systems and in-memory based big data processing schemes has been actively underway. The processing power of traditional big data processing schemes enables big data to be processed as fast as the number of nodes and memory capacity increases. However, the increase in the number of nodes inevitably raises the frequency of failures in a big data infrastructure environment, and infrastructure management points and infrastructure operating costs also increase accordingly. In addition, the increase in memory capacity raises infrastructure costs for a node configuration. Therefore, this paper proposes an in-memory-based hybrid big data processing scheme for improve the big data processing rate. The proposed scheme reduces the number of nodes compared to traditional big data processing schemes based on distributed systems by adding a combiner step to a distributed system processing scheme and applying an in-memory based processing technology at that step. It decreases the big data processing time by approximately 22%. In the future, realistic performance evaluation in a big data infrastructure environment consisting of more nodes will be required for practical verification of the proposed scheme.

• Journal of Information Processing Systems
• /
• v.15 no.1
• /
• pp.1-6
• /
• 2019

Recently, big data and artificial intelligence (AI) based on communication systems have become one of the hottest issues in the technology sector, and methods of analyzing big data using AI approaches are now considered essential. This paper presents diverse paradigms to subjects which deal with diverse research areas, such as image segmentation, fingerprint matching, human tracking techniques, malware distribution networks, methods of intrusion detection, digital image watermarking, wireless sensor networks, probabilistic neural networks, query processing of encrypted data, the semantic web, decision-making, software engineering, and so on.

• Journal of Information Technology Services
• /
• v.13 no.2
• /
• pp.185-194
• /
• 2014

Recently, as value for practical use of big data is evaluated, companies and organizations that create benefit and profit are gradually increasing with application of big data. But specifical and theoretical study about possible risk factors as introduction of big data is not being conducted. Accordingly, the study extracts the possible risk factors as introduction of big data based on literature reviews and classifies according to big data processing, data collection, data storage, data analysis, analysis data visualization and application. Also, the risk factors have order of priority according to the degree of risk from the survey of experts. This study will make a chance that can avoid risks by bid data processing and preparation for risks in order of dangerous grades of risk.

• Journal of Information Processing Systems
• /
• v.19 no.3
• /
• pp.347-354
• /
• 2023

The rapid development of information technology has accelerated the application of big data and the Internet of Things in various industries. Big data has a great potential in the development of smart tourism. With the help of innovation in emerging technologies such as big data and Internet of Things, smart tourism has a better possibility to surpass traditional tourism. Therefore, this article provides a theoretical support to this process. It has explored the innovative management model of big data and IoT in smart tourism and evaluate their effects on promoting tourism. It offers a reference for the integration and innovation of the tourism theory system. Before big data technology, the development of Internet boosted online tourism. However, tourism marketing is still inefficient due to a lack of understanding about tourists. After many practical explorations of big data technology, tourism websites begin to adopt big data technology in their daily operations. With the changes in tourists' preferences and needs, further innovation and research are needed to help smart tourism keep up with the changes in the market and create more competitive products and services. Innovation serves as the driving force for enterprises to occupy the market and develop.

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.11 no.1
• /
• pp.70-75
• /
• 2018

Smart Factory is an intelligent factory that can enhance productivity, quality, customer satisfaction, etc. by applying information and communications technology to the entire production process including design & development, manufacture, and distribution & logistics. The precise amount of data generated in a smart factory varies depending on the factory's size and state of facilities. Regardless, it would be difficult to apply traditional production management systems to a smart factory environment, as it generates vast amounts of data. For this reason, the need for a distributed big-data processing system has risen, which can process a large amount of data. Therefore, this article has designed a Gluster File System (GlusterFS)-based distributed big-data processing system that can be used in a smart factory environment. Compared to existing distributed processing systems, the proposed distributed big-data processing system reduces the system load and the risk of data loss through the distribution and management of network traffic.

• Journal of KIISE
• /
• v.44 no.1
• /
• pp.84-94
• /
• 2017

Recently, the importance of velocity, one of the characteristics of big data (5V: Volume, Variety, Velocity, Veracity, and Value), has been emphasized in the data processing, which has led to several studies on the real-time stream processing, a technology for quick and accurate processing and analyses of big data. In this paper, we propose a Squall framework using Time-triggered Message-triggered Object (TMO) technology, a model that is widely used for processing real-time big data. Moreover, we provide a description of Squall framework and its operations under a single node. TMO is an object model that supports the non-regular real-time processing method for certain conditions as well as regular periodic processing for certain amount of time. A Squall framework can support the real-time event stream of big data and micro-batch processing with outstanding performances, as compared to Apache storm and Spark Streaming. However, additional development for processing real-time stream under multiple nodes that is common under most frameworks is needed. In conclusion, the advantages of a TMO model can overcome the drawbacks of Apache storm or Spark Streaming in the processing of real-time big data. The TMO model has potential as a useful model in real-time big data processing.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.10
• /
• pp.1179-1187
• /
• 2019

In today's technology environment, most big data-based applications and solutions are based on real-time processing of streaming data. Real-time processing and analysis of big data streams plays an important role in the development of big data-based applications and solutions. In particular, in the maritime data processing environment, the necessity of developing a technology capable of rapidly processing and analyzing a large amount of real-time data due to the explosion of data is accelerating. Therefore, this paper analyzes the characteristics of NiFi, Kafka, and Druid as suitable open source among various open data technologies for processing big data, and provides the latest information on external linkage necessary for maritime service analysis in Korean e-Navigation service. To this end, we will lay the foundation for applying open data framework technology for real-time data processing.

• Journal of Information Technology Applications and Management
• /
• v.26 no.5
• /
• pp.57-65
• /
• 2019

Recommendation Systems are information technologies that E-commerce merchants have adopted so that online shoppers can receive suggestions on items that might be interesting or complementing to their purchased items. These systems stipulate valuable assistance to the user's purchasing decisions, and provide quality of push service. Traditionally, Recommendation Systems have been designed using a centralized system, but information service is growing vast with a rapid and strong scalability. The next generation of information technology such as Cloud Computing and Big Data Environment has handled massive data and is able to support enormous processing power. Nevertheless, analytic technologies are lacking the different capabilities when processing big data. Accordingly, we are trying to design a conceptual service model with a proposed new algorithm and user adaptation on dynamic recommendation service for big data environment.

• Journal of Digital Convergence
• /
• v.12 no.8
• /
• pp.351-359
• /
• 2014

The log data generated by security equipment have been synthetically analyzed on the ESM(Enterprise Security Management) base so far, but due to its limitations of the capacity and processing performance, it is not suited for big data processing. Therefore the another way of technology on the big data platform is necessary. Big Data platform can achieve a large amount of data collection, storage, processing, retrieval, analysis, and visualization by using Hadoop Ecosystem. Currently ESM technology has developed in the way of SIEM (Security Information & Event Management) technology, and to implement security technology in SIEM way, Big Data platform technology is essential that can handle large log data which occurs in the current security devices. In this paper, we have a big data platform Hadoop Ecosystem technology for analyzing the security log for sure how to implement the system model is studied.

• Journal of the Korea Convergence Society
• /
• v.9 no.10
• /
• pp.13-19
• /
• 2018

MapReduce, the Hadoop's essential core technology, is most commonly used to process big data based on the Hadoop distributed file system. However, the existing MapReduce-based big data processing techniques have a feature of dividing and storing files in blocks predefined in the Hadoop distributed file system, thus wasting huge infrastructure resources. Therefore, in this paper, we propose an efficient MapReduce-based big data processing scheme. The proposed method enhances the storage efficiency of a big data infrastructure environment by converting and compressing the data to be processed into a data format in advance suitable for processing by MapReduce. In addition, the proposed method solves the problem of the data processing time delay arising from when implementing with focus on the storage efficiency.