• Journal of Digital Convergence
• /
• v.14 no.9
• /
• pp.317-323
• /
• 2016

Big Data is now seen as a major asset in the company's competitiveness, its influence in the future is expected to grow. Companies that recognize the importance are already actively engaged with Big Data in product development and marketing, which are increasingly applied across sectors of society, including politics, sports. However, lack of knowledge of the system implementation and high costs are still a big obstacles to the introduction of Big Data and systems. It is an objective in this study to build a Big Data system, which is based on open source Hadoop and Hive among Big Data systems, utilizing POS sales data of small and medium-sized offline markets. This approach of convergence is expected to improve existing sales systems that have been simply focusing on profit and loss analysis. It will also be able to use it as the basis for the decisions of the executive to enable prediction of the consumption patterns of customer preference and demand in advance.

• KIISE Transactions on Computing Practices
• /
• v.21 no.10
• /
• pp.651-657
• /
• 2015

The amount of web traffic has increased as a result of the rapid growth of the use of web-based applications. In order to obtain valuable information from web logs, we need to develop systems that can support interactive, flexible, and efficient ways to analyze and handle large amounts of data. In this paper, we present CERES, a log-based, interactive web analytics system for backbone networks. Since CERES focuses on analyzing web log records generated from backbone networks, it is possible to perform a web analysis from the perspective of a network. CERES is designed for deployment in a server cluster using the Hadoop Distributed File System (HDFS) as the underlying storage. We transform and store web log records from backbone networks into relations and then allow users to use a SQL-like language to analyze web log records in a flexible and interactive manner. In particular, we use the data cube technique to enable the efficient statistical analysis of web log. The system provides users a web-based, multi-modal user interface.

• Journal of Internet Computing and Services
• /
• v.18 no.3
• /
• pp.71-83
• /
• 2017

In recent years, by utilizing the greatest strengths of process mining, the various research activities have been actively progressed to use auditing work of business organization. On the other hand, there is insufficient research on systematic and efficient analysis of massive data generated under big data environment using process mining, and proactive monitoring of risk management from audit side, which is one of important management activities of corporate organization. In this study, we intend to realize Hadoop-based internal audit integrated real-time monitoring system in order to detect the abnormal symptoms in prevent accidents in advance. Through the integrated real-time monitoring system for purchasing audit, we intend to realize strengthen the delivery management of purchasing materials ordered, reduce cost of purchase, manage competitive companies, prevent fraud, comply with regulations, and adhere to internal control accounting system. As a result, we can provide information that can be immediately executed due to enhanced purchase audit integrated real-time monitoring by analyzing data efficiently using process mining via Hadoop-based systems. From an integrated viewpoint, it is possible to manage the business status, by processing a large amount of work at a high speed faster than the continuous monitoring, the effectiveness of the quality improvement of the purchase audit and the innovation of the purchase process appears.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2016.01a
• /
• pp.305-306
• /
• 2016

현재 스마트 시대에 살고 있는 우리는 매우 복잡하고 거미줄처럼 연결되어 있는 빅 데이터 환경에서 살고 있다. 이런 환경에서는 대용량 데이터를 효율적으로 관리하고 활용하는 것이 개인이나 기업들이 추구하려는 목표이다. 빅 데이터 시대에 데이터의 효율적인 관리와 활용을 위해 다양한 장비에서 수집되고 저장된 대용량 데이터에 대해서 일반적인 데이터 분석을 통한 보안 기술로는 상당한 시간과 자원 낭비가 수반된다. 이를 개선하기 위해 본 논문에서는 하둡을 이용하여 대용량 데이터에 대한 처리 및 분석을 통해 효과적인 보안 시스템을 제안한다.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.11
• /
• pp.1544-1553
• /
• 2018

Recently internal information leakage in industries is severely increasing in spite of industry security policy. Thus, it is essential to prepare an information leakage prevention measure by industries. Most of the leaks result from the insiders, not from external attacks. In this paper, a real-time internal information leakage prevention system via both storage and network is implemented in order to protect confidential file leakage. In addition, a Hadoop-based user behavior analysis and statistics system is designed and implemented for storing and analyzing information log data in industries. The proposed system stores a large volume of data in HDFS and improves data processing capability using RHive, consequently helps the administrator recognize and prepare the confidential file leak trials. The implemented audit system would be contributed to reducing the damage caused by leakage of confidential files inside of the industries via both portable data media and networks.

• Journal of KIISE
• /
• v.45 no.3
• /
• pp.280-287
• /
• 2018

Recently, the use of a recommendation system and tensor data analysis, which has high-dimensional data, is increasing, as they allow us to analyze the tensor and extract potential elements and patterns. However, due to the large size and complexity of the tensor, it needs to be decomposed in order to analyze the tensor data. While several tools are used for tensor decomposition such as rTensor, pyTensor, and MATLAB, since such tools run on a single machine, they are unable to handle large data. Also, while distributed tensor decomposition tools based on Hadoop can handle a scalable tensor, its computing speed is too slow. In this paper, we propose S-PARAFAC, which is a tensor decomposition tool based on Apache Spark, in distributed in-memory environments. We converted the PARAFAC algorithm into an Apache Spark version that enables rapid processing of tensor data. We also compared the performance of the Hadoop based tensor tool and S-PARAFAC. The result showed that S-PARAFAC is approximately 4~25 times faster than the Hadoop based tensor tool.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.22 no.3
• /
• pp.443-450
• /
• 2018

Due to the improvement of quality of life, health care is a main concern of modern people, and the demand for healthcare system is increasing naturally. However, it is difficult to provide customized wellness information suitable for a specific user because there are various medical information on the Internet and it is difficult to estimate the reliability of the information. In this study, we propose a user - centered service that can provide customized service suitable for users rather than simple search function by classifying big data as text mining and providing personalized medical information. We built a big data system and measured the data processing time while increasing the Hadoop slave node for efficient big data analysis. It is confirmed that it is efficient to build big data system than existing system.

• KIPS Transactions on Computer and Communication Systems
• /
• v.2 no.12
• /
• pp.569-576
• /
• 2013

Cloud computing has been receiving increasing attention recently. Despite this attention, security is the main problem that still needs to be addressed for cloud computing. In general, a cloud computing environment protects data by using distributed servers for data storage. When the amount of data is too high, however, different pieces of a secret key (if used) may be divided among hundreds of distributed servers. Thus, the management of a distributed server may be very difficult simply in terms of its authentication, encryption, and decryption processes, which incur vast overheads. In this paper, we proposed a efficiently data storage and recovery scheme using XOR and RAID in Hadoop environment.

• Journal of the Korean Data and Information Science Society
• /
• v.24 no.5
• /
• pp.975-987
• /
• 2013

The Hadoop system was developed by the Apache foundation based on GFS and MapReduce technologies of Google. Many modern systems for managing and processing the big data have been developing based on the Hadoop because the Hadoop was designed for scalability and distributed computing. The R software has been considered as a well-suited analytic tool in the Hadoop based systems because the R is flexible to other languages and has many libraries for complex analyses. We introduced Rhipe which is a R package supporting MapReduce programming easily under the Hadoop system, and implemented a MapReduce program using Rhipe for multiple regression especially. In addition, we compared the computing speeds of our program with the other packages (ff and bigmemory) for processing the large data. The simulation results showed that our program was more fast than ff and bigmemory as the size of data increases.

• KIISE Transactions on Computing Practices
• /
• v.21 no.9
• /
• pp.579-586
• /
• 2015

As smart-phone usage increases, the number of Social Networking Service (SNS) users has also exponentially increased. SNS allows people to efficiently exchange their personal opinion, and for this reason, it is possible to collect the reaction of each individual to a given event in real-time. Nevertheless, new methods need to be developed to collect and analyze people's opinion in real-time in order to effectively evaluate the impact of a TV advertisement. Hence, we designed and constructed a system that analyzes the effect of an advertisement in real-time by using data related to the advertisement collected from SNS, specifically, Twitter. In detail, Hadoop is used in the system to enable big-data analysis in parallel, and various analyses can be conducted by conducting separate numerical analyses of the degrees of mentioning, preference and reliability. The analysis can be accurate if the reliability is assessed using opinion mining technology. The proposed system is therefore proven to effectively handle and analyze data responses to divers TV advertisement.