• Journal of Korea Multimedia Society
• /
• v.19 no.2
• /
• pp.386-395
• /
• 2016

In trajectory-based navigation systems, a huge amount of trajectory data is needed for efficient route explorations. However, it would be very hard to collect trajectories from all the possible start and destination combinations. To provide a practical solution to this problem, we suggest a method combining collected GPS trajectories data into additional generated trajectories with new start and destination combinations without road information. We present a trajectory combination algorithm and its implementation with Scala programming language on Spark platform for big data processing. The experimental results proved that the proposed method can effectively populate the collected trajectories into valid trajectory paths more than three hundred times.

• Spatial Information Research
• /
• v.19 no.4
• /
• pp.55-71
• /
• 2011

Recently, interest in cloud computing which provides IT resources as service form in IT field is increasing. As a result, much research has been done on the distributed data processing that store and manage a large amount of data in many servers. Meanwhile, in order to effectively utilize the spatial data which is rapidly increasing day by day with the growth of GIS technology, distributed processing of spatial data using cloud computing is essential. Therefore, in this paper, we review the representative distributed data processing techniques and we analyze the optimization requirements for performance improvement of the distributed processing techniques for a large amount of data. In addition, we uses the Hadoop and we evaluate the performance of the distributed data processing techniques for their optimization requirements.

• Smart Media Journal
• /
• v.4 no.3
• /
• pp.56-61
• /
• 2015

Although many successful cases regarding big data have been reported, building information systems of big data is still difficult. From the perspective of technology the builders need to understand the whole process of systems development ranging from collecting, storing, processing, and analyzing data to presenting and using information. Whereas, from the perspective of business, the builders need to understand the values of the proposed big data project and explain to top managers who have to make a decision of the risky investment. This study proposes a framework of 5W 1H that can help the builder understand things related to the development of big data information systems. In addition, big data cases from the real world have been illustrated by applying to the framework. It is expected to help builders understand and manage big data projects and lead managers to make better decisions of the investment to the development of information systems.

• Convergence Security Journal
• /
• v.15 no.7
• /
• pp.31-38
• /
• 2015

Recently the government civil affairs administration system has been advanced to a cloud computing environment from a simple network environment. The electronic civil affairs processing environment in recent years means cloud computing environment based bid data services. Therefore, there exist lots of problems in processing big data for the government civil affairs service compared to the conventional information acquisition environment. That is, it processes new information through collecting required information from different information systems much further than the information service in conventional network environments. According to such an environment, applications of providing administration information for processing the big data have been becoming a major target of illegal attackers. The objectives of this study are to prevent illegal uses of the electronic civil affairs service based on IPs nationally located in civil affairs centers and to protect leaks of the important data retained in these centers. For achieving it, the safety, usability, and security of services are to be ensured by using different authentication processes and encryption methods based on these processes.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.13 no.2
• /
• pp.1-7
• /
• 2013

Big data management is becoming increasingly important in both industry and academia of information science community. One of the important categories of big data generated from software systems is log data. Log data is generally used for better services in various service providers and can also be used as information for qualification. This paper presents a big data management architecture specialized for log data. Specifically, it provides the aggregation of log messages sent from multiple clients and provides intelligent functionalities such as analyzing log data. The proposed architecture supports an asynchronous process in client-server architectures to prevent the potential bottleneck of accessing data. Accordingly, it does not affect the client performance although using remote data store. We implement the proposed architecture and show that it works well for processing big log data. All components are implemented based on open source software and the developed prototypes are now publicly available.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.5
• /
• pp.470-476
• /
• 2015

Big data has been used in diverse areas. For example, in computer science and sociology, there is a difference in their issues to approach big data, but they have same usage to analyze big data and imply the analysis result. So the meaningful analysis and implication of big data are needed in most areas. Statistics and machine learning provide various methods for big data analysis. In this paper, we study a process for big data analysis, and propose an efficient methodology of entire process from collecting big data to implying the result of big data analysis. In addition, patent documents have the characteristics of big data, we propose an approach to apply big data analysis to patent data, and imply the result of patent big data to build R&D strategy. To illustrate how to use our proposed methodology for real problem, we perform a case study using applied and registered patent documents retrieved from the patent databases in the world.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.11
• /
• pp.2067-2075
• /
• 2017

Lately big data processing is considered as an emerging issue. As a huge amount of data is generated, data processing capability is getting important. In processing big data, both Hadoop distributed file system and unstructured date processing-based NoSQL data store are getting a lot of attention. However, there still exists problems and inconvenience to use NoSQL. In case of low volume data, MapReduce of NoSQL normally consumes unnecessary processing time and requires relatively much more data retrieval time than RDBMS. In order to address the NoSQL problem, in this paper, an interworking scheme between NoSQL and the conventional RDBMS is proposed. The developed auto-mapping scheme enables to choose an appropriate database (NoSQL or RDBMS) depending on the amount of data, which results in fast search time. The experimental results for a specific data set shows that the database interworking scheme reduces data searching time by 35% at the maximum.

• Journal of the Korea Convergence Society
• /
• v.7 no.5
• /
• pp.1-6
• /
• 2016

Many studies have been carried out for the development of big data utilization and analysis technology recently. There is a tendency that government agencies and companies to introduce a Hadoop of a processing platform for analyzing big data is increasing gradually. Increased interest with respect to the processing and analysis of these big data collection technology of data has become a major issue in parallel to it. However, study of the collection technology as compared to the study of data analysis techniques, it is insignificant situation. Therefore, in this paper, to build on the Hadoop cluster is a big data analysis platform, through the Apache sqoop, stylized from relational databases, to collect the data. In addition, to provide a sensor through the Apache flume, a system to collect on the basis of the data file of the Web application, the non-structured data such as log files to stream. The collection of data through these convergence would be able to utilize as a basic material of big data analysis.

• Journal of Information Processing Systems
• /
• v.15 no.6
• /
• pp.1306-1325
• /
• 2019

As the world is moving towards digitization, data is generated from various sources at a faster rate. It is getting humungous and is termed as big data. The financial sector is one domain which needs to leverage the big data being generated to identify financial risks, fraudulent activities, and so on. The design of predictive models for such financial big data is imperative for maintaining the health of the country's economics. Financial data has many features such as transaction history, repayment data, purchase data, investment data, and so on. The main problem in predictive algorithm is finding the right subset of representative features from which the predictive model can be constructed for a particular task. This paper proposes a correlation-based method using submodular optimization for selecting the optimum number of features and thereby, reducing the dimensions of the data for faster and better prediction. The important proposition is that the optimal feature subset should contain features having high correlation with the class label, but should not correlate with each other in the subset. Experiments are conducted to understand the effect of the various subsets on different classification algorithms for loan data. The IBM Bluemix BigData platform is used for experimentation along with the Spark notebook. The results indicate that the proposed approach achieves considerable accuracy with optimal subsets in significantly less execution time. The algorithm is also compared with the existing feature selection and extraction algorithms.

• Journal of Information Processing Systems
• /
• v.16 no.2
• /
• pp.377-393
• /
• 2020

Freight management systems require a new business model for rapid decision making to improve their business processes by dynamically analyzing the previous experience data. Moreover, the amount of data generated by daily business activities to be analyzed for making better decisions is enormous. Online-to-offline or offline-to-online (O2O) is an electronic commerce (e-commerce) model used to combine the online and physical services. Data analysis is usually performed offline. In the present paper, to extend its benefits to online and to efficiently apply the big data analysis to the freight management system, we suggested a system architecture based on O2O services. We analyzed and extracted the useful knowledge from the real-time freight data for the period 2014-2017 aiming at further business development. The proposed system was deemed useful for truck management companies as it allowed dynamically obtaining the big data analysis results based on O2O services, which were used to optimize logistic freight, improve customer services, predict customer expectation, reduce costs and overhead by improving profit margins, and perform load balancing.