• Ocean Systems Engineering
• /
• v.9 no.1
• /
• pp.79-95
• /
• 2019

The material analysis of an offshore structure is generally conducted in the contract design phase for the price quotation of a new offshore project. This analysis is conducted manually by an engineer, which is time-consuming and can lead to inaccurate results, because the data size from previous projects is too large, and there are so many materials to consider. In this study, the piping materials in an offshore structure are analyzed for contract design using a big data framework. The big data technologies used include HDFS (Hadoop Distributed File System) for data saving, Hive and HBase for the database to handle the saved data, Spark and Kylin for data processing, and Zeppelin for user interface and visualization. The analyzed results show that the proposed big data framework can reduce the efforts put toward contract design in the estimation of the piping material cost.

• Journal of Satellite, Information and Communications
• /
• v.9 no.3
• /
• pp.109-115
• /
• 2014

This paper suggest meteorological satellite processing software's structure that reduces time and efforts of modification/upgrade. This structure's key feature is "algorithm component" that works within framework and eventually to a complete Meteorological satellite processing system. Most of existing Meteorological satellite system is designed around specific function and data sets which limits range of modification and upgrade. In addition, re-use of current algorithms become difficult although re-use of similar algorithm is the case in many occasions. This inefficiency can be resolved by designing a new framework as a result of detail analysis of collected requirements. A new framework and system architecture has been designed. In addition, operational flow of Satellite image processing framework has been described.

• Journal of KIISE
• /
• v.43 no.6
• /
• pp.700-709
• /
• 2016

Recently, with the rapid growth of LOD (Linked Open Data) existing methods based on a single machine have limitation in performance. Existing solutions use distributed framework such as Mapreduce in order to improve the performance. However, the MapReduce framework for processing SPARQL queries involves multiple MapReduce jobs and additional costs incurred. In addition, the problem of unnecessary data processing arises. In this study, we proposed a method to reduce the number of MapReduce jobs during SPARQL query processing and join indexes based on Bitmap for minimizing the costs of processing unnecessary data.

• International Journal of Contents
• /
• v.12 no.1
• /
• pp.60-64
• /
• 2016

As new applications of cloud data management system (CDMS) such as online games, cooperation edit, social network, and so on, are increasing, transaction processing capabilities for CDMS are required. Several transaction processing methods for cloud data management system (CDMS) have been proposed. However, existing transaction processing methods have some problems. Some of them provide limited transaction processing capabilities. Some of them are hard to be integrated with existing CDMSs. In this paper, we proposed a new concurrency control method to support transaction processing capability for CDMS to solve these problems. The proposed method was designed and implemented based on Spark, an in-memory distributed processing framework. It uses RDD (Resilient Distributed Dataset) model to provide fault tolerant to data in the main memory. In our proposed method, database stored in CDMS is loaded to main memory managed by Spark. The loaded data set is then transformed to RDD. In addition, we proposed a multi-version concurrency control method through immutable characteristics of RDD. Finally, we performed experiments to show the feasibility of the proposed method.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.67 no.2
• /
• pp.270-276
• /
• 2018

Apache Spark is one of the high performance in-memory computing frameworks for big-data processing. Recently, to improve the performance, general-purpose computing on graphics processing unit(GPGPU) is adapted to Apache Spark framework. Previous Spark-GPGPU frameworks focus on overcoming the difficulty of an implementation resulting from the difference between the computation environment of GPGPU and Spark framework. In this paper, we propose a Spark framework based on a heterogenous pipeline computing with OpenCL to further improve the performance. The proposed framework overlaps the Java-to-Native memory copies of CPU with CPU-GPU communications(DMA) and GPU kernel computations to hide the CPU idle time. Also, CPU-GPU communication buffers are implemented with switching dual buffers, which reduce the mapped memory region resulting in decreasing memory mapping overhead. Experimental results showed that the proposed Spark framework based on a heterogenous pipeline computing with OpenCL had up to 2.13 times faster than the previous Spark framework using OpenCL.

• The Transactions of the Korea Information Processing Society
• /
• v.7 no.2S
• /
• pp.692-700
• /
• 2000

As the multimedia application services have been getting various in Internet, need of multimedia database management system which can manage various multimedia information effectively has been increased. The multimedia application generally needs the features to store, manage, retrieve the multimedia data. We need a database management system framework that may be easily integrated with these features. This paper proposes th database management system framework providing the easy integration with various multimedia information retrieval. The framework uses OQL as the retrieval interface, so that user can use it easily. It can be integrated dynamically and easily with various multimedia retrieval method as a loosely coupled system. In addition there is no performance degradation caused in a loosely coupled system.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.05a
• /
• pp.45-48
• /
• 2013

In this paper, we proposed an Indexing Structure System (ISS) based on Apache Hadoop in Wireless Sensor Network (WSN). Nowadays sensors data continuously keep growing that need to control. Data constantly update in order to provide the newest information to users. While data keep growing, data retrieving and storing are face some challenges. So by using the ISS, we can maximize processing quality and minimize data retrieving time. In order to design ISS, Indexing Types have to be defined depend on each sensor type. After identifying, each sensor goes through the Indexing Structure Processing (ISP) in order to be indexed. After ISP, indexed data are streaming and storing in Hadoop Distributed File System (HDFS) across a number of separate machines. Indexed data are split and run by MapReduce tasks. Data are sorted and grouped depend on sensor data object categories. Thus, while users send the requests, all the queries will be filter from sensor data object and managing the task by MapReduce processing framework.

• Journal of Internet Computing and Services
• /
• v.10 no.4
• /
• pp.35-44
• /
• 2009

The emerging notion of data stream has brought many new challenges to the research communities as a consequence of its conceptual difference with conventional concepts of just data. One typical example is data stream processing in sensor networks. The range of data processing considerations in a sensor network is very wide, from physical resource restrictions such as bandwidth, energy, and memory to the peculiarities of query processing including continuous and specific types of queries. In this paper, as one of the physical constraints in data stream processing, we consider the problem of limited memory and propose a new scheme for data stream reduction based on the Principal Component Analysis (PCA) technique. PCA can transform a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables. We adapt PCA for the data stream of a sensor network assuming the cooperation of a query engine (or application) with a network base station. Our method exploits the spatio-temporal correlation among multiple measurements from different sensors. Finally, we present a new framework for data processing and describe a number of experiments under this framework. We compare our scheme with the wavelet transform and observe the effect of time stamps on the compression ratio. We report on some of the results.

• Journal of the Institute of Electronics Engineers of Korea SC
• /
• v.48 no.2
• /
• pp.1-8
• /
• 2011

An underwater robot S/W framework consists of various sub-modules such as sensory data processing module, thruster control module, cognition module and behavior control module. Performance of a robot is determined by not only the efficiency of algorithms used but also effectiveness of their implementations. One most important factor of the effective implementation is the efficiency of data sharing module, as it transmits signals and data between the sub-modules and thus is directly related to the cycles of sensing and control The ideal data sharing module enables immediate access to any data source irrespective of system configurations. In reality, however, there are lots of obstacles including limitation of processing capacity of source modules, delay over network, and scheduling latency of operating systems. The paper proposes a new data sharing architecture and programming models to effectively handle such obstacles in implementation of underwater S/W framework on a small scale distributed computing system.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.05a
• /
• pp.179-182
• /
• 2013

We develop Framework architecture of Online Shopping Services based on M2M and IoT for Handheld Devices in Cloud Computing. MapReduce model will be used as a method to simplify large scale data processing when user search for purchasing products online which provide efficient, and fast respond time. Therefore, providing user with a enhanced Quality of Experience (QoE) as well as Quality of Service (QoS) when purchasing/searching products Online from big data.