• Journal of Internet Computing and Services
• /
• v.15 no.3
• /
• pp.45-52
• /
• 2014

The Ubiquitous-City (U-City) is a smart or intelligent city to satisfy human beings' desire to enjoy IT services with any device, anytime, anywhere. It is a future city model based on Internet of everything or things (IoE or IoT). It includes a lot of video cameras which are networked together. The networked video cameras support a lot of U-City services as one of the main input data together with sensors. They generate huge amount of video information, real big data for the U-City all the time. It is usually required that the U-City manipulates the big data in real-time. And it is not easy at all. Also, many times, it is required that the accumulated video data are analyzed to detect an event or find a figure among them. It requires a lot of computational power and usually takes a lot of time. Currently we can find researches which try to reduce the processing time of the big video data. Cloud computing can be a good solution to address this matter. There are many cloud computing methodologies which can be used to address the matter. MapReduce is an interesting and attractive methodology for it. It has many advantages and is getting popularity in many areas. Video cameras evolve day by day so that the resolution improves sharply. It leads to the exponential growth of the produced data by the networked video cameras. We are coping with real big data when we have to deal with video image data which are produced by the good quality video cameras. A video surveillance system was not useful until we find the cloud computing. But it is now being widely spread in U-Cities since we find some useful methodologies. Video data are unstructured data thus it is not easy to find a good research result of analyzing the data with MapReduce. This paper presents an analyzing system for the video surveillance system, which is a cloud-computing based video data management system. It is easy to deploy, flexible and reliable. It consists of the video manager, the video monitors, the storage for the video images, the storage client and streaming IN component. The "video monitor" for the video images consists of "video translater" and "protocol manager". The "storage" contains MapReduce analyzer. All components were designed according to the functional requirement of video surveillance system. The "streaming IN" component receives the video data from the networked video cameras and delivers them to the "storage client". It also manages the bottleneck of the network to smooth the data stream. The "storage client" receives the video data from the "streaming IN" component and stores them to the storage. It also helps other components to access the storage. The "video monitor" component transfers the video data by smoothly streaming and manages the protocol. The "video translator" sub-component enables users to manage the resolution, the codec and the frame rate of the video image. The "protocol" sub-component manages the Real Time Streaming Protocol (RTSP) and Real Time Messaging Protocol (RTMP). We use Hadoop Distributed File System(HDFS) for the storage of cloud computing. Hadoop stores the data in HDFS and provides the platform that can process data with simple MapReduce programming model. We suggest our own methodology to analyze the video images using MapReduce in this paper. That is, the workflow of video analysis is presented and detailed explanation is given in this paper. The performance evaluation was experiment and we found that our proposed system worked well. The performance evaluation results are presented in this paper with analysis. With our cluster system, we used compressed $1920{\times}1080(FHD)$ resolution video data, H.264 codec and HDFS as video storage. We measured the processing time according to the number of frame per mapper. Tracing the optimal splitting size of input data and the processing time according to the number of node, we found the linearity of the system performance.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.1
• /
• pp.204-226
• /
• 2018

Big data processing applications have been migrated into cloud gradually, due to the advantages of cloud computing. Hadoop Distributed File System (HDFS) is one of the fundamental support systems for big data processing on MapReduce-like frameworks, such as Hadoop and Spark. Since HDFS is not aware of the co-location of virtual machines in the cloud, the default scheme of block allocation in HDFS does not fit well in the cloud environments behaving in two aspects: data reliability loss and performance degradation. In this paper, we present a novel location-aware data block allocation strategy (LDBAS). LDBAS jointly optimizes data reliability and performance for upper-layer applications by allocating data blocks according to the locations and different processing capacities of virtual nodes in the cloud. We apply LDBAS to two stages of data allocation of HDFS in the cloud (the initial data allocation and data recovery), and design the corresponding algorithms. Finally, we implement LDBAS into an actual Hadoop cluster and evaluate the performance with the benchmark suite BigDataBench. The experimental results show that LDBAS can guarantee the designed data reliability while reducing the job execution time of the I/O-intensive applications in Hadoop by 8.9% on average and up to 11.2% compared with the original Hadoop in the cloud.

• Convergence Security Journal
• /
• v.13 no.3
• /
• pp.17-23
• /
• 2013

Clustering is required to configure clustering interdependent structural technology. Clustering handles variable workloads or impede continuity of service to continue operating in the event of a failure. Long as high-availability clustering feature focuses on server operating systems. Active-standby state of two systems when the active server fails, all services are running on the standby server, it takes the service. This function switching or switchover is called failover. Long as high-availability clustering feature focuses on server operating systems. The cluster node that is running on multiple systems and services have to duplicate each other so you can keep track of. In the event of a node failure within a few seconds the second node, the node shall perform the duties broken. Structure for high-availability clustering efficiency should be measured. System performance of infrastructure systems performance, latency, response time, CPU load factor(CPU utilization), CPU processes on the system (system process) channels are represented.

• Journal of KIISE
• /
• v.42 no.4
• /
• pp.541-549
• /
• 2015

A large volume of continuously growing BGP data files can raise two technical challenges regarding scalability and manageability. Due to the recent development of the open-source distributed computing infrastructure, Hadoop, it becomes feasible to handle a large amount of data in a scalable manner. In this paper, we present a new Hadoop-based BGP tool (BGPdoop) that provides the scale-out performance as well as the extensible and agile analysis capability. In particular, BGPdoop realizes a query-based BGP record exploration function using Hive on the partitioned BGP data structure, which enables flexible and versatile analytics of BGP archive files. From the experiments for the scalability with a Hadoop cluster of 20 nodes, we demonstrate that BGPdoop achieves 5 times higher performance and the user-defined analysis capability by expressing diverse BGP routing analytics in Hive queries.

• KIISE Transactions on Computing Practices
• /
• v.21 no.10
• /
• pp.651-657
• /
• 2015

The amount of web traffic has increased as a result of the rapid growth of the use of web-based applications. In order to obtain valuable information from web logs, we need to develop systems that can support interactive, flexible, and efficient ways to analyze and handle large amounts of data. In this paper, we present CERES, a log-based, interactive web analytics system for backbone networks. Since CERES focuses on analyzing web log records generated from backbone networks, it is possible to perform a web analysis from the perspective of a network. CERES is designed for deployment in a server cluster using the Hadoop Distributed File System (HDFS) as the underlying storage. We transform and store web log records from backbone networks into relations and then allow users to use a SQL-like language to analyze web log records in a flexible and interactive manner. In particular, we use the data cube technique to enable the efficient statistical analysis of web log. The system provides users a web-based, multi-modal user interface.

• Journal of Internet Computing and Services
• /
• v.10 no.5
• /
• pp.41-47
• /
• 2009

Routing through a backbone, which is responsible for performing and managing multipoint communication, reduces the communication overhead and overall energy consumption in wireless sensor networks. However, the backbone nodes will need extra functionality and therefore consume more energy compared to the other nodes. The power consumption imbalance among sensor nodes may cause a network partition and failures where the transmission from some sensors to the sink node could be blocked. Hence optimal construction of the backbone is one of the pivotal problems in sensor network applications and can drastically affect the network's communication energy dissipation. In this paper a distributed algorithm is proposed to generate backbone trees through robust multi-hop clusters in wireless sensor networks. The main objective is to form a properly designed backbone through multi-hop clusters by considering energy level and degree of each node. Our improved cluster head selection method ensures that energy is consumed evenly among the nodes in the network, thereby increasing the network lifetime. Comprehensive computer simulations have indicated that the newly proposed scheme gives approximately 10.36% and 24.05% improvements in the performances related to the residual energy level and the degree of the cluster heads respectively and also prolongs the network lifetime.

• Journal of Korea Spatial Information System Society
• /
• v.7 no.2 s.14
• /
• pp.47-56
• /
• 2005

Recently, with the rapid development of mobile computing, wireless positioning technologies, and the generalization of wireless internet, LBS (Location Based Service) which utilizes location information of moving objects is serving in many fields. In order to serve LBS efficiently, the location data server that periodically stores location data of moving objects is required. Formerly, GIS servers have been used to store location data of moving objects. However, GIS servers are not suitable to store location data of moving objects because it was designed to store static data. Therefore, in this paper, we designed and implemented an extended SLDS(Short-term Location Data Subsystem) for real-time Location Based Services. The extended SLDS is extended from the SLDS which is a subsystem of the GALIS(Gracefully Aging Location Information System) architecture that was proposed as a cluster-based distributed computing system architecture for managing location data of moving objects. The extended SLDS guarantees real-time service capabilities using the TMO(Time-triggered Message-triggered Object) programming scheme and efficiently manages large volume of location data through distributing moving object data over multiple nodes. The extended SLDS also has a little search and update overhead because of managing location data in main memory. In addition, we proved that the extended SLDS stores location data and performs load distribution more efficiently than the original SLDS through the performance evaluation.

• Journal of Korea Spatial Information System Society
• /
• v.8 no.2 s.17
• /
• pp.91-107
• /
• 2006

To handling the extreme situation that must manage positional information of a very large volume, at least millions of moving objects. A cluster-based sealable distributed computing system architecture, called the GALIS which consists of multiple data processors, each dedicated to keeping records relevant to a different geographical zone and a different time zone, was proposed. In this paper, we proposed a valid time management and time-zone shifting scheme, which are essential in realizing the long-term location data subsystem of GALIS, but missed in our previous prototype development. We explain how to manage valid time of moving objects to avoid ambiguity of location information. We also describe time-zone shifting algorithm with three variations, such as Real Time-Time Zone Shifting, Batch-Time Zone Shifting, Table Partitioned Batch-Time Zone Shifting, Through experiments related with query processing time and CPU utilization, we show the efficiency of the proposed time-zone shifting schemes.

• Journal of Korea Spatial Information System Society
• /
• v.10 no.3
• /
• pp.79-91
• /
• 2008

Recently, with highly Interest In Location-Based Service(LBS) utilizing location data of moving objects, the GALIS(Gracefully Aging Location Information System) which is a cluster-based distributed computing architecture was proposed as a more efficient location management system of moving objects. In the SLDS(Short-term location Data Subsystem) which Is a subsystem of the GALIS, since the SDP(Short-term Data Processor) Master transmits current location data and queries to every SDP Worker, the SDP Master reassembles and sends query results produced by SDP Workers to the client. However, the services are suspended during the SDP Master under failure and the response time to the client is increased if the load is concentrated on the SDP Master. Therefore, in this paper, the extended SLDS was designed and implemented to solve these problems. Though one SDP Master is under failure, the other can provide the services continually, and so the extended SLDS can guarantee the high reliability of the SLDS. The extended SLDS also can reduce the response time to the client by enabling two SDP Masters to perform the distributed query processing. Finally, we proved high reliability and high availability of the extended SLDS by implementing the current location data storage, query processing, and failure takeover scenarios. We also verified that the extended SLDS is more efficient than the original SLDS through the query processing performance evaluation.