• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.11
• /
• pp.5234-5251
• /
• 2018

It is well-known that virtualization technology can bring many benefits not only to users but also to service providers. From the view of system security and resource utility, higher resource sharing degree and higher system reliability can be obtained by the introduction of virtualization technology in distributed cloud. The small size time-sharing multiplexing technology which is based on virtual machine in distributed cloud platform can enhance the resource utilization effectively by server consolidation. In this paper, the concept of memory block and user satisfaction is redefined combined with user requirements. According to the unbalanced memory resource states and user preference requirements in multi-virtual machine environments, a model of proper memory resource allocation is proposed combined with memory block and user satisfaction, and at the same time a memory optimization allocation algorithm is proposed which is based on virtual memory block, makespan and user satisfaction under the premise of an orderly physical nodes states also. In the algorithm, a memory optimal problem can be transformed into a resource workload balance problem. All the virtual machine tasks are simulated in Cloudsim platform. And the experimental results show that the problem of virtual machine memory resource allocation can be solved flexibly and efficiently.

• Journal of Platform Technology
• /
• v.11 no.4
• /
• pp.71-79
• /
• 2023

In this paper, we analyze the characteristics of machine learning workloads and, based on them, propose a distributed in-memory caching technique to improve the performance of machine learning workloads. The core of machine learning workload is model training, and model training is a computationally intensive task. Performing machine learning workloads in a Kubernetes-based cloud environment in which the computing framework and storage are separated can effectively allocate resources, but delays can occur because IO must be performed through network communication. In this paper, we propose a distributed in-memory caching technique to improve the performance of machine learning workloads performed in such an environment. In particular, we propose a new method of precaching data required for machine learning workloads into the distributed in-memory cache by considering Kubflow pipelines, a Kubernetes-based machine learning pipeline management tool.

• International Journal of Advanced Culture Technology
• /
• v.9 no.3
• /
• pp.321-326
• /
• 2021

In a multi-cloud environment, it is necessary to minimize physical movement for efficient interoperability of distributed source data without building a data warehouse or data lake. And there is a need for a data platform that can easily access data anywhere in a multi-cloud environment. In this paper, we propose a new platform based on data fabric centered on a distributed platform suitable for cloud environments that overcomes the limitations of legacy systems. This platform applies the knowledge graph database technique to the physical linkage of source data for interoperability of distributed data. And by integrating all data into one scalable platform in a multi-cloud environment, it uses the holochain technique so that companies can easily access and move data with security and authority guaranteed regardless of where the data is stored. The knowledge graph database mitigates the problem of heterogeneous conflicts of data interoperability in a decentralized environment, and Holochain accelerates the memory and security processing process on traditional blockchains. In this way, data access and sharing of more distributed data interoperability becomes flexible, and metadata matching flexibility is effectively handled.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.42 no.6
• /
• pp.19-28
• /
• 2005

The multiprocessor architecture is a good method to improve the computer system performance. The CC-NUMA provides a single shared space with the physically distributed memories is used widely in the multiprocessor computer system. A CC-NUMA has the full-mapped directory for the shared memory md uses a remote cache memory for tile fast memory access. In this paper, we propose a processing node architecture for a CC-NUMA system and a cache coherency protocol on the physically distributed but logically shared system. We show an implementation result of the system which is adopted the cache coherency protocol.

• Journal of Advanced Navigation Technology
• /
• v.18 no.4
• /
• pp.401-408
• /
• 2014

Hadoop is composed with MapReduce programming model for distributed processing and HDFS distributed file system. Hadoop is suitable framework for big data processing, but processing of mass small files have many problems. The processing of mass small file in hadoop have problems to created one mapper per one file, and it have problems to needed many memory for store of meta information of file. This paper have comparison evaluation processing method of mass small file with various method in hadoop platform. The processing of general compression format is inadequate because of processing by one mapper regardless of data size. The processing of sequence and hadoop archive file is removed memory problem of namenode by compress and combine of small file. Hadoop archive file is faster then sequence file about combine time of small file. The processing using CombineFileInputFormat class is needed not combine of small file, and it have similar speed big data processing method.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.16 no.4
• /
• pp.153-161
• /
• 2021

Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

• Journal of KIISE
• /
• v.43 no.6
• /
• pp.621-626
• /
• 2016

Recursive query algorithm is used in many social network services, e.g., reachability queries in social networks. Recently, the size of social network data has increased as social network services evolve. As a result, it is almost impossible to use the recursive query algorithm on a single machine. In this paper, we implement recursive query on two popular in-memory distributed platforms, Spark and Twister, to solve this problem. We evaluate the performance of two implementations using 50 machines on Amazon EC2, and real-world data sets: LiveJournal and ClueWeb. The result shows that recursive query algorithm shows better performance on Spark for the Livejournal input data set with relatively high average degree, but smaller vertices. However, recursive query on Twister is superior to Spark for the ClueWeb input data set with relatively low average degree, but many vertices.

• Smart Structures and Systems
• /
• v.18 no.4
• /
• pp.663-682
• /
• 2016

Continuum manipulators as a kind of mechanical arms are useful tools in special robotic applications. In medical applications, like colonoscopy, a maneuverable thin and flexible manipulator is required. This research is focused on developing a basic module for such an application using shape memory alloys (SMA). In the structure of the module three wires of SMA are uniformly distributed and attached to the circumference of a flexible tube. By activating wires, individually or together, different rotation regimes are provided. SMA model is used based on Brinson work. The SMA model is combined to model of flexible tube to provide a composite model of the module. Simulating the model in Matlab provided a platform to be used to develop controller. Complex and nonlinear behavior of SMA make the control problem hard especially when a few SMA actuators are active simultaneously. In this paper, position control of the two degree of freedom module is under focus. An experimental control strategy is developed to regulate a desired position in the module. The simulation results present a reasonable performance of the controller. Moreover, the results are verified through experiments and show that the continuum module of this paper would be used in real modular manipulators.

• Journal of Korea Multimedia Society
• /
• v.23 no.2
• /
• pp.241-246
• /
• 2020

Hadoop is a framework to process large data sets in a distributed way across clusters of nodes. It has been a popular platform to process big data, but in recent years, other platforms became competitive ones depending on the characteristics of the application. Spark is one of distributed platforms to enable real-time data processing and improve overall processing performance over Hadoop by introducing in-memory processing instead of disk I/O. Whereas Hadoop is designed to work on Java and data analysis is processed using Java API, Spark provides a variety of APIs with Scala, Python, Java and R. In this paper, the goal is to find out whether the APIs of different programming languages af ect the performances in Spark. We chose two popular APIs: Python and Scala. Python is easy to learn and is used in AI domain in a wide range. Scala is a programming language with advantages of parallelism. Our experiment shows much faster processing with Scala API than Python API. For the performance issues on AI-based analysis, further study is needed.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2009.01a
• /
• pp.298-302
• /
• 2009

This paper proposes a single sign-on scheme in which a mobile user offers his credential information to a home network running the OSGi (Open Service Gateway Initiative) service platform, to obtain user authentication and control a remote device through a mobile device using this authentication scheme, based on SAML (Security Assertion Markup Language). Especially by defining the single sign-on profile to overcome the handicap of the low computing and memory capability of the mobile device, we provide a clue to applying automated user authentication to control a remote device via a mobile device for distributed mobile environments such as a home network based on OSGi.