Hybrid in-memory storage for cloud infrastructure

Abstract

Modern cloud computing is rapidly changing from traditional hypervisor-based virtual machines to container-based cloud-native environments. Due to limitations in I/O performance required for both virtual machines and containers, the use of high-speed storage (SSD, NVMe, etc.) is increasing, and in-memory computing using main memory is also emerging. Running a virtual environment on main memory gives better performance compared to other storage arrays. However, RAM used as main memory is expensive and due to its volatile characteristics, data is lost when the system goes down. Therefore, additional work is required to run the virtual environment in main memory. In this paper, we propose a hybrid in-memory storage that combines a block storage such as a high-speed SSD with main memory to safely operate virtual machines and containers on main memory. In addition, the proposed storage showed 6 times faster write speed and 42 times faster read operation compared to regular disks for virtual machines, and showed the average 12% improvement of container's performance tests.

1. Introduction

In-memory computing technology has begun to emerge again from the requirement for data storage, which is the biggest bottleneck in processing big data and artificial intelligence systems[1], [2]. Unlike conventional disk-based computing, in-memory computing does not store and manage data on a hard disk, but means loading and using the entire data in memory [3].

A typical example of in-memory computing is well-known as a caching system in a memory hierarchy. Due to the recent development of cloud and edge computing, various approaches using memory are being implemented, and as a technology suitable for the current cloud and edge computing technology, its utilization is rapidly increasing [4].

For example, technologies such as in-memory data grids [5], [6] and in-memory databases [7], [8], [9], [10] are developing, but most of the in-memory computing services are currently in their infancy, and some companies also provide streaming engines, machine learning, distributed processing, SQL, using in-memroy computing and provides an platform which is integrated with above functions.

When using disk storage or SSDs in a cloud computing, if the number of virtual machines or containers per physical machine increases, the system load is concentrated on the storage with the virtual disks, and performance deteriorates. This technology can lead to I/O bottlenecks and performance degradation in cloud environments. Therefore, virtual machines or containers used in cloud computing can be stored in main memory to improve performance [11], [12]. And the host utilizes main memory using a memory-based temporary filesystem or storage (such as tmpfs) for containers [13].

However, this technique has disadvantages in that it cannot store large amount of data due to the limited capacity of the main memory and it is difficult to overcome the volatile characteristics of the memory. Of course, in the case of Optane memory recently announced by Intel [14], it can be a good example to improve performance and overcome the shortcomings of memory device. However, there are still problems in terms of price and capacity for the commercialization. Therefore, the above-mentioned limitations and disadvantages should be overcome in order to implement in-memory computing using the main memory in spite of the high cost and resource constraints.

In this paper, data storage and operation using main memory are supported for high-speed data access when a customer uses a virtual machine or container environment, and as the user creates data, the memory capacity increases rapidly to reduce the total memory capacity. Introduce research and development to solve the problem of excess and eliminate data loss due to the volatile nature of memory.

In this paper, we propose a hybrid in-memory storage that integrates main memory (e.g. RAM) and storage (e.g. SSD) to solve the drawbacks of main memory due to capacity limitation and volatile characteristics for in-memory computing. It implements hybrid in-memory storage by uniting main memory and storage into a single storage view, and provides high-performance virtual environment and large-capacity storage for cloud services. We also want to apply hybrid in-memory storage to our cloud infrastructure to utilize it for high performance. Section 1 provides an overview of hybrid in-memory storage and section 2 describes the detailed structure and architecture and operation. Section 3 describes the virtual machine and container applications and experimental results, and section 4 provide the further studies. At last, section 5 make conlusion.

2. Overview of Hybrid In-memory Storage

Hybrid in-memory storage is a virtual block storage device that uses a specific unit (in gigabytes) of memory as a RAM disk and backs up data either synchronously or asynchronously to a high-performance non-volatile block device such as an SSD. Figure 1 shows the overall structure of the hybrid in-memory storage proposed in this white paper.

(Figure 1) Structure of hybrid in-memory storage

The entire system consists of a hybrid in-memory storage combined with virtual disks, a hybrid in-memory storage engine, main memory (RAM) and disk storage (SSD). Virtual machines create and operate virtual images (eg OS images, disk images, etc.) [15] on hybrid in-memory storage. And for containers, place the container filesystem on a virtual disk on hybrid in-memory storage. The hybrid in-memory storage engine creates a single storage and combines main memory and disk storage into a single storage for single view of storage. Hybrid in-memory storage also provides standard block storage I/O interfaces, allowing existing virtual machines to operate without modification.

Virtual disk access commands created by virtual machines or containers are forwarded to hybrid in-memory storage. Because the hybrid in-memory storage engine combines main memory and disk storage to create hybrid in-memory storage, the hybrid in-memory storage engine selects the integrated main memory and disk storage to process disk access commands. In addition, hybrid in-memory storage can significantly reduce the physical virtual desktop image by using a file system with deduplication for limited memory operations.

2.1 Architecture of Hybrid In-memory Storage Engine

Figure 2 shows the architecture of a hybrid in-memory storage engine. The hybrid in-memory storage engine consists of a hybrid storage interface module, a hybrid storage deployment module, a RAM access (main memory) control module, a RAM-based storage generator, and a disk access control module.

(Figure 2) Structure of hybrid in-memory storage engine

The hybrid in-memory storage interface module provides a standard block interface and receives virtual disk access commands created by virtual machines or containers. Received commands are forwarded to the hybrid in-memory storage distribution module. According to the nature of the command, the hybrid storage distribution module determines whether to use main memory or disk storage to perform the command, and sends the command to the RAM access control module or the disk access control module.

The RAM access control module uses main memory as a disk to process disk access commands to provide high-speed access. The main memory storage creation module performs the actual read/write operations on main memory that can be accessed on a address basis with disk access commands sent in blocks. This allows the data from the virtual disk to be stored in main memory. The disk storage control module uses disk storage to process virtual disk access commands.

2.2 Operation of Hybrid In-memory Storage

Figure 3 shows how hybrid in-memory storage initially works. When the system starts up, the data in the storage is restored to the main memory (RAM disk) of the hybrid in-memory storage.

(Figure 3) The operation of hybrid in-memrory storage

Even during restoration, the hybrid in-memory storage engine can continue service without interruption. After restoration, the hybrid in-memory storage engine performs read/write (backup) operations. In synchronous mode, all data transferred to the hybrid in-memory storage engine is simultaneously stored on RAM disk and SSD. When the save to both devices is complete, the write to the upper layer is complete. However, in asynchronous mode, data transferred to the hybrid in-memory storage engine is only stored on the RAM disk. A kernel thread inside the hybrid in-memory storage engine stores data from RAM to the SSD in Least Recent Written (LRW) order.

In figure 3, hybrid in-memory storage supports a storage expansion to overcome the limitation of memory capacity. As described above, two types of disk image operation (in-memory operation and disk operation ) are supported and used by merging them. As shown in the figure 3, when the memory capacity is full, newly created data is written to Block device 1-2 without writing to the main memory, and the data written to Block device 1-2 is read in the disk operation, not in the main memory. The data written to Block device 1-1 is operated through the existing in-memory operation.

Figure 4 shows how to configure hybrid in-memory storage with RAM and block storage. Hybrid in-memory storage provides a standard block storage device format to provide users with a common interface and physically map the RAM disk area at the front and the disk (SSD) area at the back of the hybrid in-memory storage logically.

(Figure 4) Logical mapping blocks.

If there is RAM blocks with n block IDs and a disk (SSD) blocks with m block IDs, then block ID 1 to n of RAM are mapped to block ID 1 through n of hybrid in-memory storage to combine the two regions. Block ID 1 to m disks (SSDs) are mapped to hybrid in-memory storage with block ID n+1 to n+m.

And a storage boundary is established between the hybrid in-memory storage block ID n and n+1. When a disk access command is received, the block ID of the command is checked. If the block ID is less than or equal to the storage boundary, the command is sent to the RAM access control module and the command is processed using high-speed main memory. If the block ID is greater than the storage boundary, the command is passed to the disk access control module, where it can be processed in large amounts of disk storage.

2.3 Read/Write

There are two mode in read/write operation.

In synchronous mode, read operations are independent of the SSD. Internally, it is only processed from kernel memory in hybrid in-memory storage. The DMA engine [16] transfers data from hybrid in-memory storage to user buffers. In other words, the DMA Engine runs in memory without copying data. In synchronous mode, write operations are simultaneously written to main memory and SSD. The requested write is finally completed only when the DMA copy has completed and the SSD write has also completed. All data movement is handled by DMA.

In asynchronous mode, all write operations are performed in kernel memory, and a completion message is returned when the write operation is completed in kernel memory. Data written to the memory is later written to the SSD according to the delayed write method. Even if the write speed of the SSD is slow, higher write requests are not blocked, so the final write speed will be similar to the speed of the RAM disk.

2.4 De-duplication

Read/Write commands to hybrid in-memory storage are passed to the hybrid in-memory storage engine, and then passed to memory through deduplication [17]. In the case of a virtual machine, memory loss due to the size of the OS image can be very large. To solve this problem, a deduplication function is provided. The deduplication function of hybrid in-memory storage deduplicates the image of the virtual machine and reduces it so that it can be stored in main memory. Deduplication of the virtual machine image is performed whenever a write operation to the virtual machine image occurs. When a write command to disk occurs inside the virtual machine, it is sent to the deduplication module in the host as shown in Figure 1. Therefore, the deduplication function performs the deduplication function by capturing the write event that occurs during image operation of the virtual machine in real time. After calculating the required fingerprint, it is stored in the DB. After examining the DB, if the same block to be written exists, only the metadata necessary for writing is recorded and the write operation is completed. If not, write the write block to that memory and store the metadata at the same time.

3. Applied Tests & Results

3.1 Management of Hybrid In-memory Storage

Hybrid in-memory storage is installed in the form of a kernel module. To facilitate installation and management, it is installed in the Linux kernel's Proc file system and managed through it as shown in figure 5. It is provided in the form of /proc / as shown below. And you can check various information in /proc/sys/dev// /. Also, GUI or CLI management tools are provided for user convenience. It is designed to utilize this tool to check, create, delete and restore status

(Figure 5) Proc filesystem view for hybrid in-memeory storage.

3.2 Test Environments

The architecture of the two systems for applying hybrid in-memory storage is shown in Figure 6.

(Figure 6) VM & container(docker) application ((a)VM and (b) container)

The platform for applying the virtual machine system was configured using Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz 64Core, CentOS 7.0, qemu-kvm hypervisor [18]. The virtual machine's operating system was configured as window10 and installed on hybrid in-memory storage. Virtual machines run on hybrid in-memory storage with the entire operating system image. These operating system images are available in sizes ranging from tens to hundreds of gigabytes, and user disks are also provided in divided portions. Deduplication is essential to provide these disks in memory space. In the case of operating system images, the same image used by default is provisioned through the clone method as the primary base. So we were able to reduce the 30G operating system to 512M.

For testing the container system, a container (docker)[19] creates a hybrid in-memory container storage, and configures the storage volume of the container on this storage. A container creates and operates a file system (eg/var/lib/docker for docker) volume where the container runs on hybrid in-memory container storage.

The container file system is composed of an unifying file system (eg overlayfs [20]) and is composed of layers. It consists of a merged access area, a container layer, and an image layer. Each layer operates by creating and mounting a specific directory on the in-memory container storage.

The container layer is a writable layer, and each container is created on the top layer, allowing each container to have its own state. After the container is created, all changes are made in this layer. However, the R/W speed is fast because it is done in memory. And for the efficiency of file management, the difference information between the actual image and the container image is included.

The image layer is a read-only layer that can be shared with other containers. In addition, multiple images shared with other layers can be operated in the container layer. And the integrated access area includes link information of the layer accessible to all file systems of the container layer and the image layer and is shared with users. This allows access to the file. The image layer can be shared with many different systems to increase its efficiency. Container images in the image layer should be pulled from public repositories (eg github [21]) when containers are deployed. In this case, it is efficient to store the image used in the container system locally or to bring it in advance to ensure performance. In this paper, the deployment speed is also fast because the already pooled image exists on the hybrid in-memory disk. In this paper, we tested using docker's overlayfs and proceeded without performing deduplication on the system.

(Table 1) Testing environments for hybrid in-memory storage

3.3 Test Results

3.3.1 Test for virtual machine

In a virtual machine system, the most commonly used crystal disk mark [22] for storage testing was installed and tested on the OS (window 10) of the virtual machine. For the Seq Q32T1, it refers to the speed when sequentially reading and writing files in the queue, i.e. the speed when reading and writing a size of 128 KiB with 32 instructions.

4K Q32T1 for Windows operating system. Since it uses 4 KB as a cluster on an NTFS system, which is the usual format, we measure the speed while randomizing the queue and the number of threads respectively.

For CrystalDiskMark, the Seq Q32T1 is considered the maximum for read/write speed and the 4K Q32T1 is considered the actual read/write speed. These experiments were tested using a benchmark tool (CrystalDiskMark) and are the result of 10 periodic tests for each read/write operation as shown in figure 7. The measured results were used to obtain the average of the two benchmark results. Average values ​​are given in Table 2.

(Table 2) The average Speed (MB/S) of Read/write operation

* Seq Q32T1: Sequential (Block Size=128KiB) Read/Write [Queue:32/Thread:1]

** 4K Q32T1: Random 4KiB Read/Write [Queue:32/Thread:1]

(Figure 7) The benchmark results for virtual machine. (Crystal disk Mark)

Comparing to the average values ​​as shown in Table 2, it can be seen that the two tested benchmark cases have significantly improved performance compared to normal storage. But more importantly, it's a performance boost. The 4K Q32T1 has a better performance boost than the Seq Q32T1, with performance gains of 42x for read operations and 6x for write operations. Due to the nature of Windows systems and asynchronous writes using DMA, it has been found that write operations provide less performance improvement than read operations

3.3.2 Test for container

For the test of the container system, kubernetes [23] was used, and the test was conducted by dividing a single master node, a node equipped with an SSD, and a node equipped with a hybrid in-memory storage. And, Nginx [24] image, which is the most used web server application on hybrid in-memory storage, was deployed as shown in figure 8.

(Figure 8) The test for nginx deployment in kubernetes

When the deployed image is first configured, the image is pulled through github and distributed on the hybrid in-memory storage from the completed image when redistributed. The test results are shown in Table 3 below and the test results showed a performance improvement of about 30%.

(Table 3) Nginx deployment test result (sec)

As a second test, we conducted a web load test of the deployed nginx as shown in Table 4. It was conducted through the ab test of httpd-tools [25], and ab is a benchmarking tool for the performance of the Apache HTTP server. In the test, 500 simultaneous connections to 5000 requests were repeated 50 times. As shown in the table, as a result of the test, the performance of nginx web showed a performance improvement of about 30% in the comparison of the maximum value, and an average performance of 50 times showed a performance improvement of about 12%.

(Table 4) Nginx load test result

4. Further Researches

We have tests to apply virtual machines and container environments, which are often used in cloud computing, to in-memory computing. As mentioned above, use cases for the utilization of volatile memory in in-memory computing are limited to the extent of databases.. In the future, this system will maximize the stability for applying and commercializing in-memory computing technology to edge computing by developing storage utilizing main memory by software. Also, large distrubuted system such as edge computing and cloud computing clould be needed to develop memory to share for fast data transaction or high avalabilty [26]. Another research direction is to develop a high-speed technologies that is more convenient and easier than using technologies using high speed memory devices such as PMEM device currently being developed.

5. Conclusion

In this paper, we developed a hybrid in-memory storage for stability and high performance of cloud computing infrastructure and tested it by applying a hypervisor-based virtual machine system and a container-based cloud-native environment. Hybrid in-memory storage combines the characteristics of main memory and general storage (SSD) to overcome the disadvantages of data loss due to the volatile characteristics of memory, and virtual This has resulted in improved performance of machines and containers. In the case of the presented virtual machine system, the OS image, which has the greatest influence on the speed of the virtual machine, is stored in the main memory so that high-speed virtual machines can be executed. This allows the container image to run in main memory. When a virtual machine is deployed, users continue to create data while using the virtual machine, and when this exceeds the main memory capacity, the data is stored on disk storage. Additionally, for container file systems, image data due to continuous pooling is stored on disk storage. Because user data is stored on disk storage, file access speed is slower than in-memory, but suitable for efficient operation and storage of large data. Therefore, it is stored in the main memory to support high performance of the cloud infrastructure, and large user data is stored in disk storage to overcome the capacity limit of the main memory, so that an efficient cloud environment can be built. Also, due to the volatile nature of main memory, it can reasonably cope with backup/restore.