Deep Learning based Loss Recovery Mechanism for Video Streaming over Mobile Information-Centric Network

Abstract

Mobile Edge Computing (MEC) and Information-Centric Networking (ICN) are essential network architectures for the future Internet. The advantages of MEC and ICN such as computation and storage capabilities at the edge of the network, in-network caching and named-data communication paradigm can greatly improve the quality of video streaming applications. However, the packet loss in wireless network environments still affects the video streaming performance and the existing loss recovery approaches in ICN does not exploit the capabilities of MEC. This paper proposes a Deep Learning based Loss Recovery Mechanism (DL-LRM) for video streaming over MEC based ICN. Different with existing approaches, the Forward Error Correction (FEC) packets are generated at the edge of the network, which dramatically reduces the workload of core network and backhaul. By monitoring network states, our proposed DL-LRM controls the FEC request rate by deep reinforcement learning algorithm. Considering the characteristics of video streaming and MEC, in this paper we develop content caching detection and fast retransmission algorithm to effectively utilize resources of MEC. Experimental results demonstrate that the DL-LRM is able to adaptively adjust and control the FEC request rate and achieve better video quality than the existing approaches.

1. Introduction

The emerging video streaming applications (virtual reality, 3D online games, ultra high definition TV, smart environment, etc.) pose new challenges to the wireless access networks [1-5]. According to Cisco’s report, the mobile video traffic will continuously grow and will reach 78% of total mobile data traffic by 2021 [6]. In order to fulfill the new demand, Mobile Edge Computing (MEC) [7] and Information-Centric Networking (ICN) [8] are proposed as primary network architectures for the future mobile Internet.

The MEC shifts the cloud computing to the edge of the network. The original cloud computing as general IT infrastructure adopts a centralized structure which is comprised by acluster of high performance computing resources. With the network virtualization and service-oriented architecture, the cloud computing platform can flexibly provide various types of services. However, the rapid development of the mobile devices and the Internet of Things (IoT) leads to the further explosion of mobile data traffic. The huge amount of data generated by the end hosts greatly increases the workload of the core network and backhaul [9][10]. To solve this problem, the MEC decentralizes the original cloud computing resources to the edge of the network. Due to the distributed structure, the MEC is able to achieve the computing of floading, traffic offloading, and support context-aware localized services [11].

The main goal of ICN is to transfer the host-to-host communication scheme used by TCP/IP protocols to content centric model [12]. The rationale behind the content centric model is that current Internet users mostly concentrate on information consumption, for example, YouTube, Instagram and Facebook, and the traditional host-to-host scheme is inefficient for content dissemination. In ICN protocols, named content is the core element and the name is the key property for packet processing (routing, caching, forwarding, etc.) [8]. In addition, every node along the transmission path is able to temporally store the content in its local storage which is called in-network caching functionality. As a result, if an intermediate node happens to cache a requested content, it can stop forwarding the request and send the content back. The content centric model of ICN can efficiently reduce duplicated transmission and improve the network utilization [13].

Despite the advance in mobile and wireless communications, the packet loss in wireless network environments still affects the video streaming performance [14-16]. The existing approaches mainly adopt Forward Error Correction (FEC) [17-18] methods to overcome packet loss. FEC packets are generated at the video server and transmitted through wired and wireless links to the video clients. However, a majority of packet losses are arisen from wireless link errors, for instance, signal interference, collision, signal fading. As the growth of video content, delivering redundant FEC packets causes huge waste of network resources. Based on the architectural convergence of MEC and ICN, in this paper we propose a Deep Learning based Loss Recovery Mechanism (DL-LRM) for video streaming to overcome wireless packet loss. The DL-LRM consists of deep Q-learning based FEC control algorithm and content caching detection based retransmission algorithm [19-21]. In difference to existing approaches, we utilize the computing functionality provided by the MEC to produce FEC packets. Transmission of FEC packets at the edge of the network can dramatically reduce the workload of core network and backhaul. In addition, with monitoring wireless link statues, the deep Q-learning based FEC control algorithm adjusts the FEC request rate to avoid unnecessary transmission. Although the cache capability of the edge cloud reduces the transmission latency, the edge cloud cannot store enormous amount of request video content and the ICN protocols isolate the location that content is stored. With considering the characteristics of MEC and ICN, we design a content caching detection based retransmission algorithm to make the best of MEC resources. The proposed algorithm monitors the outcomes of Interest packets and apply likelihood ratio test to detect cache missing. According to the detection result, our proposed algorithm adaptively set the retransmission timer for recovering lost packets.

The remainder of this paper is organized as follows: Section 2 describes the background and related work of MEC and ICN. Section 3 presents our proposed DL-LRM. Section 4 discusses experiment settings and simulation results. Finally, Section 5 describes our future work and concludes this paper.

 

2. Related Work

As huge mobile and IoT devices connected to the Internet, the core network and wireless backhaul face enormous data transmission pressure [22]. In addition, emerging applications such as augmented reality, self-driving cars and e-Health set higher requirements for the network infrastructure. The solution suggested by the MEC is to bring cloud computing platform to the wireless access network, and offload computing and storage tasks to the edge cloud platform [23]. The goal of MEC is to alleviate core network workload, improve network performance and user experience by using localized content distribution, computing of floading, data offloading and context-aware computing [23]. The MEC has received extensive attention from academia and industry. The European Telecommunications Standards Institute (ETSI) established a working group in 2014 to promote the standardization of MEC. In 2010, Cuervo et al. at Duke University first proposed a computational of floading prototype system that enabled the migration of running code to the edge cloud for execution [24]. Aiming at the problem of computing resource allocation, Zhao et al. proposed a joint optimization algorithm for allocation of computing resources and communication resources [25]. By migrating high-complexity and high-energy computing tasks from mobile terminals to the edge cloud, the workload and energy consumption of mobile terminals can be greatly reduced [24]. Liu et al. applied the stochastic geometry theory to propose a mobility-aware cache model for MEC based small cell networks [26]. Rebecchi et al. analyzed the content distribution pattern and proposed a delay guaranteed data offloading scheme for cellular networks [27].

Although the TCP/IP protocols have achieved great success, with the continuous advancement of computer and communication technologies, the scope and scale of Internet applications have undergone fundamental changes, far beyond the intention of original design. The information centric communication model has been proposed to adapt the new changes [28]. The major ICN projects include the Data Oriented Network Architecture (DONA) at University of California, Berkeley, the Content Centric Networking (CCN) at Xerox Palo Alto Research Center, the Publish/Subscribe for Internet Routing Paradigm (PSIRP) at Technical University of Helsinki and Named Data Networking (NDN) at the University of California, Los Angeles [28]. Jacobson et al. originally proposed the architecture of the information centric communication architecture, routing protocols and network security models [29]. Using in-network caching functionality, Katsaros et al. designed an overlay based multicast transmission scheme [30]. Jacobson et al. developed a prototype of voice communication system to verify the feasibility of the ICN architecture for multimedia applications [31]. Bai et al. applied the information centric architecture to the vehicular ad-hoc networks and proposed various new types of network applications [32]. For the problem of video packet loss in wireless ICN, Han et al. proposed a adaptive FEC algorithm [19]. However, the proposed algorithm in [19] used video server to produce FEC packets and did not exploit the advantages of the MEC.

 

3. Edge-Assistant Loss Recovery Mechanism

 

3.1 Video Streaming Scheme for MEC based Information-Centric Network

According to the MEC network architecture, a video streaming scheme is designed as presented in Fig. 1. The video delivery platform consists of three layers and the communications between each layer follows ICN protocols.

 

Fig. 1. Video Streaming Scheme for MEC based Information-Centric Network

In application layer, the video server performs raw video compression and packetization. The ICN protocols use two types of packets, Interest and Data packets. Since the ICN adopts the receiver-driven transmission model, the generated video packets will be stored in the video server's buffer instead of pushing to the video clients. Only when the interest packets are received, the video server fetches the matching data packets and delivers to the video clients. Due to the in-networking caching functionality of ICN, if any forwarding node along the transmission route caches the requested data packet, the forwarding node can directly send it back to the video client without further delivering the request to the video server. Therefore, the video server has no need to trace the status of each video streaming session. The video streaming applications usually involve a large number of users, and the in-networking caching functionality is able to tremendously reduce the transmission pressure of the video server.

The core network layer includes major communication facilities, such as control and management cloud, switches, gateways, routers and high-speed wired links to interconnect the wireless access networks. The control and management cloud coordinates each communication facilities and provides the functions of authentication, transmission policy control, resource management, quality of service provision, and network virtualization. As shown in Fig. 1, the access network layer is composed of base stations and mobile nodes. Each base station is associated with an edge cloud platform, which is the main architectural difference of MEC from the conventional access network. To take full advantage of computing and storage resources of MEC, the edge cloud platform in our proposed scheme performs FEC encoding, video data cache and FEC transmission control and the mobile node is responsible for interest packet scheduling, lost packet recovery and video decoding as represented in Fig. 2.

 

Fig. 2. Interest and Data Packets Transmission in MEC based Information-Centric Network

An ICN node has three primary data structures, Content Store (CS), Pending Interest Table (PIT) and Forwarding Information Base (FIB). CS is a local storage to support in-network caching functionality and FIB is a routing table that guides the transmission of interest packets. When sending an Interest packet, the ICN node logs corresponding route information as a PITrecord. There is a timer associated with each PIT record. If the timer is expired and the data packet is not received, the forwarding node just removes the PIT record. The end-host node can choose actions (resend the Interest packet, remove the PIT record, send FEC packets, etc.) based on the application policies. In ICN protocols, the timer value is denoted as lifetime. The video streaming applications are delay sensitive, therefore the lifetime setting is critical for recovering the packet loss.

 

3.2 Loss Recovery Model of MEC based Information-Centric Network

Supported by the MEC server, the base station plays a role of both wireless router and general ICN node with high computing and storage resources. In MEC-ICN, the base station continuously keeps track of the video transmission. After delivering a group of video packets, the MEC server uses the error correction code to generate FEC packets. The video and generated FEC packets are stored in the local buffer of the MEC server. In accordance with video decoding rate and the number of lost packets, the mobile node sends the requests for video and FEC packets.

Packet losses over wireless links possess burst property. For example, if there is error occurred during transmission of packet with sequence number i. The next packet with sequence number i + 1 has high probability to have error again. Therefore, we can use the Markov decision process (S, A, P, R, γ) to model loss recovery procedure. Let us define one step as round-trip transmissions of requests and responses for one group of video and FEC packets. At each step t, the mobile node observes the transmission state s_t= (T_video, T_FEC, R_video,R_FEC, D), and S is the state set of all transmission states, where s_t ∈ S. T_video and T_FEC are the number of interest packets for video and FEC respectively. Similarly, R_video and R_FEC are corresponding to the number of received data packets for video and FEC. D is the number of decodable packets for the group.

The number of Interest packets for FEC that the mobile node chooses to recover lost packets is defined as action A = {x∈Z |0 ≤ x ≤ FEC_max}. P is the state transition probability from current state st to next state s_t+1. The immediate reward of the mobile node taking action a_t is

 

\(r_{t}=\left\{\begin{array}{ll} \frac{T_{F E C}-R_{F E C}}{D}, & \text { if } D=T_{v i d e o} \\ \frac{R_{v i d e o}-T_{v i d e o}}{D}, & \text { if } D       (1)

 

The rationale of equation 1 is to measure the effectiveness of FEC packets for loss recovery. If the whole group of video packets can be decoded, then the immediate reward is non-negative. However, if certain packets cannot be decoded, then the immediate reward is negative. Considering effect of action a_t after n steps, we define the accumulated reward as

 

\(R_{t}=\sum_{i=t}^n r_{i} \gamma^{i-t}\)       (2)

 

where γ is a decay factor 0 < γ ≤ 1.

 

3.3 Deep Q-Learning based FEC Control Algorithm

Unlike TCP/IP networks, the mobile nodes, in ICN, need to explicitly send requests for FEC to overcome lost packets. Since the transmission of FEC packets occupies extra network band width, it is important to adjust the number of requests according to network condition variations. Following the video Streaming Scheme of MEC-ICN, we propose a Deep Q-Learning based FEC control algorithm to regulate FEC requests. The mobile node works as agent, and the MEC-ICN is the environment. Based on the proposed loss recovery model, the agent selects an action and observes transmission state at each time step. The action selection at a step can be defined as a policy \(\pi\) which is a conditional probability distribution \(\pi(a|s)= {\text{Pr}}[A_t=a|S_t=s]\). To estimate the value of an action under the policy \(\pi\), we can use the state-action value function [33]

 

\(Q_{\pi}(s, a)=E_{\pi}\left[\sum_{k=0}^{\infty} \gamma^{k} r_{t+k+1} | A_{t}=a, S_{t}=s\right]\)       (3)

 

According to Bellman equations, the goal of our proposed algorithm is to find the optimal state-action value function [33]

 

\(Q^{*}(s, a)=r_{s}^{a}+\gamma \sum_{s^{\prime} \in S} p_{s s \prime}^{a} \max _{a^{\prime} }q^{*}\left(s^{\prime}, a^{\prime}\right)\)       (4)

 

Where \(r_s^a\) is immediate reward acquired by the agent taking action a in state \(s.p_{ss'}^a\) is the state transition probability from current state s to next state sꞌ by performing action a.

As agent, the mobile node cannot directly obtain the state transition probability. Therefore, the proposed DNQ-FEC control algorithm uses a deep neural network to estimate the optimal state-action value function as \(Q\left(s_{t}, a_{t} ; \theta\right) \approx Q^{*}(s, a) . \theta\) is the weights of the deep neural network. At each step, the mobile node uses ϵ-greedy strategy to choose an action, where a random action is selected with probability ϵ and the action having maximum state-action value is selected with probability 1- ϵ. After performing the action, the mobile node can obtain the immediate reward and the next state s^ꞌ. To record the status of each step, current state, next state, action, and immediate reward are stored in a replay buffer B. A loss function is defined to evaluate the deep neural network estimation

 

\(L(\theta)=\left[r+\gamma \max _{a} Q\left(s_{t+1}, a ; \theta\right)-Q\left(S_{t}, A_{t} ; \theta\right)\right]^{2}\)       (5)

 

The deep neural network is trained to minimize the loss function by using stochastic gradient descent optimization

 

\(\theta_{t+1}=\theta_{t}+\alpha\left[r_{t+1}+\gamma \max _{a} Q\left(S_{t+1}, a ; \theta_{t}\right)-Q\left(S_{t}, A_{t} ; \theta_{t}\right)\right] \nabla_{\theta_{t}} Q\left(S_{t}, A_{t} ; \theta_{t}\right)\)       (6)

 

Where \(a\) is the learning rate and the training dataset is randomly drawn from the replay buffer B. The detail of our proposed DNQ-FEC control algorithm is presented in Algorithm 1.

Algorithm 1 DNQ-FEC Control Algorithm

 

3.4 Content Caching Detection based Adaptive Retransmission Algorithm

Due to the delay constraint, existing approaches mainly use FEC to overcome packet loss for wireless video streaming. The deployment of edge cloud platform reduces the latency and it enables recovering the lost video packets by retransmission. However, there are huge amount of videos, the edge cloud cannot guarantee to cache all requested videos. In addition, the c cache replacement algorithm can cause video data partially stored at the edge cloud. ICN protocols employ receiver driven transmission model, the end-host node needs to fast detect content caching at the edge cloud and set the PIT entry timer accordingly. We use TCP Retransmission Timeout (RTO) algorithm [34] to calculate the LifeTime as \(\text { LifeTime }=\mu_{R T T}+4 \sigma_{R T T}\), where \(\mu_{R T T}\) is smoothed RTT sample mean and \(\sigma_{R T T}\) is smoothed RTT sample variation. If video segments are cached at the edge cloud, then the LifeTime converges to the transmission latency between the edge cloud and end-host node. On the contrary, if certain video segments are not cached, it means that the video segments need to be retrieved from farther distance nodes than the edge cloud. The converged LifeTime underestimates the transmission latency and causes PIT entry timeout.

To identify the timeout caused by cache missing or packet loss, we separately observe the outcomes of video and FEC Interest packets. The FEC packets are only generated at the edge cloud, therefore the timeout of the FEC Interest packet is incurred entirely by packet loss. The average packet loss rate p can be obtained through random variable Y. In addition, we can define two mutually exclusive hypothesis for causes of video Interest packet timeout, where H₀ is packet loss, and H₁ is cache missing. With observation of consecutive video and FEC Interest packets, the likelihood ratio under hypothesis H₀ is defined

 

\(\lambda\left(H_{0}\right)=\frac{\left(\begin{array}{l}{n} \\{k}\end{array}\right) p^{k}(1-p)^{n-k}}{\left(\begin{array}{c}{m} \\{j}\end{array}\right) p^{j}(1-p)^{m-j}}\)       (7)

 

where n and m are the observation windows for video and FEC Interest packets. k and j are the number of timeout for video and FEC respectively.

If the video segments are not cached at the edge cloud and the LifeTime is not enough for transmission latency, the video Interest packets will continuously suffer timeouts. Therefore, the likelihood ratio \(\lambda\left(H_{0}\right)\) is decreased. When \(\lambda\left(H_{0}\right)\) is less than threshold ω and reset the LifeTime. The content caching detection and adaptive retransmission algorithm is further explained in Algorithm 2.

Algorithm 2 Content Caching Detection and Adaptive Retransmission Algorithm

 

4. Experimental Results and Analysis

To evaluate the efficiency of our DL-LRM mechanism, comparing to the Adaptive Loss Protection scheme (ALP-CCN) proposed in [19] we have conducted simulations in the Network Simulator 3. The experimental network topology follows the structure shown in Fig. 2. Transmission path between the video server and the video clients consists of wired and wireless links. The base station is associated the edge cloud platform. The edge cloud cache the video content and use video data to generate FEC packets. Simulation parameters are shown in Table 1.

 

Table 1. Simulation Parameters

The experiments are conducted with two stages. The first stage is to train the deep neural network which is used in our proposed deep Q-Learning based FEC control algorithm. The second stage is to evaluate the performance of the algorithms. Fig. 3 shows the values of the loss function during the training. At the beginning of iterations, the value of each action is randomly assigned. The agent needs to try different actions and estimate their values. Therefore, the variation of the loss function is high. As the agent exploits the action-state space, the value of the loss function is gradually decreased and then stabilized.

 

Fig. 3. Values of the Loss Function in Training Stage

As explained in Section 3.2, fast detection of cache missing is crucial to recover packet loss by using a retransmission approach. Fig. 4 depicts the likelihood ratio testing at different packet loss rates.

 

Fig. 4. Likelihood Ratio Testing under Different Packet Loss Rates

To simulate the cache missing, we intentionally store video data packets from index 1 to 639 in the edge cloud and remove subsequent video packets. The video data packets stored in the edge cloud cause decreasing of the LifeTime. Starting from packet index 640, the video data packets are retrieved from further nodes. The decreased LifeTime is less than the required RTT, and consecutive video PIT entries will timeout. Whereas the FEC packets are generated at the edge cloud, the timeout of FEC PIT entries is only caused by FEC packet loss. As shown in Fig. 4, the likelihood ratios are rapidly decreased under different packet loss rates.

 

Fig. 5. Video Quality Evaluation at 3% Packet Loss Rate

We also conduct experiments to compare our proposed mechanism with ALP-CCN. Although ALP-CCN adopts FEC with retransmission for loss recovery, it does not utilize the resources of the edge cloud. FEC packets are produced by the video server and transmitted through the core network, backhaul, and wireless access network to the video clients. With the increasing number of video content and users, the redundant FEC packets consume huge portion of the bandwidth. When the packet loss rate is low, the ALP-CCN can recover a majority of lost packets, and the achieved video quality is presented in Fig. 5.

 

Fig. 6. Video Quality Evaluation at 7% Packet Loss Rate

With increasing packet loss rate, the timeout of PIT entries frequently occurs. The proposed DL-LRM is able to distinguish packet loss from cache missing and efficiently adjust theretransmission timer. The achieved video quality is higher than ALP-CCN as depicted in Fig. 6 and Fig. 7.

 

Fig. 7. Video Quality Evaluation at 15% Packet Loss Rate

 

5. Conclusion

In this paper, we propose an edge-assistant loss recovery mechanism for video mobile edge computing based information-centric networks. The proposed mechanism consists of two algorithms which are deep Q-learning based FEC control algorithm and content caching detection based adaptive retransmission algorithm. A system model is built to describe the dynamic behaviors of video and FEC packet transmission. Based on the system mode, we design a deep reinforcement learning algorithm to tune the transmission rate of redundant FEC packets. The computing and storage resources provided by the edge cloud enable recovery of packet loss through retransmission. To further utilize the advantages of the edge cloud, we propose a fast retransmission algorithm with content caching detection capability. For the future work, we plan to apply the proposed mechanism to information-centric ad hoc network environment.